Things we don't have in Europe, part II.

With over 3 years spent in Hong Kong, I can share some more minor differences compared to Europe or Czech Republic that I've noticed. I've also noticed that while some things are different, other things are utterly same the everywhere: the abundance of lazy or stupid people.

Note: people seemed to like the Part I as well.

Anti-pandemic measures and crowd control

How do you know you're in Asia? Well just look around you and if there are more than 10 people in your 1m x 1m personal space then you're probably in an Asian city. A high population density increases the damage from any infectious disease and Hong Kong has learned as much when SARS hit. So now they're trying to curb the spreading of diseases by disinfecting lift buttons, escalator handles and door handles multiple times a day. Or that's what they claim, anyway. Furthermore, posters in public places are asking people to wash hands properly and refrain from spitting.

They also have a lot of experience in crowd control. On Halloween and other important party days, the entire bar street is closed and only a limited stream of people can get in. And even then the place is absolutely packed.

Unusual names

Children in HK are asked to choose an English name for themselves in school. That name is then used more often than the original Chinese name and it really is much easier for foreigners to remember because learning the correct pronunciation of a Chinese name can take a week (in my case). People from the mainland often don't choose an English name so I'm having a harder time with their name.

Anyway, I've found that HKers are much less conservative in choosing names than we are in the West. It is taboo in EU or US to pick a name outside of a pre-defined set of names. Not here. Using the name of a city or a proper noun is possible. I've even heard stories that some guy picked the name "Chocolate Milk". I'm not sure if these people realize it'll disadvantage them in dealing with Westerners because for us, such names sound silly and it ruins the first impression. But it makes one also realize how many arbitrary rules does our own culture impose.

I admit that choosing Never Wong as your name is just pure genius.

Octopus card

I love the Octopus card. Similar ^[1] to the UK Oyster card, it stores value and while primarily used for public transport, you can use it in many other places such as restaurant, convenience stores, vending machines, ferries and even as ID for building entrance. Payment is instant and refilling stored value is possible almost everywhere. If you're coming to HK for more than 3 days, don't even think about using one-off subway tickets, just get the Octopus card. Thanks to this, you can almost get rid of those annoyingly heavy coins.

Dining culture

The #1 pastime in HK must be ... eating. Hong Kong may not have as muny arty shows and culture compared to Paris or New York but what you can do every evening is trying a new restaurant. With all of the world's cousine available in thousands of restaurants around the city, there's always something new to explore with your tastebuds. The way to socialize with your buddies is not getting a beer but rather going out for a dinner. And after the dinner you may continue to a dessert shop where you'll get some Chinese style, fruit, tofu and jelly based desserts. Who cares that eating sweets when you're already pretty full at 10pm may not be the healthiest thing. The naturally slim Asians are not worried.

When not eating in a fancy restaurant or when you're at home, you may find that the table has a big sheet of plastic bag instead of the table cloth. It looks extremely ugly but it saves the work of cleaning the mess that is inevitably going to hit the table. And after all, the company of your friends matters more than some fancy table cloths. Your Chinese friends will probably offer you a paper tissue when they reach to get one for themselves. Tip: bring 2 packages if you're going to a spicy restaurant.

Chinese tea is an interesting topic. Don't think that everybody around here is an expert on tea and can explain the difference between various Oolong teas at length. I seem to be actually more knowledgeable in this topic than a typical local person. On the other hand, 7-11 convenience stores all have plastic bottles of cold tea, with or without sugar. Not Nestea but rather actual tea. And Chinese style restaurants serve tea as a basic free service. But even then, HK style milk tea and HK style lemon tea are still the most common drinks to consume with your meal.

Some local restaurants offer "Western food". That almost always means one kind of tomato-based soup, offered without fail in the same form by all of them. Apparently we Westerners only know one type of soup. Furthermore, if you order potatoes as the side dish, it's almost always going to be 1 small potato which is clearly not enough carbs to get me through the day. I don't understand it, the rice portions are usually pretty big and potatoes are not even expensive.

 

 Language

The native language of Hong Kong is Cantonese. This is actually the language of the entire southern Chinese region but immediately after you cross the border of Hong Kong to Shenzhen, everything switches to Mandarin. If you keep going North, you'll get back to Cantonese. This anomaly is caused by the numerous immigrants to the industrial megacity of Shenzhen. And this is probably not going to last forever because the Chinese government is actively trying to eradicate Cantonese so that they have a more homogeneous population that is easier to control.

As for English, it is important to HKers to learn English but many still struggle and well written English is hard to come by. On the other hand, if you compare with some European countries where people don't even bother, you have to give HKers some credit for trying.

Often you can spot an English sentence written by a Chinese person not only from the errors but also from the style. Overuse of strong adjectives is very common so typically you can "win fabulous prizes" which are actually just a branded pen and chocolate or download "breathtaking games" such as Pacman or Pong. If you buy a cheap electronic product, you can be already pretty sure it was made in China but for the sake of argument let's say you'd use the language on the box to guess the product's origin. Phrases about "enjoying your life", "enjoying every tap on the device" or "experiencing fabulous digital life" will give you a hint.
 

Freezing air-con 24/7

This one simply can't go unmentioned. The mystery of air-cons everywhere set to kill freeze remains unsolved. Locals, when interrogated, dodge the topic or remain silent. After 3 years here, though, it seems that the culture here dictates that you need to have fresh air flow at all times, otherwise you die. Considering the high humidity in this region I admit this is certainly true to an extent. But locals take it to the extreme and consider even 10 minutes without air-con a threat. Using the fan-only mode is not acceptable either, even in winter: if air is not cooled, it simply cannot be fresh. I wonder when a HKer and a Korean have to sleep in the same room: HKer will die if the fan or air-con is off, the Korean will die when the fan is left on!

What they don't have here

Going to lunch with colleagues and want to split the bill? Bad luck, waiters will usually not do that for you. Have fun giving back change that you don't have. I foresee cryptocurrency payments to be the only way out of this situation ;) wink wink

Their supermarkets are not air-tight like in Czech. That must mean that people usually wouldn't steal from a supermarket here. I cannot imagine such degree of trust in Czech and it makes me sad.

Honestly, recycling and environment conservation both appear to be rather alien ideas around here. Restaurants overflow with piles of take-away boxes, you get a plastic bag for everything, vegetables and fruit in supermarkets is already pre-wrapped in plastic, sometimes in 2 layers! Also, it's really funny to never, ever, see squished plastic bottles in recycling collection points. It could save a lot of space and almost everyone in Czech does it. Here, the idea never appeared. Makes me wonder what other useful ideas are completely missing in some parts of the world.

Everybody in cold countries knows how to walk on snow or ice. You just need to move your weight exactly over your feet before relying on that foot. Children also know how to slide on ice and can go all the way to school just sliding on the icy pavement. In HK on the other hand, even a little wet tiled floor is a serious threat. Warning triangles are deployed, floor driers are set in operation. I know it's to protect building management from being sued but it's just ridiculous. There is ice in HK only once per 35 years and when it comes, it's a little embarrassing:

[1] Fixed incorrect claim, thx Alessio

The fall of Couchsurfing and the need for DApps

Abstract: How Ethereum DApps can be applied even outside finances and how communities can benefit from technologies of the future.
 

Couchsurfing (or at least the idea of it) used to be a community of people who would welcome each other in various places around the world, show them the local culture, recommend best local places and since many people have a spare couch at home, why not let the traveller crash there for a night or two.

Of course, not everybody knows everyone so people would leave references for each other after having spent some time together. The reference would include information such as how long you've known the person, whether your experience of them was positive or negative and of course a paragraph or two. The site emphasized that references are the fundamental tool to keep people secure.

People have accumulated lots of positive references over the years by letting backpacking travelers crash on their couch and showing them around. People knew they could trust a bunch of good references. This good track record would make it easier to find a couch when they go traveling (which for a typical CSer is often). So we could say that having lots of good references on CS has some value and is not that easy to build.

Of course, nothing lasts forever and CS, no longer a non-profit, is moving away from the original idea. The home page https://www.couchsurfing.com is saying something about staying somewhere for free. No mention of cultural exchange, making new friends. The last step they've taken is removing metadata from references, leaving only the verbal description. So you can no longer see at a glance if it's positive or negative. Want to know if that stranger is trustworthy enough to let them stay? Sure, just read through all of their references!

The reason CS is doing this is because they are now owned by the same group which owns AirBnB and other paid accommodation services. "Free accommodation" using couchsurfing is not a good alternative to their paid services so they need to get rid of it. And delete years worth of good references.

We could say the problem is in the amorality of new owners of CS destroying the community and something that the people have been building. Maybe. The CS website owns all the data that users have entered there and it has control over what shows on the website. They are able to do any move that's bad for the community and the users are powerless even though they create the site's content and actually the entire value of the website.

It doesn't have to be like that. Scientists are working Computer nerds are working on a new Web, one where users are in control because they own the site and its data collectively. No longer having to trust one person or company that could become evil or simply sell out to some greedy profit-seeker. In this way, changes that wouldn't benefit the community could not be made, data could not be deleted.

Those websites are called DApps (from distributed apps, because they are owned by multiple people) and they are slowly becoming possible thanks to technologies such as Ethereum, Bitcoin, IPFS and others. They can be used to build financial services, true democratic communities or what I've described above. Sounds interesting? Get involved! Even if you're not technical at all, an ordinary user can help a lot in this early stage. Install the apps. Start playing around, get involved in discussion, many ideas still need figuring out. Tell other people about the idea.

And what is the fate of Couchsurfing? Multiple people are thinking about starting a new site. Starting from scratch, without the data that has been built in CS because there's no way to transfer the data to a new place (another problem of the old website system). Building a new CS as an DApp would not be easy at this moment because there's very little support available for DApp developers, it is a very unexplored area and needs original work to build anything, unlike traditional websites. But that will change over time.

The Demons of MiraCL

Once upon a time, an old and experienced warrior set out on a grand quest towards assembling The Scroll of Distributed Identity Based Encryption. In his home village, he had equipped himself with his trusted weapons, said a prayer for his fallen predecessors and stepped out of the village.

Go ahead and listen to some background music while reading:

First he had to fight his way across the Plains of Emptiness. Whispering ancient mantras, he wielded his weapons to prevent the Ghosts of Void from taking his soul away from this world all the while as he was making his way towards the castle named Prototype. Six days he fought and on the seventh day he rested as the Test Suite Stream ran softly under the windows of the castle and all was well.

That was, however, not the end of the quest. After consulting the the Gods he realized that he'll have to part with his old companion. For the quest ahead of him was foretold to be completed with a different tool, a weapon so powerful and ancient, that only silent whispers about the miracles it can perform roamed the world. The warrior eagerly set out on the next part of the quest, paying little heed to searching the lore for stories about his new tool.

Alas, that put him at peril, right at the gate of his castle, where he needed to defeat a many-headed beast but he found the MiraCL could only deal with a fixed size numbers and any larger number, though named Big, would end in the weapon breaking completely and dumping filthy core dumps at him. After trying a few more times, he found he could get a warning from MiraCL that the numbers were too big but not always. He was confused. How could a number be "too big" for such a mighty tool?  Courageously, he peered inside MiraCL to find the answer but found nothing but dismay.

For you see, MiraCL is an ancient weapon, forged by the Elders long before humans walked the Earth and thus is not for a mere mortal to understand. In these times, source code comments were pure blasphemy and also memory was limited and thus The Elders decreed that only 3-letter variable names shall be used. Optical illusions, out of grasp of mere men, were abundant. Different objects with the same name appeared out of nothing and weird macros obscured his vision. Wearily, he drove the beast away and set down to meditate.

In his meditations, he saw the Lore of MiraCL in front of his eyes and quickly understood that should have been the first place to look. For MiraCL was a thing out of this world and could not be wielded by mere mortals without peril.

This also gave him the knowledge on how to combine MiraCL and the mighty Address Sanitizer, his indispensable light in the Darkness of Memory Access, guiding his step away from the gaping chasms of Segfault. To forge MiraCL together with Address Sanitizer, there are two options. One is to abolish the Assembly Script and use only pure C for summoning. To do that, add to config.h this option:

#define MR_NOASM

and omit mrmuldv.c from your build spell.

Another way to forge MiraCL and the mighty Asan allows the use of Assembly but requires the 64b system. For that, invoke the following demonic spell inside the guts of Assembly implementation file, mrmuldv.g64:

#if defined(__clang__) || defined (__GNUC__)
# define ATTRIBUTE_NO_SANITIZE_ADDRESS __attribute__((no_sanitize_address))
#else
# define ATTRIBUTE_NO_SANITIZE_ADDRESS
#endif

and bless every function in that file with that attribute.

However, this was not the end of the quest, far from it. Our warrior went back to the Prototype castle and started replacing the OpenSSL in construction with MiraCL as was foretold. He worked hard, day and night, painstakingly looking at each brick, each beam. Once everything was in place, he rested and looked at the Test Suite Stream. But what horror he saw there! The stream was not crystal clear water as it once was but a stream of pure blood, splashing around, staining the walls. What once was in harmony with OpenSSL, now was in shatters. An ancient curse in the heart of MiraCL perhaps?

The warrior had no other choice but to fight any curse because it was his destiny. And so he toiled on. He separated the stream into smaller parts, let it go through a part of the castle only. He let it test one part at a time to see the result of each. After much work, he could see which rooms of the castle left each of the little trickles crystal clear and which turned it nasty. Divide and conquer, understand parts separately and know when they work. Then you can rely on them and use them in fight to achieve correctness of larger components. He still remembered the teachings of his Temple well.

But the curse was not so simple. He saw many streams running clear but once he put them together, suddenly all individual streams turned to blood! He could go through everything, seeing nothing but harmony but on the way back from the last room, everything would be in ruin again. The warrior could not believe his eyes. How was this possible? He inspected everything in detail again until he found it. He wept for hours, for with MiraCL he has awakened an evil ancient curse, thought to be long gone from our world. The Curse of Shifting Global State. Indeed, somewhere, somehow, MiraCL would shift and make all his work worthless.

But he persisted, as was foretold. He knew the curse had to be stopped. Covered in bloody mud, he rose again, the flaming sword of Divide & Conquer in his hands. Unresting, he slashed and slashed through. He found the time and place of the shift. He watched the shift occur. It was a call to powmod() which, behind his back, changed everything in the castle into ruin. The warrior couldn't believe what was happening. The powmod() function looked like a little harmless bird at first. Who would imagine it causing such a havoc? And yet, it managed to trash the whole building. Such was the Curse of Shifting Global State.

He waded through the misty Source of MiraCL, doing his best to decipher hidden meaning and ignoring any illusions of obfuscated C. There he saw that powmod() partners with prepare_monty() in its evil deed of changing a global parameter. That parameter was, however, crucial for the representation of his elliptic curves. When changed, the curves would collapse into singularity.


powmod() assumes that numbers use a Montgomery n-residue representation with a constant modulus. That was the case for objects of type G2 (an elliptic curve point) that are used in our app. Calling powmod() with a different modulus will change the global Montgomery settings and quietly break any existing instance of G2.
Cleansing himself from the effects of this sin required much meditation, but eventually clarity descended on him and he saw that MiraCL can only deal with a single modulus in the whole computation. Alas, he needed to work with different moduli and that was why The Elders sent a curse upon him. This curse was too great; the warrior had no choice but to masterfully avoid it. He locked powmod() in a chest and buried it meters underground in a stone grave. After finishing this hard labour, he seeked for a replacement. Destiny was generous with him for another function, power() turned out to be safe and powerful enough for his purpose.

Our warrior felt much lighter once this burden has been taken off his chest. Walking through the castle, with crystal water returning back to its stream, he felt in a bliss. And then he stumbled and fell down a few broken stairs. These stairs were also coming from MiraCL, they were the Big.operator+=(). Are they cursed too? They caused him to crash, only thanks to mighty Asan did he didn't suffer much pain. There were many other stairs in the castle, how come only these were so treacherous?

This time he chose to unleash the Watchdogs of LLDB on this issue and they led him right to the tapestry on the wall that was not present anywhere else. The tapestry was named otstr()
 and it displayed many numbers in hexadecimal, unfortunately it also unleashed the potential to crash in the stairs. It was a very dangerous overlook from The Elders.

It has turned out that a hashing function in code for IBE was implemented in a careless way, causing overflow of the Big type. It relied on the fact that such overflows are normally detected and avoided. Unfortunately this detection could be turned off as was done in otstr(). The otstr() function never enabled overflow checking again, an obvious bug. Watchpoints in LLDB were able to help detect places where mip->check was changed.

All of this made the warrior suspect the MiraCL, but wise as he was, he remembered similar perils with his other trusted tools as well, mastering was never an easy task.

No, I wasn't high when writing this, just a little frustrated and this felt like fun. Maybe we should write all programming blogs like this ;-)

Compiling openssl with emscripten

a.k.a. the days of 10kB JavaScript are gone.

We are doing some crypto app prototypes and figured that having demos on the web, without having to download or install anything are quite valuable. And despite the issues on the SSL side of OpenSSL, the crypto library is still quite useful. Let's see how to build it into JavaScript using the amazing emscripten.

I'm using openssl v1.0.2a which is commit 3df69d3aefde7671053d4e3c242b228e5d79c83f in the git repository. First I have emscripten prepare my environment for compilation to make sure I'm using the correct compiler, archiver and linker (emcc, ar, ld). I do

emmake bash

or any other shell such as fish. I wasn't able to run emmake ./Configure or emconfigure directly so I just run a new shell. From the shell I can configure openssl as usual:

./Configure -no-asm -no-apps no-ssl2 no-ssl3 no-comp no-hw no-engine no-deprecated shared no-dso --openssldir=built linux-generic32

note that 64b architecture cannot be used. I also did have to modify the generated Makefile a bit.

1. on line 63, delete the path after $(CROSS_COMPILE) so that it looks like this:
   CC= $(CROSS_COMPILE)cc
2. on line 64, remove the -O3 flag just to be sure because not all enscriptem optimizations may be compatible with openssl

after this you're able to build the library using

make

To test, I did build one of the demos:

emcc  demos/sign/sign.c -lcrypto  -o demos/sign/sign.html -Iinclude -L. --preload-file demos/sign@/

The resulting library is almost 4 MB, it may be useful to try and remove some more features. Now it's not really clear if crypto software running in this way is still secure. I know that browser Crypto API + enscriptem ensure that randomness in /dev/urandom is correct but I may need to dig into the debugger to be sure it's really used correctly.

Building an ethereum ÐApp, part IV: The Frontier

What is ethereum and ÐApps? Check here  or search
This is part IV of a series. Part I

Welcome to explore what's behind the Frontier!

The first real release of Ethereum is out and it mostly works! First, let's get out some updates to previous blog posts.

Some updates

• You can now open the JavaScript console using geth attach which will connect to geth you've already started on your machine. But on Windows, this is still not working very well. A fix is underway. See more here.
• You may want to use the eth.contract interface to create and manipulate your contract
• Of course it's always a good idea to keep in sync with the JavaScript API reference!
• eth.sendTransaction() now returns the tx hash. To get the contract address if you've sent some code, use eth.getTransactionReceipt(tx hash).contractAddress

An update on running a private chain

This is the commandline I use for development:

geth.exe --rpc --rpccorsdomain="*" --datadir geth_private --rpcapi "admin,db,eth,debug,miner,net,shh,txpool,personal,web3" --nodiscover --networkid 7938 --genesis private_genesis.json --solc "your/path/to/solc.exe" --unlock 0

and my genesis file is in https://github.com/Quiark/eth-devchain . Actually all you need for a private dev chain is there.

Note that:

• the difficulty is set to 4 so that you can create blocks immediately and even the DAG is tiny
• the command above enables ALL management APIs to the RPC which would be a totally unsafe thing to do on the livenet.
• change your path to solc.exe (can be downloaded with the cpp-ethereum or eth++ package)
• for fake test ether, you can either just mine or edit the genesis file to assign some balance to one account. You just need to have a private and public key for that account in advance. You can create them on the live net first.

Back to coding

I've come to the stage where I need to implement payouts in my Roboth.web3 dapp based on which user has the most upvotes. In a few words, this app lets user post a problem (a job) and ask the crowd to provide solutions. Users up/down vote solutions and after a fixed amount of time, the highest rated solution gets selected and paid the amount initially offered with the problem. There are a number of problems with that, two of them I'm going to discuss.

Timed automatic payouts

Payout to the highest rated user should occur at a certain time, ideally automatically. Ethereum by itself doesn't support auto-triggering function calls. In this case, the solution is simple: let the supposed receiver of the payout ask for it themselves. After the contract verifies he is indeed the correct receiver, it can send out the payment.

To make it even better, our centralized server or the JS application can handle this automatically so that the human does not need to think about it and can instead focus on whatever thing humans like to do. The JS side of the dapp can query our contract if user is eligible for a payout using a const function in the contract - one that only reads data and is free to execute.

I haven't implemented this in my dapp yet, wait for next blog post to see how it turns out.

Finding highest rated solution

Each solution can be up or down voted by any user, much like this happens on StackExchange. That means the top position can change dynamically. When payout time comes, we need to find the top player for that particular job. Depending on what data structure is used, this can be time consuming and time equals gas equals money. If you have all solutions in one list, finding the max is just a linear operation and could be fine if you don't expect too many of them. In my case, solutions for a single job are not located together so to find it, I would have to iterate over all solutions for all jobs which would be very costly. 

The top rated solution can be cached so that it can be retrieved immediately. Since the top solution can come to the top and leave it again when downvoted, we need to use a heap data structure to perform such changes efficiently. A heap can be implemented using a simple array so the lack of pointers in Solidity should not be an issue.
Another factor to weigh is the gas price of storage. Having too many repeated storage slots can be costly. Writing new item to storage is priced at 20k gas, reading is at 5k and deleting that item (by setting it to 0) actually refunds 10k.

Again, implementation is pending so check out my next blog post :)

For reference,

• currently used gas costs,
• storage manipulation costs

Correct SCons variantdir and emitters

I'm using SCons to build my C++ stuff across platforms and as usual, my build config is gradually getting more complex. I always like to have build output in a separate directory, for cleanliness. I use a VariantDir command to do that. The problem is that variant dirs are always a bit tricky to understand and do properly, so here are some notes on how to avoid screwing up.

Use the Node, Luke!

Items in the SCons build tree are represented as Nodes, not only plain file names. In the case of an output into your VariantDir, the node will remember the output path (such as build/file.o) as well as the original source input path (file.o) and for both of these, it also knows the absolute path. These properties are something you'll always want to see when debugging.

print n.abspath

print n.srcnode().abspath

See the section File and Directory Nodes for specific property documentation.

Use the Emitters, Leia!

SCons is a little obsessive and really likes to keep track of everything. It likes to know what files come in and what will fall out. With this information, it can make sure everything is properly rebuilt on any change and it can nicely clean your directory with the -c switch.

If you need to call some external command, it's a good idea to provide this information to SCons so that it knows what will happen. In my build, I need to generate header files for JNI classes using javah. The built-in tool doesn't really work for me because it needs Java compilation first so I ended up writing my own.

The file and class names in Java are tightly coupled, you can pretty much just do 

file = clsname.replace('.', '/') + '.java'

to find the source file for a class. I'm using this fact to make my emitter. I take great care to have the correct .java files listed as the source for the Builder. Having only the directory just doesn't cut it, I have to Glob() in subdirs too. To have a good idea of what's happening, I first debug-print my source and target nodes in the emitter:

def emit_javah(target, source, env):
    print 'emit source', [x.abspath for x in source]
    print 'emit target orig', [x.abspath for x in target]


The emitted target node doesn't need to have an absolute path or contain the VariantDir name, that should be handled by SCons. Just imagine you are building in the same directory and return a relative path.

Building an ethereum ÐApp, part III

What is ethereum and ÐApps? Check here  or search
This is part III of a series. Part I

Diving into the code

My simple proof-of-concept app can be seen at https://github.com/Quiark/Roboth.web3 and is based on the meteor-dapp-boilerplate project. The smart contract is called Roboth and is deployed on the (currently testing) blockchain, registered with the Global Registrar under the same name. However, I'm still working on it so be prepared to encounter a broken, invalid or a stupid deployment at any time.

Thoughts on deploying beta contract versions

Now this is clearly not a best practice to push stupid code right into the public production environment. I could register the work-in-progress update with the Registrar under a different name such as "Roboth.RC-1" and config my JS frontend to interface with this instance. Alternatively, I could run geth (the ethereum client) on a private testnet using the command line switch

geth --networkid <random number here> --maxpeers 0


or by disconnecting my wifi. It would also require me to clean my blockchain database because I would be starting from scratch effectively. In this way, I could mine all ether by myself and thus have enough for funding any experiments.

Simple Python compile & deploy script

If you prefer your cozy text editor over cool web based development environments, you may find my Python script for compilation and deployment mildly useful. It's included right there in the Roboth.web3 repository as tools/contract.py, for free without any hidden costs.

It can handle the following tasks:

• compile contract code on your geth node (I'm using Windows and don't have a solc binary)
• deploy compiled contract
• register the newly deployed contract's address with the Registrar
• remember compiled code, ABI and address so you can go back and use any earlier-deployed version in case you forgot some semi-important data there (you don't have any really-important data because otherwise you'd be using some more serious and stable software)
• save the new ABI as JSON to a JS file automatically loaded by Meteor
• invoke some methods of the contract after deployment so you are not testing with an empty database (must be customised for your particular contract)
• use hard-coded file paths so you know where to put your files by reading source code (ehm)

Currently it cannot do:

• watch files for changes
• report solc compilation errors very precisely
• integrate very well with your cozy text editor
• find the meaning of life

To use it, you'll need to modify the code a bit, edit the geth RPC address where EthRpc is instantiated, edit your primary account in prim_acc and possibly also the contract name variable con_name. When running, current working directory must be tools (that's where the script is located). The tool currently doesn't accept commandline arguments, it must be configured by changing the code at the end of the file.

It also has some dependencies, this one and this one too.

Working with your contract from the JS app

By now you may already be rather familiar with the incantation that takes your contract's binary ABI and its blockchain address and creates a proxy object to call it. It looks like this

this. RegistrarABI = [{"constant":true,"inputs":[{"name":"_owner","........

this. RegistrarAddr = "0xc6d9d2cd449a754c494264e1809c50e34d64562b";

this. RegistrarAPI = web3.eth.contract(this.RegistrarABI);

this. Registrar = this.RegistrarAPI.at(this.RegistrarAddr);

This is required because even though we write the contract code in Solidity, it's compiled into EVM bytecode and even though we use functions, arrays and mappings, these have a different representation on the blockchain (which is also different from linear memory layout we are used with RAM). The JSON RPC we are using is operating at the low level and it doesn't really know how to call Solidity functions. But the web3.js library knows how to call it, assuming you provide the ABI description that fell out of the solidity compiler.

So in this code snippet, there's a hardcoded ABI for the official testnet registrar contract that I stole directly from geth source code, its official testnet address which I also stole from the same place. Next, the RegistrarAPI creates a class as you know it from OOP languages (if you are coming from C++ or Java, you may not believe that a single function call can create a class but yeah, dynamic languages can do that). On the last line, we instantiate this class using its static method at() and the instance will communicate with the contract on the given blockchain address.

The same procedure would be used for our own contract except that its ABI is automatically generated by the Python script and included by Meteor from client/lib/compatibility/Roboth.abi.js because it's under rapid development and thus changing all the time. Furthermore, the address is fetched from the Registrar where it's stored by the same script on each deployment. See here for yourself.

Once you have a proxy instance, you can call methods and send transactions almost the same way as in regular OOP languages. These are the 2 ways to invoke a method and it's explained in the Frontier Guide.

The simplest way ever to store a growing mapping in Solidity

Assigning some data to an user or an address in Solidity is quite easy, just use the mapping type:

mapping (address => MyData) mydatas;

What happens, however, when you want to iterate over the keys or values to display it in your app? This is not currently supported and I believe wouldn't be so easy to implement because the data layout is not linear. A simple solution is to add an integer index

mapping (uint => address) users;
uint next_user_ix;

Now we can iterate from 0 to next_user_ix and get all users in the range. Of course this requires that you maintain the index manually, adding to it each time a value is added to the original mapping. This approach is very simple but it doesn't really work well when you also need to remove values. You can see the forum post on this problem for other people's ideas.

Ethereum values data types

I recommend always storing account balances in wei as Strings or BigNumbers. Javascript doesn't handle large integers correctly and wei balances are always going to be pretty large. Furthermore, given the number of units or denominations for ether, mixing them up in the code is a really big danger. The only way to stay sane is to stick with wei, just like the JSON RPC and only convert to human-friendly in the templates (using the toEth template helper).

Similarly with addresses, they come as hex string and should stay in that format

Reacting to data from blockchain

Meteor has a neat functionality that enables auto-refreshing your HTML DOM when source data changes. It's called being reactive™. We can use this function to some extent but keep in mind that operations on the blockchain are not instant (and also not immediately reliable until all small forks are abandoned).

The most reliable way to observe changes in the blockchain is to use Solidity events and install filters from the RPC. However, if you don't have that for whatever reason, you can just keep polling every 6 seconds or so.

The class BlockchainTracker is a simple wrapper that will fire an update on its ReactiveVar when the latest block number changes. This can be observed in an autorun function to trigger a refresh from the blockchain. See UserDataManager for an example of a dataset that needs to be updated when a new item gets added. This simple solution doesn't handle updates from other users and it may miss changes that appear 2 blocks later.

Conclusion

The app is still very much in development with many rough edges but I hope people starting out with ÐApps may find these notes useful.