Thursday, May 28, 2015

Building an ethereum ÐApp, part III

What is ethereum and ÐApps? Check here  or search
This is part III of a series. Part I

Diving into the code

My simple proof-of-concept app can be seen at and is based on the meteor-dapp-boilerplate project. The smart contract is called Roboth and is deployed on the (currently testing) blockchain, registered with the Global Registrar under the same name. However, I'm still working on it so be prepared to encounter a broken, invalid or a stupid deployment at any time.

Thoughts on deploying beta contract versions

Now this is clearly not a best practice to push stupid code right into the public production environment. I could register the work-in-progress update with the Registrar under a different name such as "Roboth.RC-1" and config my JS frontend to interface with this instance. Alternatively, I could run geth (the ethereum client) on a private testnet using the command line switch

geth --networkid <random number here> --maxpeers 0

or by disconnecting my wifi. It would also require me to clean my blockchain database because I would be starting from scratch effectively. In this way, I could mine all ether by myself and thus have enough for funding any experiments.

Simple Python compile & deploy script

If you prefer your cozy text editor over cool web based development environments, you may find my Python script for compilation and deployment mildly useful. It's included right there in the Roboth.web3 repository as tools/, for free without any hidden costs.

It can handle the following tasks:
  • compile contract code on your geth node (I'm using Windows and don't have a solc binary)
  • deploy compiled contract
  • register the newly deployed contract's address with the Registrar
  • remember compiled code, ABI and address so you can go back and use any earlier-deployed version in case you forgot some semi-important data there (you don't have any really-important data because otherwise you'd be using some more serious and stable software)
  • save the new ABI as JSON to a JS file automatically loaded by Meteor
  • invoke some methods of the contract after deployment so you are not testing with an empty database (must be customised for your particular contract)
  • use hard-coded file paths so you know where to put your files by reading source code (ehm)
Currently it cannot do:
To use it, you'll need to modify the code a bit, edit the geth RPC address where EthRpc is instantiated, edit your primary account in prim_acc and possibly also the contract name variable con_name. When running, current working directory must be tools (that's where the script is located). The tool currently doesn't accept commandline arguments, it must be configured by changing the code at the end of the file.

It also has some dependencies, this one and this one too.

Working with your contract from the JS app

By now you may already be rather familiar with the incantation that takes your contract's binary ABI and its blockchain address and creates a proxy object to call it. It looks like this

this. RegistrarABI = [{"constant":true,"inputs":[{"name":"_owner","........
this. RegistrarAddr = "0xc6d9d2cd449a754c494264e1809c50e34d64562b";

this. RegistrarAPI = web3.eth.contract(this.RegistrarABI);

this. Registrar =;

This is required because even though we write the contract code in Solidity, it's compiled into EVM bytecode and even though we use functions, arrays and mappings, these have a different representation on the blockchain (which is also different from linear memory layout we are used with RAM). The JSON RPC we are using is operating at the low level and it doesn't really know how to call Solidity functions. But the web3.js library knows how to call it, assuming you provide the ABI description that fell out of the solidity compiler.

So in this code snippet, there's a hardcoded ABI for the official testnet registrar contract that I stole directly from geth source code, its official testnet address which I also stole from the same place. Next, the RegistrarAPI creates a class as you know it from OOP languages (if you are coming from C++ or Java, you may not believe that a single function call can create a class but yeah, dynamic languages can do that). On the last line, we instantiate this class using its static method at() and the instance will communicate with the contract on the given blockchain address.

The same procedure would be used for our own contract except that its ABI is automatically generated by the Python script and included by Meteor from client/lib/compatibility/Roboth.abi.js because it's under rapid development and thus changing all the time. Furthermore, the address is fetched from the Registrar where it's stored by the same script on each deployment. See here for yourself.

Once you have a proxy instance, you can call methods and send transactions almost the same way as in regular OOP languages. These are the 2 ways to invoke a method and it's explained in the Frontier Guide.

The simplest way ever to store a growing mapping in Solidity

Assigning some data to an user or an address in Solidity is quite easy, just use the mapping type:

mapping (address => MyData) mydatas;

What happens, however, when you want to iterate over the keys or values to display it in your app? This is not currently supported and I believe wouldn't be so easy to implement because the data layout is not linear. A simple solution is to add an integer index

mapping (uint => address) users;
uint next_user_ix;

Now we can iterate from 0 to next_user_ix and get all users in the range. Of course this requires that you maintain the index manually, adding to it each time a value is added to the original mapping. This approach is very simple but it doesn't really work well when you also need to remove values. You can see the forum post on this problem for other people's ideas.

Ethereum values data types

I recommend always storing account balances in wei as Strings or BigNumbers. Javascript doesn't handle large integers correctly and wei balances are always going to be pretty large. Furthermore, given the number of units or denominations for ether, mixing them up in the code is a really big danger. The only way to stay sane is to stick with wei, just like the JSON RPC and only convert to human-friendly in the templates (using the toEth template helper).

Similarly with addresses, they come as hex string and should stay in that format

Reacting to data from blockchain

Meteor has a neat functionality that enables auto-refreshing your HTML DOM when source data changes. It's called being reactive™. We can use this function to some extent but keep in mind that operations on the blockchain are not instant (and also not immediately reliable until all small forks are abandoned).

The most reliable way to observe changes in the blockchain is to use Solidity events and install filters from the RPC. However, if you don't have that for whatever reason, you can just keep polling every 6 seconds or so.

The class BlockchainTracker is a simple wrapper that will fire an update on its ReactiveVar when the latest block number changes. This can be observed in an autorun function to trigger a refresh from the blockchain. See UserDataManager for an example of a dataset that needs to be updated when a new item gets added. This simple solution doesn't handle updates from other users and it may miss changes that appear 2 blocks later.


The app is still very much in development with many rough edges but I hope people starting out with ÐApps may find these notes useful.

Friday, May 22, 2015

Building an ethereum ÐApp, pt. II

Hiccups on the way to ÐApping

What is ethereum and ÐApps? Check here  or search
This is part I of a series. Part I

Not using enough gas for transactions

You know it - you send a transaction, scratch your head but nothing happens. You wait, see blocks being crafted but your transaction is just sitting there, forgotten. When you check its status using

// 0xTRANSACTIONID is the return value from eth.sendTransaction
// or you can see it in the verbose logs of geth

you get either 0 or an error. 

Make sure you are using enough gas for the operation. Each transaction in ethereum can be one of the following 3 types
  1. just value transfer (sending eth to your friend)
  2. invoke a contract (possibly with value transfer)
  3. create a contract
And each requires a different amount of gas. Excess gas is refunded so you can beef it up easily. For example to deploy my contract, I would use this call:

eth.sendTransaction({from: eth.accounts[0], data=code, gas=1800000})

Not enabling CORS for HTML5 apps

If you decided to access the geth client from a JS+HTML5 app, you may find that the web3.js module is unable to connect because of the Cross-Origin Resource Sharing restrictions in the browser. You can see it in the F12 Developer Tools console. To fix this, make sure you have launched your geth instance with the correct arguments. If your JS app is served from http://localhost:3000, it would look like this:

geth --rpc --rpcaddr="localhost" --rpcport="8545" --rpccorsdomain=" http://localhost:3000"

The --rpccorsdomain argument is key, it allows these origins to access the RPC. Note that you can specify more than one domain, just separate with spaces. This commandline will allow you to run the Cosmo web app at with your local geth client.

Note that if you try to invoke the HTTP RPC requests manually, you won't get the Access-Control-Allow-Origin: header unless you add the Origin: header first.

Incorrect compilation or construction

When you pack your little contract's lunch and send it off to the cloud, it may fail to stick even though you gave him enough gas to fly all the way to the cloud and the transaction was processed. But when you execute


you get '0x' nada nothing. It will be useful to know that the EVM bytecode that you send as data is executed and the result will be the actual contract code living on the blockchain. This is how the constructors work.

If your code is corrupt or the constructor encounters a problem, you may end up in this situation. In my case, I incorrectly copied the hex contract output from the compiler.

Meteor: global variables

I tried to instantiate a contract client in my Meteor app in a template.js file like this:

RegistrarABI = [{"constant":true,"inp.....snip RegistrarAddr = "0xc6d9d2cd449a754c494264e1809c50e34d64562b"; RegistrarAPI = web3.eth.contract(RegistrarABI); Registrar =;

but alas, these variables were not available in my helpers or elsewhere.

Turns out that Meteor executes template JS code in a different context and so these variables were not global. My quick&dirty solution was to attach them to the window which is global but a better approach is clearly putting that into the client/compatibility folder which is designated for "outsiders".

ADDED: Development FAQ

You can join the go-ethereum gitter channel and search, many questions have been asked there already and there are some examples / clarifications too. Just remember to not be an ass and try to search a bit first before distracting the devs.

Friday, May 15, 2015

Building an ethereum ÐApp, pt. I

What is ethereum and ÐApps? Check here  or search
This is part I of a series. Part II
This article is from May 2015, check the update in Part IV

This is a collection of notes as I was going through writing a simple proof of concept ÐApp, hopefully it would prove useful to others and reduce their bleeding when working with such cutting-edge technology. I hope to address also some practical concerns which are beyond the scope of other basic tutorials.

I first met the ethereum project on a meetup in Hong Kong in summer 2014 where Vitalik Buterin presented the project himself. Since then, I've been watching it and growing more interested and after realising its potential (through Vitalik's posts on I got so excited that I went ahead and started writing my own ÐApp, like a true hacker nerd.

ÐApp components, structure

The most important part is, of course, the ethereum client (currently go-ethereum or geth). It is also called a node because it connects with other nodes to form the network (nodes usually also run mining) and you may also think of it as a wallet (in the bitcoin sense) because it keeps your private key and allows you to send transactions. As such, users of your ÐApp will either need to run their own ethereum client or use a web based service (a parallel to but that means increased centralisation and having to trust that service.

Another part of any app is the GUI. That's something you'll be building yourself. You can go ahead and use any of the old boring GUI frameworks such as Qt, HTML5, Android as long as you know how to connect to the wallet of your user. The connection happens over HTTP JSON-RPC (documentation here) which means that even an JS/HTML5 GUI served over the web can still connect to a wallet on the localhost.

Since storage and processing on ethereum blockchain is not so cheap, you may also want to run your centralised server in the old fashioned way, such as Node.js on Amazon EC2 or a Haskell server on your Commodore 64 in your grandma's basement (that would be slower than the ethereum blockchain actually, but equal in coolness factor). This server would handle data that doesn't need to be protected by the blockchain. Remember, the point of blockchain is to have a global consensus on sensitive data (such as people's account balances, domain name registrations) and making sure they are not modified behind anyone's back. Other, more trivial or sizeable data for your app, however, can be stored outside the blockchain, for example uploaded files / pictures / videos. Your server may need to run the ethereum client too, to have access to the latest blockchain state.

Installing and running geth

There's not much to say here, just follow the homepage I recommend just downloading the binaries, they are built automatically for Windows, OSX, Ubuntu. Currently, go ahead with the develop branch. Don't bother with the Mist user interface, geth is all you need (also love). I also recommend going through the Frontier Guide that is being collected at and trying out the examples to understand more.

If, upon starting geth, you can't connect to any peers, try to start with 

geth --vmodule=udp=6,server=6,downloader=6 console

to get extended logging for the network. Make sure your computer clock is correct. Note that the message about no UPnP device found means that it couldn't setup port forwarding on your router using UPnP automatically. No big deal.

Getting a Meteor app skeleton

For this project GUI+centralised server, I chose the Meteor webapp client and server framework because it's new, hip, cool and everybody else seems to be using it too. Hopefully it'll make me look cool too. If you've never heard about it, let me summarize it as a batteries-included, everything-prepared framework that bundles Node.js, MongoDB, reactive (autoupdating) templating engine and other tools to make building and deploying webapps really easy. Both server and client code are written in Javascript. You may want to go through the tutorial to get some idea on how things are working there. 

I started by installing meteor from the homepage at and then cloning this useful repo as the basic template. Meteor is by default running on port 3000. Then, start geth with JSON RPC enabled (default port is 8545) and allow CORS so that your Meteor app can access it from the browser:

geth --rpc --rpccorsdomain "http://localhost:3000" console 2> geth_stderr.txt

Note: do NOT use --rpcaddr "" or you'll lose money. Also, enable firewall to prevent access to your node from the outside.

To see log messages, watch the file geth_stderr.txt, ideally using tail -f geth_stderr.txt. You'll still need to interact with the client in the JS console it provides, don't forget to check out the documentation.

I can already feel your head exploding from the overflow of information in this and the linked articles. Let's wrap it now, have a good sleep and next continue with some actual code, perhaps even with some troubleshooting tips (for free!). I'm planning to put my little ÐApp on github as well, sometime very soon.