In January 2009, Bitcoin was released into the world by its pseudonymous founder, Satoshi Nakamoto. In the ensuing years, this cryptocurrency and its underlying technology, called the blockchain, have gone on a rollercoaster ride that few could have predicted at the time of its deployment. It’s been praised by governments around the world, and people have predicted that “the blockchain” will one day be like “the Internet.” It’s been banned by governments around the world, and people have declared it “adrift” and “dead.”
After years in which discussions focused entirely on Bitcoin, people began to realize the more abstract potential of the blockchain, and “next-generation” platforms such as Ethereum, Steem, and Zcash were launched. More established companies also realized the value in the more abstract properties of the blockchain — resilience, integrity, etc. — and repurposed it for their particular industries to create an even wider class of technologies called distributed ledgers, and to form industrial consortia such as R3 and Hyperledger. These more general distributed ledgers can look, to varying degrees, quite unlike blockchains, and have a somewhat clearer (or at least different) path to adoption given their association with established partners in industry.
Amidst many unknowns, what is increasingly clear is that, even if they might not end up quite like “the Internet,” distributed ledgers — in one form or another — are here to stay. Nevertheless, a long path remains from where we are now to widespread adoption and there are many important decisions to be made that will affect the security and usability of any final product. In what follows, we present the top ten obstacles along this path, and highlight in some cases both the problem and what we as a community can do (and have been doing) to address them. By necessity, many interesting aspects of distributed ledgers, both in terms of problems and solutions, have been omitted, and the focus is largely technical in nature.
10. Usability: why use distributed ledgers?
The problem, in short. What do end users actually want from distributed ledgers, if anything? In other words, distributed ledgers are being discussed as the solution to problems in many industries, but what is it that the full public verifiability (or accountability, immutability, etc.) of distributed ledgers really maps to in terms of what end users want?
9. Governance: who makes the rules?
The problem, in short. The beauty of distributed ledgers is that no one entity gets to control the decisions made by the network; in Bitcoin, e.g., coins are generated or transferred from one party to another only if a majority of the peers in the network agree on the validity of this action. While this process becomes threatened if any one peer becomes too powerful, there is a larger question looming over the operation of these decentralized networks: who gets to decide which actions are valid in the first place? The truth is that all these networks operate according to a defined set of rules, and that “who makes the rules matters at least as much as who enforces them.”
In this process of making the rules, even the most decentralized networks turn out to be heavily centralized, as recent issues in cryptocurrency governance demonstrate. These increasingly common collapses threaten to harm the value of these cryptocurrencies, and reveal the issues associated with ad-hoc forms of governance. Thus, the problem is not just that we don’t know how to govern these technologies, but that — somewhat ironically — we need more transparency around how these structures operate and who is responsible for which aspects of governance.
8. Meaningful comparisons: which is better?
The problem. Bitcoin was the first cryptocurrency to be based on the architecture we now refer to as the blockchain, but it certainly isn’t the last; there are now thousands of alternative cryptocurrencies out there, each with its own unique selling point. Ethereum offers a more expressive scripting language and maintains state, Litecoin allows for faster block creation than Bitcoin, and each new ICO (Initial Coin Offering) promises a shiny feature of its own. Looking beyond blockchains, there are numerous proposals for cryptocurrencies based on consensus protocols other than proof-of-work and proposals in non-currency-related settings, such as Certificate Transparency, R3 Corda, and Hyperledger Fabric, that still fit under the broad umbrella of distributed ledgers.
The natural issue that arises in such an increasingly crowded landscape is how to distinguish between these solutions and pick the one that is best for a given application. Do you need a blockchain, or just a database? Maybe an Excel spreadsheet is sufficient for what you need to do? Related back to Item 10, what even are the properties that you want to satisfy? Even if someone could specify a list of these, it is not still not clear which current platforms support which properties, and to what extent.
Potential and developing solutions. Some recent research has looked at the different parameters chosen by different cryptocurrencies and the effect these parameters have on the security of the system, finding, for example, that the same level of security against so-called “selfish mining” attacks is achieved by 37 blocks in Ethereum as by 6 blocks in Bitcoin (due to the relative stale block rates of the two platforms). In considering distributed ledgers as a whole, recent research explored the connections between the security properties provided by Certificate Transparency and Bitcoin, finding that the tradeoff between the two seemed to be between trust in a distributed set of authorities on the one hand and the need to (inefficiently) broadcast messages and engage in consensus protocols like proof-of-work on the other hand.
7. Key management: how to transact?
The problem. Over the years, many Bitcoin users have reported incidents in which they have lost all their bitcoins because they threw away a hard drive, forgot a password, or didn’t take appropriate measures to secure the wallets on their computers; in the worst publicly reported incident, a user lost 7,500 bitcoins. In another very public example, an Ethereum smart contract called The DAO (short for Decentralized Autonomous Organization) that was designed to act as a collective investment vehicle managed to attract over 150 million USD in what is considered the largest crowdfunding campaign ever. After an attacker exploited a loophole in the code of the smart contract behind The DAO, the community was essentially powerless to do anything except watch them steal all the funds stored inside (until, that is, some of the governing Ethereum developers discussed in Item 9 created and advocated for a hard fork to undo the damage).
While these incidents obviously have serious financial implications for the individuals involved in them, they should in some sense not be particularly surprising to these individuals given the unregulated “Wild West” world of cryptocurrencies. As people are proposing more mainstream uses of cryptocurrencies — e.g., integrating Bitcoin wallets into Linux distributions — that will put these wallets in the hands of less experienced users who may be less prepared for these types of risks, however, this will lead to wider and more serious consequences. We thus need robust solutions that are able to better tolerate both the loss and the theft of keys.
6. Agility: which algorithms do we use?
The problem, in short. In many cryptocurrencies the rules seem to have been handed down from on high, and are not easy to change. Aside from the governance issues raised by these rigid specifications, cryptographic primitives break, and different users might be willing to make different trust assumptions about both the people with whom they transact and the people maintaining the system. There is thus an argument to be made for providing users with multiple options in how they use the system, although of course significant caution must be taken in the case of using different cryptographic primitives (as TLS has demonstrated).
5. Interoperability: how to talk to each other?
The problem, in short. Some people believe that the future will contain one single distributed ledger, but the more likely scenario is that different companies will gravitate toward different ledgers based on their particular requirements. To achieve the much-discussed potential of distributed ledgers to eliminate (or at least significantly open) the existing landscape of proprietary data silos, it is thus essential to achieve some kind of interoperability, or a set of methods that allow these disparate ledgers to talk to each other.
4. Scalability: why store every transaction?
The problem, in short. One aspect of scalability is that, as the system becomes more popular, it should not significantly increase the storage load placed on users of the system, as this deters participation and increases the barrier to entry. For integrity purposes, however, it is important in systems in which we expect full public verifiability (discussed further in Item 1) to not delete entries from the ledger. This results in, e.g., the Bitcoin blockchain requiring currently over 120 GB to store, as it contains every transaction that has ever taken place. The main problem thus lies in how to provide a balance between these two (seemingly contradictory) requirements. In certain applications, it may be sufficiently important to provide full auditability that we cannot help but increase the storage load of participants. Especially in consumer applications, however, a relatively compelling case can be made that is is not actually necessary to store every transaction, as — for example — we are unlikely today to want or need to meaningfully examine someone’s coffee purchases from early 2010. In fact, retaining such information over a long period of time is clearly at odds with the privacy of users (Item 2).
3. Cost-effectiveness: what is the cheapest way?
The problem. People like to complain a lot about Bitcoin and its proof-of-work-based consensus algorithm. They compare the electricity it uses to the electricity used by whole nations, or at least by large power plants. They call it an “environmental disaster.” While many of these arguments are quite overblown, the fact remains that proof-of-work is indeed very expensive, and it would of course be a good thing if the same result could be achieved in a cheaper way.
To this end, there have been a huge number of alternative consensus protocols proposed, and even used in alternative cryptocurrencies. For example, proof-of-stake is a protocol in which, rather than prevent Sybil attacks on the mining process by requiring participants to consume some expensive resource, as is done in proof-of-work, so-called validators prove their “stake” in the system, in the form of an investment they have made that provides an economic disincentive for them to misbehave by forming bad blocks. Peercoin, Nxt, and BlackCoin use a version of proof-of-stake, and Ethereum has a planned transition (i.e., hard fork) to their own version of proof-of-stake, Casper, within the next two years. Intel’s Sawtooth Lake platform uses proof-of-elapsed-time (PoET), which relies on its SGX architecture. One of the most popular Bitcoin-based proposals, Bitcoin-NG, proposes isolating the usage of proof-of-work to elect a periodic “leader,” who can then quickly (i.e., without performing large amounts of computation) bear witness to individual transactions.
Potential and developing solutions. In the distributed ledger research community, this seems to be the issue that is being explored most actively. In addition to the protocols mentioned above, there are dozens more articles with their own proposals, both for the protocols mentioned above (especially proof-of-stake, which seems to be a sort of “holy grail” right now) and for new and increasingly exotic ones.
If one expands outwards to general distributed ledgers, the list of alternative consensus protocols grows and begins to include more classical consensus protocols: two-phase commit, PBFT, etc. These tend to be significantly more efficient than the “proof-of-X” protocols used in cryptocurrencies, but with the tradeoff that they require a fixed set of known participants. As in Item 8, what is needed is a way to understand this tradeoff, provide meaningful comparisons within this growing landscape of consensus protocols, and understand the benefits that each provides in a given setting.
2. Privacy: how to protect data?
The problem, in short. While significant research has focused on the anonymity of users in cryptocurrencies (i.e., protecting the identity of participants), little research has focused on their privacy. For example, Bitcoin users are technically pseudonymous, as their on-chain identities are not inherently linked to their real-world identity, but the details of each transaction — e.g., the amount of bitcoins being sent — are still completely transparent. A naïve solution for more expressive platforms (in which privacy is even more of a concern) would be to encrypt this data and provide decryption keys only to the necessary parties, but the fact is that encryption schemes, as with all cryptographic primitives, break or are compromised, so this does not provide a long-term solution for privacy.
1. Scalability: do we need full agreement?
The problem. Arguably the biggest hurdle for fully distributed ledgers is the insistence that every node in the network needs to agree on the full state of the entire ledger. This means that ultimately distributed ledgers cannot and do not scale in terms of their ability to process growing numbers of transactions (throughput) while still ensuring that users do not have to wait for their transactions to be included (latency). In other words, the more computational power that joins the network, the worse it will perform in terms of throughput and latency. This is precisely because of this requirement that every node must agree on every transaction, as it means the more transactions are in the system, the longer the nodes must wait for them to flood the network.
Since this insistence on full replication thus violates one of the most basic properties of distributed systems, we might naturally ask ourselves: why do it in the first place? One of the primary benefits of cryptocurrencies, enabled by this requirement, is the idea of full public verifiability: any participant can verify for themselves the correct functioning of the system by, for example, replaying all transactions and ensuring that any agreed-upon rules haven’t been violated. If only certain nodes agree on certain parts of the ledger, then there is no one participant that can satisfy this notion of verifiability. To allow for increased throughput and decreased latency, one must therefore provide a balance between avoiding full replication — which is often accomplished using a technique called sharding, in which each participant processes transactions only within a given shard — and enabling at least some degree of openness and verifiability.
Potential and developing solutions. This topic has received significant attention, and several proposals adopt some form of sharding. While these approaches achieve better scalability, they raise questions about verifiability that fully decentralized solutions do not. For example, if only certain participants see certain transactions, how can other participants tell that their transactions obey the global set of rules? In the absence of such a global set of rules, what meaningful notions of integrity can we even satisfy? As with all the items on this list, we again see that each platform does not provide one uniquely perfect solution, but rather a set of tradeoffs that — to come full circle back to Item 10 — must ultimately be balanced according to the individual use case in which the technology will be used.
This post is a heavily condensed version of an article that will be published in a special blockchain issue of the IEEE S&P magazine. For the full version please check out the issue in early 2018!