Security economics

A Regulatory “Trial and Error” Phase Shaping the Cryptocurrency Ecosystem

In general, the broad aim of regulation is to organise the relationship between persons and to protect their rights and interests in society. However as we have discovered, this has not been the case within the cryptocurrency space.

It has been almost 15 years since Satoshi Nakamoto published the Bitcoin whitepaper. Since then, this seminal document posted on a mailing list, catalysed the emergence of an entirely new-transnational field. Considering the substantial time that has passed and the widespread popularity of cryptocurrencies, one would expect regulators to have at least figured out ‘how’ and ‘what’ to regulate, right? But it appears not so. Regulators do not seem to want to provide long-term, serious solutions, but rather a collection of outdated bandages of reactions that mostly portray the illusion of a solution, with not-so-hidden intentions in mind.

The cryptocurrency ecosystem itself has often been compared to the Wild West. Nevertheless, this also applies to its journey to be regulated. A journey full of ill-suited regulatory approaches that resulted in a plethora of dilemmas. We explore this tension between the technology and regulation in our latest paper “Shaping Cryptocurrency Gatekeepers with a Regulatory ‘Trial and Error’”, with a primary focus on the Financial Action Task Force’s recommendations, and the EU’s 5th Anti-Money Laundering Directive.

Historically speaking, it was only after the rising popularity of the Silk Road and the collapse of the most popular (at the time) exchange Mt. Gox, that regulators realised that they needed to take action. The advertised main objective here is the curbing of criminal activity and providing regulatory protection to consumers/users. However, until only recently, most of the regulatory steps taken by most regulators choosing to act, were approaches mainly targeting money laundering and terrorist financing, with other limited initiatives here and there. Whilst this approach might have had some potential benefits, it was not 1. comprehensive, 2. global, 3. stable/constant, or 4. tailored to address the specific risks and characteristics of cryptocurrencies. In other words, different regulators have been testing diverse approaches, simultaneously, without engaging with one another, and without properly acknowledging the true needs and risks of the ecosystem.

Continue reading A Regulatory “Trial and Error” Phase Shaping the Cryptocurrency Ecosystem

Rugpull reports in the DeFi jungle

A rising category of cryptocurrency scams called ‘rugpulls’ accounted for 37% of all cryptocurrency scam revenue in 2021. A rugpull is an exit scam in the DeFi (Decentralized Finance) ecosystem where developers abandon a project without fully delivering and run away with investors’ funds. Thodex, a Turkish centralized exchange, ran away with $2 billion from victims. In March 2022, the U.S. Department of Justice charged two defendants for a $1.1 million NFT rugpull scam called Frosties.

In our paper to be presented next week at Financial Cryptography and Data Security 2023, we analyze an updated list of rugpulls from an online discussion forum – bitcointalk.org. This forum provides a platform for everyone to discuss anything on crypto that also attracts scammers to advertise their projects. We observe that since 2020, the number of rugpull threads has increased, while the ones containing exit scams have decreased; the total mention of either of these terms is relatively stable over time. This means that users have started using the term ‘rugpull’ instead of ‘exit scam’ since the DeFi space emerged.

Using keywords to search for threads discussing rugpulls, we found 101 rugpulls from six services, summarised in Table 1. Our dataset is available from the Harvard Dataverse as doi:10.7910/DVN/SMGMW8.

Table 1: DeFi service types by quantity of observed rugpulls (N=101)
Service Type	Definition	Observation
Initial Coin Offerings (ICOs)	Raising money to create a new ERC20 token	73
Yield farms	Lending crypto assets to earn interest on the loan	16
Exchanges	Platforms for users to buy/sell cryptocurrency	5
Non-Fungible Tokens (NFTs)	Unique, non-interchangeable digital asset that can be bought and sold	5
Initial Dex Offerings (IDOs)	Similar to ICO, but on a decentralized exchange	1
Cloud mining	Fractional shares of a mining operation	1

We find that Initial Coin Offerings (ICOs) form the majority of rugpulls, and most of them pulled the rug in less than six months. For example, the SquidGame Token, named after a famous TV show, rugpulled within days in 2021.

Continue reading Rugpull reports in the DeFi jungle

US proposes to protect bank customers from Authorised Push Payment fraud

This week, at the US House Financial Services Committee hearing, Representative Stephen F. Lynch announced a draft of the Protecting Consumers From Payment Scams Act. If enacted, this would expand the existing protection for US customers (Regulation E) who have funds transferred out of their account without their consent, to also cover when the customer is tricked into performing the fraudulent transfer themselves. This development is happening in parallel with efforts in the UK and elsewhere to reduce fraud and better protect victims. However, the draft act’s approach is notably different from the UK approach – it’s simpler, gives stronger protection to customers, and shifts liability to the bank receiving fraudulent transfers. In this post, I’ll discuss these differences and what the implications might be.

The type of fraud the proposed law deals with, where criminals coerce victims into making payment under false pretences, is known as Authorised Push Payment (APP) fraud and is a problem worldwide. In the UK, APP fraud is now by far the most common type of payment fraud, with losses of £355 million in the first half of 2021, more than all types of card fraud put together (£261 million).

APP fraud falls outside of existing consumer protection, so victims are commonly held liable for the losses. The effects can be life-changing, with people losing 6-figure sums within minutes. It’s therefore welcome to see moves to better consumer protection. The UK was one of the first to tackle this problem, with a voluntary code of practice being put in place following years of campaigning by consumer rights organisations, particularly Which.

Continue reading US proposes to protect bank customers from Authorised Push Payment fraud

Apple letting the content-scanning genie out of the bottle

When Apple announced that they would be scanning iPhones for child sexual abuse material (CSAM), the push-back appears to have taken them by surprise. Since then, Apple has been engaging with experts and developing their proposals to mitigate risks that have been raised. In this post, I’ll discuss some of the issues with Apple’s CSAM detection system and what I’ve learned from their documentation and events I’ve participated in.

Technically Apple’s CSAM detection proposal is impressive, and I’m pleased to see Apple listening to the community to address issues raised. However, the system still creates risks that will be difficult to avoid. Governments are likely to ask to expand the system to types of content other than CSAM, regardless of what Apple would like to happen. When they do, there will be complex issues to deal with, both for Apple and the broader technology community. The proposals also risk causing people to self-censor, even when they are doing nothing wrong.

How Apple’s CSAM detection works

The iPhone or iPad scans images for known CSAM just before it uploads the image to Apple’s cloud data storage system – iCloud. Images that are not going to be uploaded don’t get scanned. The comparison between images and the database is made in such a way that minor changes to CSAM, like resizing and cropping, will trigger a match, but any image that wasn’t derived from a known item of CSAM should be very unlikely to match. The results of this matching process go into a clever cryptographic system designed to ensure that the user’s device doesn’t learn the contents of the CSAM database or which of their images (if any) match. If more than a threshold of about 30 images match, Apple will be able to verify if the matching images are CSAM and, if so, report to the authorities. If the number of matching images is less than the threshold, Apple learns nothing.

Risk of scope creep

Now that Apple has built their system, a risk is that it could be extended to search for content other than CSAM by expanding the database used for matching. While some security properties of their system are ensured through cryptography, the restriction to CSAM is only a result of Apple’s policy on the content of the matching database. Apple has clearly stated that it would resist any expansion of this policy, but governments may force Apple to make changes. For example, in the UK, this could be through a Technical Capability Notice (under the Investigatory Powers Act) or powers proposed in the Online Safety Bill.

If a government legally compelled them to expand the matching database, Apple may have to choose between complying or leaving the market. So far, Apple has refused to say which of these choices they would take.

Continue reading Apple letting the content-scanning genie out of the bottle

Evidence Critical Systems: Designing for Dispute Resolution

On Friday, 39 subpostmasters had their criminal convictions overturned by the Court of Appeal. These individuals ran post office branches and were prosecuted for theft, fraud and false accounting based on evidence from Horizon, the Post Office computer system created by Fujitsu. Horizon’s evidence was asserted to be reliable by the Post Office, who mounted these prosecutions, and was accepted as proof by the courts for decades. It was only through a long and expensive court case that a true record of Horizon’s problems became publicly known, with the judge concluding that it was “not remotely reliable”, and so allowing these successful appeals against conviction.

The 39 quashed convictions are only the tip of the iceberg. More than 900 subpostmasters were prosecuted based on evidence from Horizon, and many more were forced to reimburse the Post Office for losses that might never have existed. It could be the largest miscarriage of justice the UK has ever seen, and at the centre is the Horizon computer system. The causes of this failure are complex, but one of the most critical is that neither the Post Office nor Fujitsu disclosed the information necessary to establish the reliability (or lack thereof) of Horizon to subpostmasters disputing its evidence. Their reasons for not doing so include that it would be expensive to collect the information, that the details of the system are confidential, and disclosing the information would harm their ability to conduct future prosecutions.

The judgment quashing the convictions had harsh words about this failure of disclosure, but this doesn’t get away from the fact that over 900 prosecutions took place before the problem was identified. There could easily have been more. Similar questions have been raised relating to payment disputes: when a customer claims to be the victim of fraud but the bank says it’s the customer’s fault, could a computer failure be the cause? Both the Post Office and banking industry rely on the legal presumption in England and Wales that computers operate correctly. The responsibility for showing otherwise is for the subpostmaster or banking customer.

Continue reading Evidence Critical Systems: Designing for Dispute Resolution

Thoughts on the Future Implications of Microsoft’s Legal Approach towards the TrickBot Takedown

Just this week, Microsoft announced its takedown operation against the TrickBot botnet, in collaboration with other cybersecurity partners, such as FS-ISAC, ESET, and Symantec. This takedown followed Microsoft’s successful application for a court order this month, enabling them to enact technical disruption against the botnet. Such legal processes are typical and necessary precursors to such counter-operations.

However, what was of particular interest, in this case, was the legal precedent Microsoft (successfully) sought, which was based on breaches of copyright law. Specifically, they founded their claim on the alleged reuse (and misuse) of Microsoft’s copyrighted software – the Windows 8 SDK – by the TrickBot malware authors.

Now, it is clear that this takedown operation is not likely to cripple the entirety of the TrickBot operation. As numerous researchers have found (e.g., Stone-Gross et al., 2011; Edwards et al., 2015), a takedown operation often works well in the short-term, but the long-term effects are highly variable. More often than not, unless they are arrested, and their infrastructure is seized, botnet operators tend to respond to such counter-operations by redeploying their infrastructure to new servers and ISPs, moving their operations to other geographic regions or new targets, and/or adapting their malware to become more resistant to detection and analysis. In fact, these are just some of the behaviours we observed in a case-by-case longitudinal study on botnets targeted by law enforcement (one of which involved Dyre, a predecessor of the TrickBot malware). A pre-print of this study is soon to be released.

So, no, I’m not proposing to discuss the long-term efficacy of takedown operations such as this. That is for another blog post.

Rather, what I want to discuss (or, perhaps, more accurately, put forward as some initial thoughts) are the potential implications of Microsoft’s legal approach to obtaining the court order (which is incumbent for such operations) on future botnet takedowns, particularly in the area of malicious code reuse.

The role of usability, power dynamics, and incentives in dispute resolutions around computer evidence

As evidence produced by a computer is often used in court cases, there are necessarily presumptions about the correct operation of the computer that produces it. At present, based on a 1997 paper by the Law Commission, it is assumed that a computer operated correctly unless there is explicit evidence to the contrary.

The recent Post Office trial (previously mentioned on Bentham’s Gaze) has made clear, if previous cases had not, that this assumption is flawed. After all, computers and the software they run are never perfect.

This blog post discusses a recent invited paper published in the Digital Evidence and Electronic Signature Law Review titled The Law Commission presumption concerning the dependability of computer evidence. The authors of the paper, collectively referred to as LLTT, are Peter Bernard Ladkin, Bev Littlewood, Harold Thimbleby and Martyn Thomas.

LLTT examine the basis for the presumption that a computer operated correctly unless there is explicit evidence to the contrary. They explain why the Law Commission’s belief in Colin Tapper’s statement in 1991 that “most computer error is either immediately detectable or results from error in the data entered into the machine” is flawed. Not only can computers be assumed to have bugs (including undiscovered bugs) but the occurrence of a bug may not be noticeable.

LLTT put forward three recommendations. First, a presumption that any particular computer system failure is not caused by software is not justified, even for software that has previously been shown to be very reliable. Second, evidence of previous computer failure undermines a presumption of current proper functioning. Third, the fact that a class of failures has not happened before is not a reason for assuming it cannot occur.

Continue reading The role of usability, power dynamics, and incentives in dispute resolutions around computer evidence

Transparency, evidence and dispute resolution

Despite the ubiquity of computers in everyday life, resolving a dispute regarding the misuse or malfunction of a system remains hard to do well. A recent example of this is the, now concluded, Post Office trial about the dispute between Post Office Limited and subpostmasters who operate some Post Office branches on their behalf.

Subpostmasters offer more than postal services, namely savings accounts, payment facilities, identity verification, professional accreditation, and lottery services. These services can involve large amounts of money, and subpostmasters were held liable for losses at their branch. The issue is that the accounting is done by the Horizon accounting system, a centralised system operated by Post Office Limited, and subpostmasters claim that their losses are not the result of errors or fraud on their part but rather a malfunction or malicious access to Horizon.

This case is interesting not only because of its scale (a settlement agreement worth close to £58 million was reached) but also because it highlights the difficulty in reasoning about issues related to computer systems in court. The case motivated us to write a short paper presented at the Security Protocols Workshop earlier this year – “Transparency Enhancing Technologies to Make Security Protocols Work for Humans”. This work focused on how the liability of a party could be determined when something goes wrong, i.e., whether a customer is a victim of a flaw in the service provider’s system or whether the customer has tried to defraud the service provider.

Applying Bayesian thinking to dispute resolution

An intuitive way of thinking about this problem is to apply Bayesian reasoning. Jaynes makes a good argument that any logically consistent form of reasoning will lead to taking this approach. Following this approach, we can consider the odd’s form of Bayes’ theorem expressed in the following way.

Odds form of Bayes' theorem

There is a good reason for considering the odd’s form of Bayes’ theorem over its standard form – it doesn’t just tell you if someone is likely to be liable, but whether they are more likely to be liable than not: a key consideration in civil litigation. If a party is liable, the probability that there is evidence is high so what matters is the probability that if the party is not liable there would be the same evidence. Useful evidence is, therefore, evidence that is unlikely to exist for a party that is not liable.

Continue reading Transparency, evidence and dispute resolution

By revisiting security training through economics principles, organisations can navigate how to support effective security behaviour change

Here I describe analysis by myself and colleagues Albesë Demjaha and David Pym at UCL, which originally appeared at the STAST workshop in late 2019 (where it was awarded best paper). The work was the basis for a talk I gave at Cambridge Computer Laboratory earlier this week (I thank Alice Hutchings and the Security Group for hosting the talk, as it was also an opportunity to consider this work alongside themes raised in our recent eCrime 2019 paper).

Secure behaviour in organisations

Both research and practice have shown that security behaviours, encapsulated in policy and advised in organisations, may not be adopted by employees. Employees may not see how advice applies to them, find it difficult to follow, or regard the expectations as unrealistic. Employees may, as a consequence, create their own alternative behaviours as an effort to approximate secure working (rather than totally abandoning security). Organisational support can then be critical to whether secure practices persist. Economics principles can be applied to explain how complex systems such as these behave the way they do, and so here we focus on informing an overarching goal to:

Provide better support for ‘good enough’ security-related decisions, by individuals within an organization, that best approximate secure behaviours under constraints, such as limited time or knowledge.

Traditional economics assumes decision-makers are rational, and that they are equipped with the capabilities and resources to make the decision which will be most beneficial for them. However, people have reasons, motivations, and goals when deciding to do something — whether they do it well or badly, they do engage in thinking and reasoning when making a decision. We must capture how the decision-making process looks for the employee, as a bounded agent with limited resources and knowledge to make the best choice. This process is more realistically represented in behavioural economics. And yet, behaviour intervention programmes mix elements of both of these areas of economics. It is by considering these principles in tandem that we explore a more constructive approach to decision-support in organisations.

Contradictions in current practice

A bounded agent often settles for a satisfactory decision, by satisficing rather than optimising. For example, the agent can turn to ‘rules of thumb’ and make ad-hoc decisions, based on a quick evaluation of perceived probability, costs, gains, and losses. We can already imagine how these restrictions may play out in a busy workplace. This leads us toward identifying those points of engagement at which employees ought to be supported, in order to avoid poor choices.

Continue reading By revisiting security training through economics principles, organisations can navigate how to support effective security behaviour change

Resolving disputes through computer evidence: lessons from the Post Office Trial

On Monday, the final judgement in the Post Office trial was handed down, finding in favour of the claimants on all counts. The outcome will be of particular interest to the group of 587 claimants who brought the case against Post Office Limited, but the judgement also illustrates problems handling evidence generated by computers that have much broader applicability. I think this trial demonstrates that the way such disputes are resolved is not fit for purpose and that changes are needed in both in how computers generate evidence and how such evidence is reasoned about in litigation.

This case centres around disputes between Post Office Limited and sub-postmasters who operate Post Office branches on its behalf. Post Office Limited supplies these sub-postmasters with products to sell, and the computer accounting system – Horizon – for managing the branch. The claimants contend that shortfalls between the money that was in their branch and what Horizon says result from bugs in Horizon or someone maliciously accessing it. The Post Office instead claims that the shortfalls are real, and it is the responsibility of the sub-postmaster to reimburse the Post Office.

Such disputes have resulted in sub-postmasters being bankrupted, and others have even been jailed because the Post Office contends that evidence produced by Horizon demonstrates fraud by the sub-postmaster. The judgement vindicates the sub-postmasters, concluding that Horizon “was not remotely robust”.

This trial is actually the second in this case, with the prior one also finding in favour of the sub-postmasters – that the contractual terms set by Post Office regarding how they investigate and handle shortfalls are unfair. There would have been at least two more trials, had the parties not settled last week with Post Office Limited offering an apology and £58m in compensation. Of this, the vast majority will go towards legal costs and to the fund which bankrolled the litigation – leaving claimants lucky to get much more than £10k on average. Disappointing, sure, but better than nothing and that is what they could have got had the trials and inevitable appeals continued.

As would be expected for a trial depending on highly technical arguments, expert evidence featured heavily. The Post Office expert took a quantitative approach, presenting a statistical argument that claimant’s losses were implausibly high. This argument went by making a rough approximation as to the total losses of all sub-postmasters resulting from bugs in Horizon. Then, by assuming that these losses were spread over all sub-postmasters equally, losses by the 587 claimants would be no more than £25,000 – far less than the £18.7 million claimed. On this basis, the Post Office said that it is implausible for Horizon bugs to be the cause of the losses, and instead they are the fault of the affected sub-postmasters.

This argument is fundamentally flawed; I said so at the time, as did others. The claimant group was selected specifically as people who thought they were victims of Horizon bugs so it’s quite reasonable to think this group might indeed be disproportionally affected by Horizon bugs. The judge agreed, saying, “The group has a bias, in statistical terms. They plainly cannot be treated, in statistical terms, as though they are a random group of 587 [sub-postmasters]”. This error can be corrected, but the argument becomes circular and a statistical approach adds little new information. As the judgement concludes, “probability theory only takes one so far in this case, and that is not very far”.

Continue reading Resolving disputes through computer evidence: lessons from the Post Office Trial