Attack papers are case studies

We should treat attack papers like case studies. When we read them, review them, use them for evidence, and learn from them. This claim is not derogatory. Case studies are useful. But like anything, to be useful case studies need to be done and used appropriately.

Let’s be clear what I mean by attack paper. Any paper that reports how to attack some system. Any paper that includes details of an exploit, discloses a vulnerability, or demonstrates a proof-of-concept for breaching the security of a system. The efail paper that Steven discussed recently is an example. Security conferences are full of these; the ratio of attack papers to total papers varies per conference. USENIX Security tends to contain a fair few.

Let’s be clear what I mean by case study. I mean a scientific report that details a specific occurrence of interest as observed by the author. Case studies can be active, and include interviews or other questioning. They can be solely passive observation. Case studies can follow just one case in isolation, or might follow a series of related cases in similar ways for comparison. Case studies usually do not involve a planned intervention by the observer, otherwise we start to call them experiments. But they may track changes as the result of interventions outside the observer’s control.

What might change if we think about attack papers as case studies? We can apply our scientific experience from other disciplines. I’ve argued before that security is a science. We need to adapt scientific techniques, and other sciences might learn from what we do in security. But we need to be in a dialogue there. Calling attack papers what they are opens up this dialogue in several ways.

What expectations should we have when reading or reviewing a case study? The SAGE encyclopedia on qualitative research methods provides several properties we might expect to see. Transferability, trustworthiness, and confirmability, for example. But mostly, we should expect the case study to be done well and explain how it integrates with existing knowledge. We also know not to expect features associated with experiments, like statistical analysis of p-values.

What expectations should we have when doing or planning a case study? There are several great books about how to do case studies well. For example Stake’s is a classic. Sure, these may need to be adapted. The ethical considerations should look more like the Menlo Report than the norms in social sciences, for one example. But the details on what kinds of things to report, such as methodology, background, situational data, and avoiding common observer biases should transfer relatively easily. We should talk and write about what adjustments are found to be useful.

How can we generalize and learn from a case study? Case studies have formed the basis for scientific study in many fields for decades or even centuries. Astronomy is built up out of cases. So was Darwin’s theory of evolution (the experiments confirming speciation in the laboratory came only a century later). The early history of cognitive neuroscience are likewise built up out of cases (for example, the famous Phineas Gage).

Case studies also eventually need to be supplemented with controlled interventions. When we decide whether we care about a new case study, it depends on our existing knowledge. Is there a theory of how a phenomenon occurs in the relevant field? If not, most case studies are useful. If so, the case study is useful if it confirms important under-evidenced predictions of the theory, or if it challenges some part of the theory. Case studies are particularly useful as statements of existence or possibility. They are less useful at determining prevalence or distribution. If the latter are more important to our decision making, we should de-prioritize (though not eliminate) case studies. In all these situations, the case study is much more useful if it is self-aware of the existing theory and the author situates the case as providing evidence for or against certain aspects of various theories.

Can we say there is a theory of computer network attack and defense? I think so. There have been taxonomies of attacks since the mid 1990s. The Howard and Longstaff one is a favorite. There are also developed theories of how organizations respond to attacks, what the distribution of exploits is in the wild (Exploit Intelligence Project) and competing ideas about whether bugs are dense or sparse (e.g., Dan Geer’s talk). I’ve also helped put forward ideas of how scientific models (mechanistic models from biology) can be adapted to the kill chain and its steps of delivery and exploitation. And of course there are economic perspectives. This list certainly isn’t exhaustive. But I think it’s suggestive enough that there is some theory around exploiting computers that attack paper case studies could link in to.

Attack papers are case studies. I hope this is uncontroversial. What we should do about it, especially in the details, will require much further conversation. But whatever the details, the discussion should improve our thoughtfulness when conducting and writing such papers.

Thanks to the folks at RISCS for spurring me to write this post.

Security code AutoFill: is this new iOS feature a security risk for online banking?

A new feature for iPhones in iOS 12 – Security Code AutoFill – is supposed to improve the usability of Two Factor Authentication but could place users at risk of falling victim to online banking fraud.

Two Factor Authentication (2FA), which is often referred to as Two Step Verification, is an essential element for many security systems, especially those online and accessed remotely. In most cases, it provides extended security by checking if the user has access to a device. In SMS-based 2FA, for example, a user registers their phone number with an online service. When this service sees a login attempt for the corresponding user account, it sends a One Time Password (OTP), e.g. four to six digits, to the registered phone number. The legitimate user then receives this code and is able to quote it during the login process, but an impersonator won’t.

In a recent development by Apple, announced at its developer conference WWDC18, they are set to automate this last step to improve user experience with 2FA with a new feature that is set to be introduced to iOS in version 12. The Security Code AutoFill feature, currently available to developers in a beta version, will allow the mobile device to scan incoming SMS messages for such codes and suggest them at the top of the default keyboard.

Description of new iOS 12 Security Code AutoFill feature (source: Apple)

Currently, these SMS codes rely on the user actively switching apps and memorising the code, which can take a couple of seconds. Some users deploy alternative try strategies such as memorising the code from the preview banner and hastily typing it down. Apple’s new iOS feature will require only a single tap from the user. This will make the login process faster and less error prone, a significant improvement to the usability of 2FA. It could also translate into an increased uptake of 2FA among iPhone users.

Example of Security Code AutoFill feature in operation on iPhone (source: Apple)

If users synchronise SMS with their MacBook or iMac, the existing Text Message Forwarding feature will push codes from their iPhone and enable Security Code AutoFill in Safari.

Example of Security Code AutoFill feature synchronised with macOS Mojave (source: Apple)

Reducing friction in user interaction to improve technology uptake for new users, and increase the usability and satisfaction for existing users, is not a new concept. It has not only been discussed in academia at length but is also a common goal within industry, e.g. in banking. This is evident in how the financial and payment industry has encouraged contactless (Near Field Communication – NFC) payments, which makes transactions below a certain threshold much quicker than traditional Chip and PIN payments.

Continue reading Security code AutoFill: is this new iOS feature a security risk for online banking?