The first part of this article introduced the malicious file download dataset and the delivery network structure. This final part explores the types of files delivered, discusses how the network varies over time, and concludes with challenges for the research community.
The Great Divide: A PUP Ecosystem and a Malware Ecosystem
We found a notable divide in the delivery of PUP and malware. First, there is much more PUP than malware in the wild: we found PUP-to-malware ratios of 5:1 by number of SHA-2s, and 17:2 by number of raw downloads. Second, we found that mixed delivery mechanisms of PUP and malware are not uncommon (e.g., see our Opencandy case study in the paper). Third, the highly connected Giant Component is predominantly a PUP Ecosystem (8:1 PUP-to-malware by number of SHA-2s), while the many “islands” of download activity outside of this component are predominantly a Malware Ecosystem (1.78:1 malware-to-PUP by number of SHA-2s).
Comparing the structures of the two ecosystems,we found that the PUP Ecosystem leverages a higher degree of IP address and autonomous system (AS) usage per domain and per dropper than the Malware Ecosystem, possibly indicating higher CDN usage or the use of evasive fast-flux techniques to change IP addresses (though, given earlier results, the former is the more likely). On the other hand, the Malware Ecosystem was attributed with fewer SHA-2s being delivered per domain than the PUP Ecosystem with the overall numbers in raw downloads remaining the same, which could again be indicative of a disparity in the use of CDNs between the two ecosystems (i.e., CDNs typically deliver a wide range of content). At the same time, fewer suspicious SHA-2s being delivered per domain could also be attributable to evasive techniques being employed (e.g., malicious sites delivering a few types of files before changing domain) or distributors in this ecosystem dealing with fewer clients and smaller operations.
We tried to estimate the number of PPIs in the wild by defining a PPI service as a network-only component (or group of components aggregated by e2LD) that delivered more than one type of malware or PUP family. Using this heuristic, we estimated a lower bound of 394 PPIs operating on the day, 215 of which were in the PUP Ecosystem. In terms of proportions, we found that the largest, individual PPIs in the PUP and Malware Ecosystems involved about 99% and 24% of all e2LDs and IPs in their ecosystems, respectively.
With there being a number of possible explanations for these structural differences between ecosystems, and such a high degree of potential PPI usage in the wild (especially within the PUP Ecosystem), this is clearly an area in which further research is required.
Keeping Track of the Waves
The final part of the study involved tracking these infrastructures and their activities over time. Firstly, we generated tracking signatures of the network-only (server-side) and file-only (client-side) delivery infrastructures. In essence, this involved tracking the root and trunk nodes in a component, which typically had the highest node degrees, and thus, were more likely to be stable, as opposed to the leaf nodes, which were more likely to be ephemeral.
We monitored the churn of these delivery infrastructures. The daily churn of these infrastructures (as well as individual files) showed a weekly periodicity in their activities with activity peaking on the weekends, indicating an increase of infections during business hours, perhaps due to more computers being turned on at these times. This periodicity has been observed by other researchers and follows the Routine Activity Theory from criminology, which postulates that crime follows opportunity.
The long-term analysis revealed that 26% of IPs and hosts are stable for at least a year, and malware-only delivery operations tend to last longer than PUP-only ones (medians of 5 weeks to 3 weeks). Perhaps more interesting was the significant drop in suspicious download activity between mid-November and mid-December of 2015, which we correlated with a Russian takedown operation against the Dyre financial fraud trojan and its operators (as reported by Symantec) and verified with our own independent analysis. A novel finding in our analysis was the observed simultaneous drop in the activities of other unwanted software not previously associated with the Dyre operation, including the Amonetize, Installcore, Eorezo, and Convertad PUP PPIs, and the Neshta malware family, potentially alluding to some shared infrastructure or business interests between their operators.
A Retrospective Evaluation and A Look on the Horizon
In summary, we conducted a comprehensive, data-driven analysis of malware delivery networks on the Web, with a methodology to identify its key elements. Though mixed delivery operations are not uncommon, we found two disjoint ecosystems, with the stable PUP Ecosystem conducting the lion’s share of suspicious download activity. We explicitly estimated the ratios of PUP-to-malware in the wild (17:2) and differentiated on the two ecosystems’ characteristics. We also found that most network hosts are volatile, but 26% are stable for over a year. Our research indicates that PUP is a bigger problem than previously portrayed, thus requiring more attention from the security community.
However, there were a number of issues that we experienced with this study, ranging from sampling biases in the dataset from the Symantec users for which we cannot account (behavioral, geographical, socioeconomic, etc.) to limitations in our analysis in identifying inter-URL connections (hyperlinks), the types of infection vectors used (exploit kits, social engineering), or clusters of a single malware that appears as many file nodes (SHA-2s) due to re-packing.
The most prominent issue, in our opinion, was the severe lack of ground-truth data. At the time (late 2016), VirusTotal only held records for about 10% of the observed file SHA-2s for activity on 1 October 2015, many of which were then classified as either benign or malicious but without a family label. To put this into perspective, over a year after being observed in the wild, VirusTotal was unable to classify at least 90% of files. In fact, as of July 2019 when we first presented our work, some of the most prolific droppers in this snapshot were only recorded by VirusTotal a few weeks prior (i.e., over three and a half years after first observation). Of course, VirusTotal is still a highly valuable resource, but the issue of ground-truth coverage is a common problem in the research community that needs to be addressed.
Moving forward, some areas where further research is needed are already clear. For instance, what can we find when we look deeper into the structure of the PUP Ecosystem (GC), and the structures of individual PUP, malware, and mixed delivery operations? What does this ecosystem look like in 2019, and how does it compare with mobile file delivery? Is there homophily between the types of operations and their distributors (e.g., PUP being delivered more with other PUP), and can we uncover more previously unknown business relationships?
By reducing the snapshot window from 24 hours to a smaller increment, can we confirm the use of fast-flux techniques with malware and/or PUP delivery operations? Are there differences in the short-term activities and evolutions of PUP operations versus malware ones? Alternatively, can we detect and track botnet activity? If we introduce some benign ground-truth data, can we (automatically) differentiate between these and benign software delivery campaigns through static and dynamic analysis? Moreover, in relation to aiding in the ground-truth problem, can we cluster unknown file delivery campaigns and their SHA-2s with known PUP and malware families?
Finally, looking towards disrupting this phenomenon, can we evaluate the success rate of current intervention methods, and possibly identify better targets? How do malicious delivery operations respond to law enforcement takedowns? Also, how much of this ecosystem overlaps with other underground markets, activities, and their economies?
Indeed, as is the parlance, more research is required.