Do You See Me Now? Sparsity in Passive Observations of Address Liveness

Do You See Me Now? Sparsity in Passive Observations of Address Liveness

Mirkovic, Jelena and Bartlett, Genevieve and Heidemann, John and Shi, Hao and Deng, Xiyue
USC/Information Sciences Institute

Jelena Mirkovic, Genevieve Bartlett, John Heidemann, Hao Shi and Xiyue Deng 2017. Do You See Me Now? Sparsity in Passive Observations of Address Liveness. IEEE International Workshop on Traffic Monitoring and Analaysis (Dublin, Ireland, Jul. 2017), to appear.

Abstract

Accurate information about address and block usage in the Internet has many applications in planning address allocation, topology studies, and simulations. Prior studies used active probing, sometimes augmented with passive observation, to study \emphmacroscopic phenomena, such as the overall usage of the IPv4 address space. This paper instead studies the completeness of \emphpassive sources: how well they can observe \emphmicroscopic phenomena such as address usage within a given network. We define \textitsparsity as the limitation of a given monitor to see a target, and we quantify the effects of interest, temporal, and coverage sparsity. To study sparsity, we introduce \emphinverted analysis, a novel approach that uses complete passive observations of a few end networks (three campus networks in our case) to infer what of these networks would be seen by millions of \emphvirtual monitors near their traffic’s destinations. Unsurprisingly, we find that monitors near popular content see many more targets and that visibility is strongly influenced by bipartite traffic between clients and servers. We are the first to quantify these effects and show their implications for the study of Internet liveness from passive observations. We find that visibility is heavy-tailed, with only 0.5% monitors seeing more than 10% of our targets’ addresses, and is most affected by \emphinterest sparsity over temporal and coverage sparsity. Visibility is also strongly bipartite. Monitors of a different class than a target (e.g., a server monitor observing a client target) outperform monitors of the same class as a target in 82-99% of cases in our datasets. Finally, we find that adding active probing to passive observations greatly improves visibility of both server and client target addresses, but is not critical for visibility of target blocks. Our findings are valuable to understand limitations of existing measurement studies, and to develop methods to maximize microscopic completeness in future studies.

Reference

@inproceedings{Mirkovic17a,
  author = {Mirkovic, Jelena and Bartlett, Genevieve and Heidemann, John and Shi, Hao and Deng, Xiyue},
  title = {Do You See Me Now? Sparsity in Passive
                    Observations of Address Liveness},
  booktitle = { IEEE International Workshop on Traffic Monitoring and Analaysis},
  year = {2017},
  pages = {to appear},
  month = jul,
  address = {Dublin, Ireland},
  publisher = {IEEE},
  location = {johnh: pafile},
  keywords = {passive observation, internet census},
  url = {http://www.isi.edu/%7ejohnh/PAPERS/Mirkovic17a.html},
  pdfurl = {http://www.isi.edu/%7ejohnh/PAPERS/Mirkovic17a.pdf},
  blogurl = {https://ant.isi.edu/blog/?p=1001},
  myorganization = {USC/Information Sciences Institute},
  sortdate = {2017-07-21},
  project = {ant, lacrend, lander, retrofuture},
  jsubject = {topology_modeling},
  doi = {xxx},
  copyrightholder = {IEEE}
}