Exploring Visible Internet Hosts through Census and Survey

Heidemann, John and Pradkin, Yuri and Govindan, Ramesh and Papadopoulos, Christos and Bannister, Joseph
USC/Information Sciences Institute

citation

John Heidemann, Yuri Pradkin, Ramesh Govindan, Christos Papadopoulos and Joseph Bannister 2007. Exploring Visible Internet Hosts through Census and Survey. Technical Report ISI-TR-2007-640. USC/Information Sciences Institute. [PDF] [Software] [Dataset]

abstract

Measurement studies published in the literature have, for the most part, ignored the population of hosts. While many hosts are hidden behind firewalls and in private address space, there is much to be learned from examining the population of visible Internet hosts–one can better understand network growth and accessibility and this understanding can help to assess vulnerabilities, deployment of new technologies, and improve network models. This paper is, to our knowledge, the first attempt to measure the population of visible Internet edge hosts. We measure hosts in two ways: via periodic Internet censuses, where we query all accessible Internet addresses every few months, and via surveys of a small fraction of the responsive address space, probing each address every 11 minutes for one week. These approaches are complementary: a census is effective at evaluating the Internet as a whole, while surveys validate the census and allow observation of the lifetime of typical address occupancy. We find that only 3.6% of allocated addresses are actually occupied by visible hosts, and that occupancy is unevenly distributed, with a quarter of responsive /24 subnets less than 5% full, and only 9% of subnets more than half full. We establish an upper-bound on the number of servers in the Internet at 36 million, about 16% of the responsive addresses. Many firewalls are visible and we observe significant diversity in the distribution of firewalled block size. While the absolute number of firewalled blocks appears stable, the ratio of coverage of visible firewalls to the number of visible addresses is declining, perhaps suggesting increasing use of invisible firewalls.

reference

@techreport{Heidemann07c,
  author = {Heidemann, John and Pradkin, Yuri and Govindan, Ramesh and Papadopoulos, Christos and Bannister, Joseph},
  title = {Exploring Visible Internet Hosts through Census and Survey},
  institution = {USC/Information Sciences Institute},
  year = {2007},
  sortdate = {2007-05-01},
  number = {ISI-TR-2007-640},
  month = may,
  jlocation = {johnh: pafile},
  keywords = {visible internet, exploration, census, survey},
  copyrightholder = {author},
  url = {https://ant.isi.edu/%7ejohnh/PAPERS/Heidemann07c.html},
  pdfurl = {https://ant.isi.edu/%7ejohnh/PAPERS/Heidemann07c.pdf},
  otherurl = {ftp://ftp.isi.edu/isi-pubs/tr-640.pdf},
  myorganization = {USC/Information Sciences Institute},
  project = {ant, lacrend},
  softwareurl = {https://ant.isi.edu/software/},
  dataseturl = {https://ant.isi.edu/datasets/all.html},
  blogurl = {}
}