LANDER:internet address survey reprobing it43w-20110913 From Predict README version: 2286, last modified: 2011-09-29. This file describes the trace dataset "internet_address_survey_reprobing_it43w-20110913" provided by the LANDER project. Contents • 1 LANDER Metadata • 2 Dataset Contents • 3 Data Format • 4 Collection Method • 4.1 Probing Location(s) • 4.2 Beginning/Ending Date and Time Zone • 5 Citation • 6 Results Using This Dataset • 7 User Annotations LANDER Metadata ┌───────────────────────────┬────────────────────────────────────────────────────────────────────────────────────┐ │ dataSetName │ internet_address_survey_reprobing_it43w-20110913 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ status │ usc-web-and-predict │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ shortDesc │ multi-ping survey of some IPv4 addresses │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ longDesc │ To collect this data, a subset of Internet IP addresses was pinged by sending ICMP │ │ │ ECHO_REQUEST (PING) packet. The response (if it ever came within 11 minutes time │ │ │ interval) was recorded in this data set. Probe was repeated every 11 minutes. In │ │ │ all, 40283 netblocks (/24 subnets) were periodically reprobed. │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ datasetClass │ Quasi-Restricted │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ commercialAllowed │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ requestReviewRequired │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ productReviewRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ ongoingMeasurement │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ submissionMethod │ Upload │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartDate │ 2011-09-13 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartTime │ 21:34:46 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndDate │ 2011-09-27 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndTime │ 21:59:14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartDate │ 2012-01-27 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartTime │ 17:06:13 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndDate │ 2030-01-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ anonymization │ none │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ archivingAllowed │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ keywords │ category:address-space-status-data, subcategory:internet-census-and-survey-data, │ │ │ ip-address, sweep, address-collection, ping, icmp │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ format │ binary │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ access │ https │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ hostName │ USC-LANDER │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ providerName │ USC │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingId │ internet address surveys │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingSummaryFlag │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ retrievalInstructions │ download │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ byteSize │ 151483580416 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ expirationDays │ 14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ uncompressedSize │ 604756207675 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ impactDoi │ 10.23721/109/1353710 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ useAgreement │ dua-ni-160816 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ irbRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ privateAccessInstructions │ See https://ant.isi.edu/datasets/#getting-datasets for information on obtaining │ │ │ this dataset. │ │ │ See │ └───────────────────────────┴────────────────────────────────────────────────────────────────────────────────────┘ Dataset Contents internet_address_survey_reprobing_it43w-20110913.README.txt      copy of this README ips_all.it43w.txt all subnets probed during this survey first random part of the survey. These are subnets randomly selected over all responsive ips_random1.it43w.txt subnets. This part is typically copied from second random part of the previous survey (subject to the do-not-probe removals), i.e. ips_random2.it42w.txt. second random part of the survey (subnets randomly selected over all responsive subnets for ips_random2.it43w.txt this and next survey). This part is expected to be copied to the first random part of the next survey (minus do-not-probe requests), i.e. ips_random1.it44w.txt ips_stable.it43w.txt stable part of the survey (subnets that are expected to be probed in the future surveys) subset of ips_stable.it43w.txt that was initially ips_stable_random.it43w.txt chosen at random from responsive subnets (responsive to an early census) subset of ips_stable.it43w.txt that was selected ips_stable_select.it43w.txt by picking IPs spread all over AU-plane (from census data) ips_stable_usc-isi.it43w.txt if present: subset of ips_stable.it43w.txt that comprises subnets belonging to USC and ISI if present: subset of ips_stable.it43w.txt that ips_stable_wireless.it43w.txt contains a few long-latency prefixes associated mostly with mobile wireless devices data/     reprobing.pinger2.43w.*.bz2 binary data files     .sha1sum SHA-1 checksum pcap/     reprobing.pinger2.43w.*.pcap.bz2 pcap traces of "non-conformant" responses     .sha1sum SHA-1 checksum Subdirectory "data" contains four bzipped binary files containing probe records. Each file is named "reprobing.X.Y.Z.bz2", where: • X stands for the system name used to conduct the survey. E.g. "jar" corresponds to host jar.isi.edu with static IP address 128.9.160.132. Hosts are in different clusters as described in the "Probing Location" section below. • Y is the survey number, corresponding to the Yth census we've taken • Z is an index given sequentially (in time) to each file, starting from 0, so that files have a modest size The file ".sha1sum" contains SHA1 checksums of individual compressed files. The integrity of the distribution thus can be checked by independently calculating SHA1 sums of files and comparing them with those listed in the file. If you have the sha1sum utility installed on your system, you can do that by executing: sha1sum --check .sha1sum This has to be done before files are uncompressed. Subdirectory "pcap" holds gzipped pcap files containing raw ICMP responses other than ECHO_REPLY or ICMP_UNREACHABLE. See format description for more information. Data Format Binary format of trace files is described in detail here: http://www.isi.edu/ant/traces/topology/address_surveys/binformat_description.html Collection Method Data collection involves periodic pingging of all addresses within a large number of selected /24 subnets. A full description of this method is in: > John Heidemann, Yuri Pradkin, Ramesh Govindan, Christos Papadopoulos, Genevieve Bartlett, and Joseph Bannister. > Census and Survey of the Visible Internet. In Proceedings of the ACM Internet Measurement Conference, p.169-182. > Vouliagmeni, Greece, ACM. October, 2008 http://www.isi.edu/~johnh/PAPERS/Heidemann08c.html. Probing Location(s) The probing locations for surveys and surveys are indicated in their names. A "w" means west, which is from isi.edu in Marina del Rey, California; "c" means center, from colostate.edu, in Ft. Collins, Colorado; "e" means east which is from east.isi.edu, from Arlington, Virginia; "j" means Japan, which is from WIDE in Fujisawa-shi, Kanagawa, Japan; "g" stands for Athens, Greece. See Dataset Contents for more information on the system who did the survey. Beginning/Ending Date and Time Zone Dates/Times specified in the metadata are in UTC. Earlier censuses (before it37) used local time in their metadata description. Their metadata will be updated to effectively switch to UTC in the near future. Citation If you use this trace to conduct additional research, please cite it as: Internet Addresses Survey dataset, PREDICT ID: USC-LANDER/internet_address_survey_reprobing_it43w-20110913/rev2286. Traces taken 2011-09-13 to 2011-09-27. Provided by the USC/LANDER project (http://www.isi.edu/ant/lander). Results Using This Dataset Traces similar to this one containing collections of "live" IP addresses have been used the following previously published work: • John Heidemann, Yuri Pradkin, Ramesh Govindan, Christos Papadopoulos, Genevieve Bartlett, and Joseph Bannister. Census and Survey of the Visible Internet. In Proceedings of the ACM Internet Measurement Conference, p.169-182. Vouliagmeni, Greece, ACM. October, 2008 http://www.isi.edu/~johnh/PAPERS/Heidemann08c.html. • Yuri Pryadkin, Robert Lindell, Joseph Bannister, and Ramesh Govindan An Empirical Evaluation of IP Address Space Occupancy Technical Report ISI-TR-2004-598, USC/Information Sciences Institute, November 2004 ftp://ftp.isi.edu/isi-pubs/tr-598.pdf. • Lin Quan, John Heidemann, Yuri Pradkin. Detecting Internet Outages with Precise Active Probing (extended). Technical Report ISI-TR-2012-678b, USC/Information Sciences Institute, May, 2012 ftp://ftp.isi.edu/isi-pubs/tr-678b.pdf. User Annotations Currently no annotations. Categories: • Datasets • LANDER • LANDER:Datasets • LANDER:Datasets:AddressSpace:Survey • LANDER:Datasets:AddressSpace