LANDER:internet outage adaptive a37 supplement-20190701 From Predict README version: 11479, last modified: 2021-01-27. This file describes the trace dataset "internet_outage_adaptive_a37_supplement-20190701" provided by the LANDER project. Contents • 1 LANDER Metadata • 2 Dataset Contents • 2.1 Geolocation of the blocks • 3 Data Format • 3.1 Block to AS mapping • 3.2 Block to organization mapping • 3.3 Block to Linktype • 3.4 Block to RDNS • 4 Collection Method • 5 Citation • 6 User Annotations LANDER Metadata ┌───────────────────────────┬────────────────────────────────────────────────────────────────────────────────────┐ │ dataSetName │ internet_outage_adaptive_a37_supplement-20190701 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ status │ usc-web-and-predict │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ shortDesc │ supplement data for outage survey │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ longDesc │ supplement data of IP mappings AS numbers, DNS names, Organization and linktypes │ │ │ for datasets │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ datasetClass │ Quasi-Restricted │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ commercialAllowed │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ requestReviewRequired │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ productReviewRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ ongoingMeasurement │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ submissionMethod │ Upload │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartDate │ 2019-07-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndDate │ 2019-07-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartDate │ 2021-01-27 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndDate │ 2030-01-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ anonymization │ none │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ archivingAllowed │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ │ category:address-space-status-data, │ │ keywords │ subcategory:internet-address-block-classification, IP-mapping, AS-mapping, │ │ │ Org-mapping │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ format │ text │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ access │ https │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ hostName │ USC-LANDER │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ providerName │ USC │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingId │ Adaptive Outage Detection Supplemental │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingSummaryFlag │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ retrievalInstructions │ download │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ byteSize │ 37748736 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ expirationDays │ 14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ uncompressedSize │ 271399597 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ impactDoi │ │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ useAgreement │ dua-ni-160816 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ irbRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ privateAccessInstructions │ See https://ant.isi.edu/datasets/#getting-datasets for information on obtaining │ │ │ this dataset. │ │ │ See │ └───────────────────────────┴────────────────────────────────────────────────────────────────────────────────────┘ Dataset Contents This dataset contains supplemental information to help in classifying blocks probed in Adaptive Outage Datasets: This information is derived from public sources and helps put outages in context by considering ASes, hostnames, and linktypes for all blocks probed in the above outage datasets. internet_outage_adaptive_a37_supplement-20190701.README.txt copy of this README      block_to_as.fsdb.bz2 AS information for each block (if present) Names of ASes (if present) This file is retrieved from https://www.cidr-report.org/as2.0/autnums.html. Up as_to_name.fsdb.bz2 until a31 this file was added way after data collection and may not be 100% accurate if applied retroactively. However, our assumption is that AS renumbering is a very infrequent event. block_to_linktype.fsdb.bz2 Inferred linktype for each block (if present) block_to_org*.fsdb.bz2 Organization information for each block (if present) block_to_rdns.fsdb.bz2 Reverse dns for each block (if present) Note, that some files mentioned above may be missing if the corresponding mapping information is not available. Geolocation of the blocks Licensing restrictions prohibit us from redistributing geolocation data for blocks, which we obtain from Maxmind. If such data is required, users can download it from MaxMind themselves and use code published by ANT: https://ant.isi.edu/software/outage_adaptive_geolocation/index.html to generate such mappings. Data Format Each included mapping file is a compressed FSDB file described by the following schemas: Block to AS mapping IP to AS mapping is obtained from cymru block_to_as.fsdb.bz2, format: #fsdb -F t block IP AS ASname 01000400 1.0.4.0 56203 BIGRED-NET-AU Big Red Group,AU 01000500 1.0.5.0 56203 BIGRED-NET-AU Big Red Group,AU 01002600 1.0.38.0 24155 APBW-AS-TW Asia Pacific Broadband Wireless Communications Inc,TW 01002b00 1.0.43.0 - - 01002e00 1.0.46.0 - - 01003000 1.0.48.0 - - 01005000 1.0.80.0 18144 AS-ENECOM Energia Communications,Inc.,JP 01005100 1.0.81.0 18144 AS-ENECOM Energia Communications,Inc.,JP 01005200 1.0.82.0 18144 AS-ENECOM Energia Communications,Inc.,JP • block: is each /24, in hex format, with trailing zeros • IP: is the block in dotted-quad format (this field is redundant) • AS: is the AS number • ASName: is the AS name Block to organization mapping We map block to organization. An organization could be an ISP or a company. We use Caida AS to organization and block_to_org_caida.fsdb.bz2, format: #fsdb -F t block IP org_id org_name country 01000400 1.0.4.0 56203 @aut-56203-APNIC Big Red Group AU 01000500 1.0.5.0 56203 @aut-56203-APNIC Big Red Group AU 01002600 1.0.38.0 24155 @aut-24155-APNIC Asia Pacific Broadband Wireless Communications Inc TW 01002b00 1.0.43.0 - - - - 01002e00 1.0.46.0 - - - - 01003000 1.0.48.0 - - - - 01005000 1.0.80.0 18144 @aut-7670-JPNIC Energia Communications, Inc. JP 01005100 1.0.81.0 18144 @aut-7670-JPNIC Energia Communications, Inc. JP 01005200 1.0.82.0 18144 @aut-7670-JPNIC Energia Communications, Inc. JP • block: is hex format with trailing zeros • IP: is dotted-quad format • org_id: is the organization id from Caida dataset. • org_name:is the organization name according to Caida dataset. • country: is the country where the organization is from , according to Caida. block_to_org_ant.fsdb.bz2, format: #fsdb -F t block IP org_id org_name country 01000400 1.0.4.0 - - - - 01000500 1.0.5.0 - - - - 01002600 1.0.38.0 24155 JLB5 Universidad Nacional del Litoral - 01002b00 1.0.43.0 - - - - 01002e00 1.0.46.0 - - - - 01003000 1.0.48.0 - - - - 01005000 1.0.80.0 - - - - 01005100 1.0.81.0 - - - - 01005200 1.0.82.0 - - - - • block: is hex format with trailing zeros • IP: is dotted-quad format • org_id: is the organization id from Ant dataset. • org_name:is the organization name according to Ant dataset. • country: is disabled in this dataset Block to Linktype • DNS reverse names are obtained from block_to_linktype.fsdb.gz format: #fsdb -F t block sta dyn srv rtr gw dhcp pool dsl dial ppp cable ded biz res client sql wireless wifi features num_features 01000400 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 252 srv 1 01000500 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 242 srv 1 • block:is hex-encoded for each /24, with trailing zeros • sta...wifi are counts of those keywords in the reverse DNS names of all 256 addresses in the block • features are the features that pass the algorithm described in section 2.3.3 of ISI-TR-2014-691 (features with non-zero counts, supprsing those less than 1/15 the most common feature) • multiple features are separated by the underscore character • warning: this approach overcounts the "gw" and "rtr" features for blocks with 3 gw names but no other common naming pattern • num_features is the count of how many features were identified by the above algorithm Block to RDNS Block to RNDS data is obtained from rapid7 (https://opendata.rapid7.com/sonar.rdns_v2/) and has the following format: #fsdb -F t block hexip answer 01000000 01000001 one.one.one.one 01000400 01000404 ns1.gtelecom.com.au 01000500 01000505 ns2.gtelecom.com.au 01000600 01000606 ns3.gtelecom.com.au The dns domain name for the lowest resolvable IP address (hexip) from the block (block) is listed (answer) for each block. Collection Method All included mappings were compiled from public sources. Citation If you use this trace to conduct additional research, please cite it as: Internet Outages Dataset Supplement, PREDICT ID: USC-LANDER/internet_outage_adaptive_a37_supplement-20190701. Provided by the USC/LANDER project http://www.isi.edu/ant/lander. User Annotations Currently no annotations. Categories: • Datasets • LANDER • LANDER:Datasets • LANDER:Datasets:AddressSpace:Adaptive Supplement • LANDER:Datasets:AddressSpace