LANDER:CloudAvailability-20130311 From Predict README version: 3793, last modified: 2014-02-13. This file describes the trace dataset "CloudAvailability-20130311", generated on 2014-02-11, provided by the LANDER project. Contents • 1 LANDER Metadata • 2 Dataset Contents • 3 Probing Method • 4 Data Format • 5 Citation • 6 User Annotations LANDER Metadata ┌───────────────────────────┬────────────────────────────────────────────────────────────────────────────────────┐ │ dataSetName │ CloudAvailability-20130311 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ status │ usc-web-and-predict │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ shortDesc │ Cloud Availability │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ longDesc │ Measure the availability of cloud services with different probing method (ICMP vs. │ │ │ HTTP) │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ datasetClass │ Quasi-Restricted │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ commercialAllowed │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ requestReviewRequired │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ productReviewRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ ongoingMeasurement │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ submissionMethod │ Upload │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartDate │ 2013-03-11 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndDate │ 2013-09-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartDate │ 2017-05-24 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndDate │ 2030-01-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ anonymization │ none │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ archivingAllowed │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ keywords │ category:internet-topology-data, subcategory:anycast-enumeration, cloud │ │ │ availability, end-to-end. │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ format │ text │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ access │ https │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ hostName │ USC-LANDER │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ providerName │ USC │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingId │ │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingSummaryFlag │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ retrievalInstructions │ download │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ byteSize │ 922746880 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ expirationDays │ 14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ uncompressedSize │ 8833980775 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ impactDoi │ 10.23721/109/1358229 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ useAgreement │ dua-ni-160816 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ irbRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ privateAccessInstructions │ See http://www.isi.edu/ant/traces/index.html#getting_datasets for information on │ │ │ obtaining this dataset. │ │ │ See │ └───────────────────────────┴────────────────────────────────────────────────────────────────────────────────────┘ Dataset Contents This dataset contains the cloud availability measurement data collected between 20130311 and 20130901, which is devided into two parts: 20130311_20130419, 20130618_20130901 (the difference between the two parts are explained in the "Collection Method" section). CloudAvailability.README.txt      copy of this README kilobyte_url URLs file 20130311_20130419/YYYYMMDDhhmmss/*.tar.gz: probing data collected by PL nodes 20130618_20130901/YYYYMMDDhhmmss/*.tar.gz probing data collected by PL nodes "kilobyte_url" contains a list of URLs which point to the objects we put in different cloud servers. "20130311_20130419/YYYYMMDDhhmmss/*.tar.gz" and "20130618_20130901/YYYYMMDDhhmmss/*.tar.gz" are the probing data we collected from different PL nodes. Our central server fetches data from all PL nodes at around 10:00pm PDT every day: "YYYYMMDDhhmmss" records the start time of data fetching. E.g. "20130311220001" means the central server started fetching data on 20130311 at 22:00:01. Each "*.tar.gz" file is named with the IP of the PL node that was collecting data. E.g. "132.170.3.32.tar.gz" was collected by the PL node with IP address "132.170.3.32". Probing Method We probe each of our target with both ICMP and HTTP. The file "kilobyte_url" contains a list of URLs which point to the objects we put in different cloud servers. each PL node periodically fetchs those objects every 10 or 11 minutes by HTTP via the program CURL (result: kilobyte_url-result.fsdb). each PL node also pings the hostname of the URL at the same frequency (result: ping_data.fsdb). We do immediate retries on failure, however the mechanism is different between the data in "20130311_20130419" and the data in "20130618_20130901": For the dataset "20130311_20130419", ICMP retries are conditioned on HTTP failure. when the HTTP probe fails, the node does 2 immediate retries for both the HTTP and ICMP probes. For the dataset "20130618_20130901", ICMP retries and HTTP retries are independent with each other. HTTP failures only cause HTTP retries, while ICMP failures only cause ICMP retires. And we do 8 immediate retries here. Moreover, each PL node also does both TCP and ICMP traceroute to the hostname in each URL at the same frequency as the ICMP/HTTP probing. A full description of this method is in: > Zi Hu, Liang Zhu, Calvin Ardi, Ethan Katz-Bassett, Harsha Madhyastha, John Heidemann, Minlan Yu. Passive and > Active Measurements Conference (PAM). Los Angeles, March 2014. > http://www.isi.edu/~zihu/paper/cloud_availability.pdf Data Format Each ".tar.gz" file contains four text files: kilobyte_url-result.fsdb, ping_data.fsdb, tcp_trace_data.fsdb, trace_data.fsdb All data are recorded in flat ASCII format. "kilobyte_url-result.fsdb" contains the HTTP fetching result recorded by CURL. sample data: #fsdb time_stamp location http_code http_connect num_connects num_redirects size_download size_header size_request size_upload speed_download speed_upload time_connect time_namelookup time_pretransfer time_redirect time_starttransfer time_total Fri-May--3-05:00:07-UTC-2013 Amazon-APNorth-kilobyte 200 000 1 0 1024 360 219 0 1011.000 0.000 0.397 0.200 0.804 0.000 1.012 1.012 Fri-May--3-05:00:08-UTC-2013 Amazon-APSouth-kilobyte 200 000 1 0 1024 360 219 0 1161.000 0.000 0.325 0.200 0.726 0.000 0.881 0.882 "ping_data.fsdb" contains the ICMP probing result recorded by the default ping program on linux. #fsdb time_stamp location host rtt Fri-May--3-05:00:22-UTC-2013 Amazon-APNorth-kilobyte s3-ap-northeast-1.amazonaws.com 194.727 Fri-May--3-05:00:22-UTC-2013 AmazonAmazon-APSouth-kilobyte s3-ap-southeast-1.amazonaws.com 124.976 "tcp_trace_data.fsdb" contains the TCP traceroute result recorded by the default traceroute program on linux, with an additional information line at the beginning of each record, which records the timestamp, location and host for that traceroute record. sample data: #Fri-May--3-05:00:58-UTC-2013 Amazon-APNorth-kilobyte s3-ap-northeast-1.amazonaws.com traceroute to s3-ap-northeast-1.amazonaws.com (27.0.2.76), 30 hops max, 60 byte packets 1 192-107-171-158.network.waikato.ac.nz (192.107.171.158) 0.149 ms 2 192-107-171-49.network.waikato.ac.nz (192.107.171.49) 0.786 ms 3 pegasus.network.waikato.ac.nz (130.217.2.6) 0.899 ms 4 114-134-15-62.rurallink.co.nz (114.134.15.62) 0.441 ms 5 TenGigabitEthernet0-2-0-830.akkin-rt1.fx.net.nz (131.203.117.9) 2.921 ms 6 ten-0-3-0-127.bdr02.akl02.akl.VOCUS.net.au (114.31.203.137) 3.621 ms 7 ten-0-2-0-300.bdr02.akl01.akl.VOCUS.net.au (114.31.202.40) 123.237 ms 8 ten-0-0-0-2.cor03.syd03.nsw.VOCUS.net.au (114.31.199.116) 123.235 ms 9 ten-0-0-0-0.cor01.per02.wa.VOCUS.net.au (114.31.206.127) 123.205 ms 10 ten-0-1-0-1.cor01.per02.wa.VOCUS.net.au (114.31.206.81) 123.209 ms 11 ge-0-1-6.bdr01.per02.wa.VOCUS.net.au (114.31.206.91) 123.150 ms 12 pos-0-3-0.bdr01.sin01.sin.VOCUS.net (114.31.206.141) 122.981 ms 13 63-218-213-21.static.pccwglobal.net (63.218.213.21) 127.997 ms 14 63-218-251-54.static.pccwglobal.net (63.218.251.54) 194.509 ms 15 27.0.0.133 (27.0.0.133) 201.892 ms 16 27.0.0.161 (27.0.0.161) 254.266 ms 17 27.0.2.76 (27.0.2.76) 195.130 ms "trace_data.fsdb" contains the ICMP traceroute result recorded by the default traceroute program on linux, with an additional information line at the beginning of each record, which records the timestamp, location and host for that traceroute record. sample data: #Fri-May--3-05:00:57-UTC-2013 Amazon-APNorth-kilobyte s3-ap-northeast-1.amazonaws.com traceroute to s3-ap-northeast-1.amazonaws.com (27.0.2.76), 30 hops max, 60 byte packets 1 192-107-171-158.network.waikato.ac.nz (192.107.171.158) 0.185 ms 2 192-107-171-49.network.waikato.ac.nz (192.107.171.49) 0.602 ms 3 pegasus.network.waikato.ac.nz (130.217.2.6) 3.805 ms 4 114-134-15-62.rurallink.co.nz (114.134.15.62) 0.466 ms 5 TenGigabitEthernet0-2-0-830.akkin-rt1.fx.net.nz (131.203.117.9) 2.789 ms 6 ten-0-3-0-127.bdr02.akl02.akl.VOCUS.net.au (114.31.203.137) 3.727 ms 7 ten-0-2-0-300.bdr02.akl01.akl.VOCUS.net.au (114.31.202.40) 123.198 ms 8 ten-0-0-0-2.cor03.syd03.nsw.VOCUS.net.au (114.31.199.116) 123.526 ms 9 ten-0-0-0-0.cor01.per02.wa.VOCUS.net.au (114.31.206.127) 123.384 ms 10 ten-0-1-0-1.cor01.per02.wa.VOCUS.net.au (114.31.206.81) 123.524 ms 11 ge-0-1-6.bdr01.per02.wa.VOCUS.net.au (114.31.206.91) 123.305 ms 12 pos-0-3-0.bdr01.sin01.sin.VOCUS.net (114.31.206.141) 123.293 ms 13 63-218-213-21.static.pccwglobal.net (63.218.213.21) 127.174 ms 14 63-218-251-54.static.pccwglobal.net (63.218.251.54) 194.376 ms 15 27.0.0.133 (27.0.0.133) 201.209 ms 16 27.0.0.161 (27.0.0.161) 193.930 ms 17 27.0.2.76 (27.0.2.76) 194.631 ms Citation If you use this trace to conduct additional research, please cite it as: Cloud Availability Measurement Dataset, PREDICT ID: USC-LANDER/CloudAvailability-20130311/rev3793. Provided by the USC/LANDER project (http://www.isi.edu/ant/lander). User Annotations Currently no annotations. Categories: • Datasets • LANDER • LANDER:Datasets • LANDER:Datasets:AddressSpace • LANDER:Datasets:AddressSpace:CloudAvailability