LANDER:NCCDC logs zeek-20170413 From Predict Jump to navigation Jump to search The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead. README version: 10749, last modified: 2019-10-30. This file describes the trace dataset "NCCDC_logs_zeek-20170413" provided by the LANDER project. The most recent version of this file can be found on-line at https://wiki.isi.edu/predict/index.php?title=LANDER:NCCDC_logs_zeek-20170413. [ ] Contents • 1 LANDER Metadata • 2 Dataset Contents • 3 Dataset Generation • 3.1 Background • 4 Citation • 5 Results Using This Dataset • 6 User Annotations LANDER Metadata (https://wiki.isi.edu/predict/index.php?title=LANDER:NCCDC_logs_zeek-20170413/landermeta) ┌───────────────────────────┬────────────────────────────────────────────────────────────────────────────────────┐ │ dataSetName │ NCCDC_logs_zeek-20170413 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ status │ usc-web-and-predict │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ shortDesc │ 2017 NC Cyber Defense Competition, Zeek output │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ longDesc │ This dataset is Zeek output from packet captures from the 2017 National Collegiate │ │ │ Cyber Defense Competition (nccdc.org). CCDC is a multi-day competition that │ │ │ specifically focuses on the operational aspects of managing and protecting an │ │ │ existing commercial" network infrastructure. Teams of undergraduate/graduate │ │ │ students are provided with a fully functional (but insecure) small business │ │ │ network they must secure; maintain; and defend against a live Red Team. Teams must │ │ │ also respond to business tasks called "injects" throughout the competition. Zeek │ │ │ is a popular network monitoring tool. │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ datasetClass │ Quasi-Restricted │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ commercialAllowed │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ requestReviewRequired │ true │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ productReviewRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ ongoingMeasurement │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ submissionMethod │ Upload │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartDate │ 2017-04-13 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionStartTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndDate │ 2017-04-15 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ collectionEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartDate │ 2019-11-11 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityStartTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndDate │ 2030-01-01 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ availabilityEndTime │ 00:00:00 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ anonymization │ none │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ archivingAllowed │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ keywords │ category:synthetically-generated-data, subcategory:experimental-data, synthetic │ │ │ data, nccdc, zeek │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ format │ csv │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ access │ https │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ hostName │ USC-LANDER │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ providerName │ USC │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingId │ │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ groupingSummaryFlag │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ retrievalInstructions │ download │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ byteSize │ 42308993024 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ expirationDays │ 14 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ uncompressedSize │ 279818144724 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ impactDoi │ 10.23721/115/1364431 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ useAgreement │ dua-ni-160816 │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ irbRequired │ false │ ├───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────┤ │ privateAccessInstructions │ See http://www.isi.edu/ant/traces/index.html#getting_datasets for information on │ │ │ obtaining this dataset. │ │ │ See https://wiki.isi.edu/predict/index.php?title=LANDER:NCCDC_logs_zeek-20170413 │ │ │ for details on this dataset. │ └───────────────────────────┴────────────────────────────────────────────────────────────────────────────────────┘ Dataset Contents NCCDC_logs_zeek-20170413.README.txt     copy of this README dayone/     nccdc_2017_dayone_zeek_csv_logs/ zeek output in csv format from the first day of competition         conn.log.xz, conn_long.log.xz, etc. specific logs     nccdc_2017_dayone_zeek_extracted_files.zip 73,647 files extracted by zeek from the first day of competition     nccdc_2017_dayone_zeek_json_logs/ zeek output in JSON format from the first day of competition     nccdc_2017_daytwo_zeek_csv_logs/ zeek output in csv format from the first day of competition     nccdc_2017_daytwo_zeek_json_logs/ zeek output in JSON format from the first day of competition     .sha1sum SHA-1 checksum daytwo/     daytwo.NNN.pcap.xz sequentially numbered pcap traces from the second day of competition     .sha1sum SHA-1 checksum The file ".sha1sum" contains SHA1 checksums of individual compressed files. The integrity of the distribution thus can be checked by independently calculating SHA1 sums of files and comparing them with those listed in the file. If you have the sha1sum utility installed on your system, you can do that by executing: sha1sum --check .sha1sum This has to be done before files are uncompressed. Dataset Generation Background This dataset contains Zeek (Bro) output from the pcaps processed in the 2017 National Collegiate Cyber Defense Competition (NCCDC) held in April in San Antonio, TX (http://www.nccdc.org), as present in the dataset LANDER:NCCDC_logs-20170413 That dataset contained packet captures from a simulated network attack and defense scenario. Please see that dataset for details about the scenario, network topology, capture mechanism. Zeek processing was done by Benjamin Bornholm from RIT as part of his Masters Degree. He says: I generated CSV logs, JSON logs, and extracted all known file types. The CSV logs zipped up is 22GBs, the JSON logs zipped up is 25GBs, and the extracted files are 8GBs zipped up. If you would prefer the logs in raw text (no compression) it will be about ~200Gbs of space per format (JSON or CSV), ~400GBs for both format To convert the data set I setup Zeek to listen on a dummy Linux network interface and then used tcpdreplay to pipe data to that interface. This allowed me to play the PCAP in real time to preserve PCAP timestamps. I also used some Zeek scripts from the internet and the ZIPs should contain a Zeek_script_loaded.log (or similar name) to indicate which Zeek scripts I used. A typical zeek command line (from daytwo csv): zeek -i dummy0 -U .status -p zeekctl -p zeekctl-live -p standalone -p local -p bro local.zeek zeekctl zeekctl/standalone zeekctl/auto Citation If you use this trace to conduct additional research, please cite it as: NCCDC Logs Zeek, IMPACT ID: USC-LANDER/NCCDC_logs_zeek-20170413/rev10749 . Traces taken 2017-04-13 to 2017-04-25. Traces provided by the Center for Infrastructure Assurance and Security (UTSA/CIAS); Zeek evaluation 2019-08 by Benjamin Bornholm, RIT; data hosted by the USC/LANDER project (http://www.isi.edu/ant/lander). Results Using This Dataset No results yet. User Annotations Suggestion: Edit the annotations at https://wiki.isi.edu/predict/index.php?title=LANDERNOTES:NCCDC_logs_zeek-20170413&action=edit Currently no annotations. Retrieved from "https://wiki.isi.edu/predict/index.php?title=LANDER:NCCDC_logs_zeek-20170413&oldid=10749" Categories: • LANDER • LANDER:Datasets • LANDER:Datasets:PCH • LANDER:Datasets:NCCDCLogs • Datasets Navigation menu Personal tools • Wikiexport • Talk • Preferences • Watchlist • Contributions • Log out Namespaces • LANDER • Discussion [ ] English Views • Read • Edit • View history • Watch [ ] More • Move _____________________ [ Search ] [ Go ] Navigation • Main page • Providers • Datasets • Results • Categories • Recent changes • Random page • Help Tools • What links here • Related changes • Upload file • Special pages • Permanent link • Page information • This page was last edited on 30 October 2019, at 09:02. • Content is available under Attribution-Share Alike 3.0 Unported unless otherwise noted. • Privacy policy • About Predict • Disclaimers • Attribution-Share Alike 3.0 Unported • Powered by MediaWiki