The ANT Stream Merger Tool

stream_merger

Stream merger is a tool to merge multiple traffic streams by feeding them through a FIFO/Drop tail queue and adjusting packet timing due to queueing. Its input is several packet trace files. The output is a single merged packet trace.

Execution

merger Number of input ports Buffer size of output port in kBytes Transmission rate of output port in Mb/s

Input traffic trace file format

  • name: resPORT.in (where PORT is the port number starting from 0)
  • Format: timestamp pktid srcaddr srcport dstaddr dstport size – pktid: Unique number identifying each packet of a trace file. – ipport: Input port that the packet arrived on. – timestamp: Arrival time of packet to queue.

Output traffic trace file format

  • Name: res0.out
  • Format: timestamp ipport:pktid srcaddr srcport dstaddr dstport size – timestamp: Time when the packet reaches the end of the output link – ipport: Input port that the packet arrived on

Note: the timestamp in the output file reflects when the packet reaches the end of the output link. i.e. the timestamp is the time when the router FINISHES outputting the packet

Authors

This code is primarily by Purushotham Kamath <pkamath@ISI.EDU> with contributions by Kun-chan Lan, Gautam Thatte, Sean McPherson.

References

This code has been used in the following papers:

  • Gautam Thatte, Urbashi Mitra and John Heidemann 2010. Parametric Methods for Anomaly Detection in Aggregate Traffic. ACM/IEEE Transactions on Networking. 19, 2 (Aug. 2010), 512–525. [DOI] [PDF] Details
  • Gautam Thatte, Urbashi Mitra and John Heidemann 2009. Parametric Methods for Anomaly Detection in Aggregate Traffic. Technical Report ISI-TR-2009-663b. USC/Information Sciences Institute. [PDF] Details
  • Gautam Thatte, Urbashi Mitra and John Heidemann 2008. Detection of Low-Rate Attacks in Computer Networks. Proceedings of the 11th IEEE Global Internet Symposium (Phoenix, Arizona, USA, Apr. 2008), 1–6. [DOI] [PDF] Details
  • Purushotham Kamath, Kun-chan Lan, John Heidemann, Joe Bannister and Joe Touch 2002. Generation of High Bandwidth Network Traffic Traces. Proceedings of the International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (Fort Worth, Texas, USA, Oct. 2002), 401–410. [PDF] Details

Datasets

It was used to generate the publicly available Internet traffic traces:

The traces listed below are synthetic Denial of Service attacks merged with real Internet traffic

  • UniformAttack_Traces_Generated20070821-20041202
  • UniformAttack_traces-20070115

The traces are described in detail here (http://www.isi.edu/ant/traces/dataset_list.html) and available through the PREDICT database.

Example from the Test Suite

The following demonstration of stream merger operation uses traces from the TEST_07 subdirectory of the Test Suite (v.0.2).

The input (res0.in, res1.in) and output (res0.out) files have been included in the stream_merger_0.4 subdirectory as well for reference.

The following test is processed by the stream merger application by issuing the following command (this assumes all res.in files are in the same directory as the merger application):

merger 2 1000 5

where 2 is the number of input ports (res0 and res1), 1000 kB is the output port buffer size, and the output line speed is 5 Mbps.

Example input and output files:

TEST 07

Both input line speeds at 5 Mbps each, and slightly staggered packets (have included random time increments into the second stream to cause overlapping)

output line speed set at 5 Mbps.

Input Trace Files

((stream01))

1.000000 1 1677277 1942 1677256 80 1500

1.004800 3 1677277 1942 1677256 80 40

1.004928 5 1677277 1942 1677256 80 40

1.005056 7 1677277 1942 1677256 80 800

1.007616 9 1677277 1942 1677256 80 1500

1.012416 11 1677277 1942 1677256 80 40

1.012544 13 1677277 1942 1677256 80 1500

((stream02))

1.002000 2 1677277 1942 1677256 80 1500

1.004850 4 1677277 1942 1677256 80 40

1.004950 6 1677277 1942 1677256 80 40

1.005750 8 1677277 1942 1677256 80 800

1.009016 10 1677277 1942 1677256 80 1500

1.012450 12 1677277 1942 1677256 80 40

1.014000 14 1677277 1942 1677256 80 1500</p>

Output Trace File

1.002400000000 0:1 1677277 1942 1677256 80 1500
1.004800000000 1:2 1677277 1942 1677256 80 1500
1.004864000000 0:3 1677277 1942 1677256 80 40
1.004928000000 1:4 1677277 1942 1677256 80 40
1.004992000000 0:5 1677277 1942 1677256 80 40
1.005056000000 1:6 1677277 1942 1677256 80 40
1.006336000000 0:7 1677277 1942 1677256 80 800
1.007616000000 1:8 1677277 1942 1677256 80 800
1.010016000000 0:9 1677277 1942 1677256 80 1500
1.012416000000 1:10 1677277 1942 1677256 80 1500
1.012480000000 0:11 1677277 1942 1677256 80 40
1.012544000000 1:12 1677277 1942 1677256 80 40
1.014944000000 0:13 1677277 1942 1677256 80 1500
1.017344000000 1:14 1677277 1942 1677256 80 1500

Matlab plot of the input traces and the combined output trace

matlab view of input and output