Tag: lacrend

Categories
Publications Technical Report

new technical report “Detecting ICMP Rate Limiting in the Internet”

We have released a new technical report “Detecting ICMP Rate Limiting in the Internet” as an ISI technical report ISI-TR-717.

From the abstract of our technical report:

Comparing model and experimental effects of rate limiting (Figure 2.a from [Guo17a] )

Active probing with ICMP is the center of many network measurements, with tools like ping, traceroute, and their derivatives used to map topologies and as a precursor for security scanning. However, rate limiting of ICMP traffic has long been a concern, since undetected rate limiting to ICMP could distort measurements, silently creating false conclusions. To settle this concern, we look systematically for ICMP rate limiting in the Internet. We develop a model for how rate limiting affects probing, validate it through controlled testbed experiments, and create FADER, a new algorithm that can identify rate limiting from user-side traces with minimal requirements for new measurement traffic. We validate the accuracy of FADER with many different network configurations in testbed experiments and show that it almost always detects rate limiting. Accuracy is perfect when measurement probing ranges from 0 to 60 times the rate limit, and almost perfect (95%) with up to 20% packet loss. The worst case for detection is when probing is very fast and blocks are very sparse, but even there accuracy remains good (measurements 60 times the rate limit of a 10% responsive block is correct 65% of the time). With this confidence, we apply our algorithm to a random sample of whole Internet, showing that rate limiting exists
but that for slow probing rates, rate-limiting is very, very rare. For our random sample of 40,493 /24 blocks (about 2\% of the responsive space), we confirm 6 blocks (0.02%!) see rate limiting
at 0.39 packets/s per block. We look at higher rates in public datasets
and suggest that fall-off in responses as rates approach 1 packet/s per /24 block (14M packets/s from the prober to the whole Internet),
is consistent with rate limiting. We also show that even very slow probing (0.0001 packet/s) can encounter rate limiting of NACKs that are concentrated at a single router near the prober.

Datasets we used in this paper are all public. ISI Internet Census and Survey data (including it71w, it70w, it56j, it57j and it58j census and survey) are available at https://ant.isi.edu/datasets/index.html. ZMap 50-second experiments data are from their WOOT 14 paper and can be obtained from ZMap authors upon request.

This technical report is joint work of Hang Guo and  John Heidemann from USC/ISI.

Categories
Papers Publications

new conference paper “Does Anycast Hang up on You?” in TMA 2017

The paper “Does Anycast hang up on you?” will appear in the 2017 Conference on Network Traffic Measurement and Analysis (TMA) July 21-23, 2017 in Dublin, Ireland.

In each anycast-based DNS root service, there are about 1% VPs see a route flip happens every one or two observation during a week with an observation interval as 4 minutes. (Figure 2 from [Wei17b]).
From the abstract:

Anycast-based services today are widely used commercially, with several major providers serving thousands of important websites. However, to our knowledge, there has been only limited study of how often anycast fails because routing changes interrupt connections between users and their current anycast site. While the commercial success of anycast CDNs means anycast usually work well, do some users end up shut out of anycast? In this paper we examine data from more than 9000 geographically distributed vantage points (VPs) to 11 anycast services to evaluate this question. Our contribution is the analysis of this data to provide the first quantification of this problem, and to explore where and why it occurs. We see that about 1\% of VPs are anycast unstable, reaching a different anycast site frequently (sometimes every query). Flips back and forth between two sites in 10 seconds are observed in selected experiments for given service and VPs. Moreover, we show that anycast instability is persistent for some VPs—a few VPs never see a stable connections to certain anycast services during a week or even longer. The vast majority of VPs only saw unstable routing towards one or two services instead of instability with all services, suggesting the cause of the instability lies somewhere in the path to the anycast sites. Finally, we point out that for highly-unstable VPs, their probability to hit a given site is constant, which means the flipping are happening at a fine granularity—per packet level, suggesting load balancing might be the cause to anycast routing flipping. Our findings confirm the common wisdom that anycast almost always works well, but provide evidence that a small number of locations in the Internet where specific anycast services are never stable.

This paper is joint work of Lan Wei and John Heidemann.  A pre-print of paper is at http://ant.isi.edu/~johnh/PAPERS/Wei17b.pdf, and the datasets from the paper are at https://ant.isi.edu/datasets/anycast/index.html#stability.

Categories
Papers Publications

new conference paper “Do You See Me Now? Sparsity in Passive Observations of Address Liveness” in TMA 2017

The paper “Do You See Me Now? Sparsity in Passive Observations of Address Liveness” will appear in the 2017 Conference on Network Traffic Measurement and Analyais (TMA) July 21-23, 2017 in Dublin, Ireland.   The datasets from the paper that we can make public will be at https://ant.isi.edu/datasets/sparsity/.

Visibility of addresses and blocks from possible /24 virtual monitors (Figure 2 from [Mirkovic17a])
From the abstract of the paper:

Accurate information about address and block usage in the Internet has many applications in planning address allocation, topology studies, and simulations. Prior studies used active probing, sometimes augmented with passive observation, to study macroscopic phenomena, such as the overall usage of the IPv4 address space. This paper instead studies the completeness of passive sources: how well they can observe microscopic phenomena such as address usage within a given network. We define sparsity as the limitation of a given monitor to see a target, and we quantify the effects of interest, temporal, and coverage sparsity. To study sparsity, we introduce inverted analysis, a novel approach that uses complete passive observations of a few end networks (three campus networks in our case) to infer what of these networks would be seen by millions of virtual monitors near their traffic’s destinations. Unsurprisingly, we find that monitors near popular content see many more targets and that visibility is strongly influenced by bipartite traffic between clients and servers. We are the first to quantify these effects and show their implications for the study of Internet liveness from passive observations. We find that visibility is heavy-tailed, with only 0.5% monitors seeing more than 10\% of our targets’ addresses, and is most affected by interest sparsity over temporal and coverage sparsity. Visibility is also strongly bipartite. Monitors of a different class than a target (e.g., a server monitor observing a client target) outperform monitors of the same class as a target in 82-99% of cases in our datasets. Finally, we find that adding active probing to passive observations greatly improves visibility of both server and client target addresses, but is not critical for visibility of target blocks. Our findings are valuable to understand limitations of existing measurement studies, and to develop methods to maximize microscopic completeness in future studies.

Categories
Announcements Collaborations Papers

best paper award at PAM 2017

The PAM 2017 best paper award for “Anycast Latency: How Many Sites Are Enough?”

Congratulations to Ricardo de Oliveira Schmidt (U. Twente), John Heidemann (USC/ISI), and Jan Harm Kuipers (U. Twente) for the award of  best paper at the Conference on Passive and Active Measurement (PAM) 2017 to their paper “Anycast Latency: How Many Sites Are Enough?”.

See our prior blog post for more information about the paper and its data, and the U. Twente blog post about the paper and the SIDN Labs blog post about the paper.

Categories
Announcements Data Internet

ANT IPv4 census appears in Library of Congress Blog on Innovative Mapping

John Hessler, a member of the US Library of Congress’ Geography and Map Division wrote a nice blog post about our IPv4 Internet maps: “Computing Space V: Mapping the Web or Pinging your Way to Infinity“.  Check out his take on our IPv4 data!

You too can browse the IPv4 Internet at our website.  Or for detailed analysis, get the data from IMPACT or us.

Thanks to the DHS IMPACT program for supporting collection of this data.

Categories
Presentations

new talk “Infrastructure for Experimental Replay and Mutation of DNS Queries” at the AIMS Workshop 2017

John Heidemann gave the talk “Infrastructure for Experimental Replay and Mutation of DNS Queries” at CAIDA’s Active Internet Measurement (AIMS) Workshop in San Diego, California, USA on March 2, 2017.  Slides are available at http://www.isi.edu/~johnh/PAPERS/Zhu17a.pdf.
From the abstract:

Emulating the DNS hierarchy both efficiently and correctly.

The DNS ecosystem today is revisiting basic design questions: should it encourage TCP? TLS? DTLS? Something completely new like QUIC or HTTP? While modeling and analysis help answer some of these questions, experimental evaluation is necessary for validation, and in some cases the only way to get accurate estimates of software memory use and performance. This talk will discuss our recent work in supporting experimental evaluation of DNS with components that support trace replay and evaluation. Trace replay is supported by a DNS data archive to prime replay with real data, and a query mutation system to support what-if evaluation using variations of that data.

The trace replay system is the work with Liang Zhu; this work is part of a larger system to support DNS experimentation, joint work with Wes Hardaker.

The software discussed in the talk is available at https://ant.isi.edu/software/ldplayer, and this work is part of our progress towards the NIPET testbed.

 

Categories
Presentations

new talk “Collecting and Visualizing Outages Over the Long Haul” at the AIMS Workshop 2017

John Heidemann gave the talk “Collecting and Visualizing Outages Over the Long Haul” at CAIDA’s Active Internet Measurement (AIMS) Workshop in San Diego, California, USA on March 2, 2017.  Slides are available at http://www.isi.edu/~johnh/PAPERS/Heidemann17b.pdf.
From the abstract:

Unmeasurable blocks over time, a challenge in long-haul outage measurement, from [Alwabel15a]
We have been collecting data about outages in the Internet since Oct. 2014. Our outage detection system, Trinocular, uses active probing from four sites to study about 4 million /24 IPv4 address blocks. Long-duration measurements bring challenges that don’t occur in short observations. Most importantly, our target (“the Internet”) changes as we measure it, as new blocks come on-line, old blocks are reused in different ways, and ISPs observe and sometimes block our traffic. Our measurement platform also sees occasional hardware failures. Visualization can assist detection of these problems, allowing human perception to detect changes in data collection that have not previously been anticipated. This talk will discuss the challenges of long-term outage measurement and describe our new algorithm that scales to support clustering of 4M blocks and 3 months of observations for visualization.
Our visualization is joint work with Yuri Pradkin, and analysis of our long-term outages includes work with Abdulla Alwabel.

This talk draws on work from [Alwabel15a].  Data from this talk is available at https://ant.isi.edu/datasets/outage/, and visualizations can be found at https://ant.isi.edu/outage/browse/.

Categories
Presentations

new talk “DNS Privacy, Service Management, and Research: Friends or Foes” at the NDSS DNS Privacy Workshop 2017

John Heidemann gave the talk “DNS Privacy, Service Management, and Research: Friends or Foes” at the NDSS DNS Privacy Workshop in San Diego, California, USA on Feburary 26, 2017.  Slides are available at http://www.isi.edu/~johnh/PAPERS/Heidemann17a.pdf.
The talk does not have a formal abstract, but to summarize:

A slide from the [Heidemann17a] talk, looking at what different DNS stakeholders may want.
A slide from the [Heidemann17a] talk, looking at what different DNS stakeholders may want.

This invited talk is part of a panel on the tension between DNS privacy and service management.  In the talk I expand on that topic and discuss
the tension between DNS privacy, service management, and research.
I give suggestions about how service management and research can adapt to proceed while still providing basic privacy.

Although not discussed in the talks, we distribute some DNS datasets,  available at https://ant.isi.edu/datasets/ and at https://impactcybertrust.org.  We also provide dnsanon, a tool to anonymize DNS queries.

Categories
Publications Technical Report

new technical report “Does Anycast hang up on You? (extended)”

We have released a new technical report “Does Anycast hang up on you?(extended)”, ISI-TR-716, available at http://www.isi.edu/~weilan/PAPER/anycast_instability.pdf

From the abstract:

In each anycast-based DNS root service, there are about 1% VPs see a route flip happens every one or two observation during a week with an observation interval as 4 min.

Anycast-based services today are widely used commercially, with several major providers serving thousands of important websites. However, to our knowledge, there has been only limited study of how often anycast fails because routing changes interrupt connections between users and their current anycast site. While the commercial success of anycast CDNs means anycast usually work well, do some users end up shut out of anycast? In this paper we examine data from more than 9000 geographically distributed vantage points (VPs) to 11 anycast services to evaluate this question. Our contribution is the analysis of this data to provide the first quantification of this problem, and to explore where and why it occurs. We see that about 1% of VPs are anycast unstable, reaching a different anycast site frequently sometimes every query. Flips back and forth between two sites in 10 seconds are observed in selected experiments for given service and VPs.
Moreover, we show that anycast instability is persistent for some VPs—a few VPs never see a stable connections to certain anycast services during a week or even longer. The vast majority of VPs only saw unstable routing towards one or two services instead of instability with all services, suggesting the cause of the instability lies somewhere in the path to the anycast sites. Finally, we point out that for highly-unstable VPs, their probability to hit a given site is constant, which means the flipping are happening at a fine granularity —per packet level, suggesting load balancing might be the cause to anycast routing flipping. Our findings confirm the common wisdom that anycast almost always works well, but provide evidence that a small number of locations in the Internet where specific anycast services are never stable.

This technical report is joint work of  Lan Wei,  John Heidemann, from USC/ISI.

Categories
Software releases

mtracecap: New utility for multi-point capture released

mtracecap v0.1 (beta) has been released (available at https://ant.isi.edu/software/mtracecap/index.html)

This tool is designed to capture packets from multiple sources and write its output to a single file.  Its build requires a local install of libtrace library (version 4.0 or older) and supports all sources supported by the library, such as pcap based interfaces, linux-specific ring interfaces, pcap and erf outputs and many more!  See them all listed when you run mtracecap with -H option.  DAG device capture is optional, depending on local DAG libraries being present.

An important feature of this tool is being able to roll output into multiple files either based on either maximum file size (e.g.  “-S 100” option will make it write output in 100MB chunks), or system time (e.g. “-G 180” option will rotate output every 180 seconds).

Finally, the tool can use external commands to work on the input before writing it to a file using a pipe (see –pipeout option).  This can be useful if you want to compute some statistics on the fly or compress output using an external compressor.  Using this option will eliminate extra disk read-write operations if all you want to do is to compress the output.