<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
<font face="Helvetica, Arial, sans-serif">Signal condtions were
excellent, 5 bars, line of sight out the window of a 6th floor hotel
room, clear weather.<br>
<br>
<br>
</font><br>
On 09/11/2009 09:24 AM, Stiliadis, Dimitrios (Dimitri) wrote:
<blockquote
cite="mid:F5EF891E30B2AE46ACA20EB848689C2124EDBA4085@USNAVSXCHMBSA3.ndc.alcatel-lucent.com"
type="cite">
<pre wrap="">btw, the bimodal can be also explained if your card is attaching
to two different cells (or bands), depending on signal conditions/fading etc.
(i.e. it oscillates between cell A and cell B or band A and band B).
Also, it is not rare to see that the device will switch
between HSDPA and EDGE if the signal conditions on HSDPA are marginal.
Several tests have shown that EDGE delays of 500ms are normal.
In order to fully understand the issue, you need to also log all the
modem information (such as cell ID, band that it is attached etc.etc.),
or you need to perform the test in a anechoic chamber with test
equipment. Usually you can do that, by issuing the right AT commands
to the modem.
David:
As for the uplink bandwidth bottleneck, the problem there other than
larger router buffers is that ARQ interprets losses on the wire
as losses on the air, and tries retransmissions. Your picture
is not accurate without an RNC in the middle that implements some form
of ARQ/RLP
There are just two many things going on in the background, that pings
alone cannot show.
Cheers,
Dimitri
</pre>
<blockquote type="cite">
<pre wrap="">-----Original Message-----
From: <a class="moz-txt-link-abbreviated" href="mailto:end2end-interest-bounces@postel.org">end2end-interest-bounces@postel.org</a>
[<a class="moz-txt-link-freetext" href="mailto:end2end-interest-bounces@postel.org">mailto:end2end-interest-bounces@postel.org</a>] On Behalf Of
Dominik Kaspar
Sent: Thursday, September 10, 2009 8:38 PM
To: David P. Reed
Cc: <a class="moz-txt-link-abbreviated" href="mailto:end2end-interest@postel.org">end2end-interest@postel.org</a>
Subject: Re: [e2e] What's wrong with this picture?
Hi David,
Thanks for the explanations about the bottleneck link to the
backbone ISP. The illustrated system architecture and the
overuse of buffers certainly sounds like reasonable cause for
those huge delays you have posted at the beginning of this thread.
The "bimodal" behaviour of delays > 5000 ms and delays < 200
ms that you have measured is really extreme and it seems to
differ somewhat from what I have observed. In my experiments,
the delay abruptly switches between two rather stable
"modes"... sometimes every few minutes, sometimes just once a
day. It is completely unpredictable and I have not yet found
_the_ explanation for its cause. I doubt it has anything to
do with TCP... it seems much more likely to be one of the
HSDPA-specific properties that Detlef has pointed out (line
coding, MAC-layer ACKs, ...).
Here is the entire 24h ping log that clearly illustrates the
two "modes":
<a class="moz-txt-link-freetext" href="http://home.simula.no/~kaspar/static/ping-hsdpa-24h-bimodal-00.txt">http://home.simula.no/~kaspar/static/ping-hsdpa-24h-bimodal-00.txt</a>
Greetings,
Dominik
On Wed, Sep 9, 2009 at 1:07 AM, David P. Reed <a class="moz-txt-link-rfc2396E" href="mailto:dpreed@reed.com"><dpreed@reed.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I'm willing to bet that you are seeing the same problem I
</pre>
</blockquote>
<pre wrap="">am, and that
</pre>
<blockquote type="cite">
<pre wrap="">it has nothing to do with the modem or wireless protocol.
Instead you are seeing what would happen if you simulate in ns2 the
following system structure:
-------------------------\
--------------------------\
---------------------------\
wireless medium [WIRELESS
HUB]------[ROUTER]-----------backbone ISP
</pre>
</blockquote>
<pre wrap="">---------------------------/
</pre>
<blockquote type="cite">
<pre wrap="">--------------------------/
When the link between the ROUTER and backbone ISP is of
</pre>
</blockquote>
<pre wrap="">lower bitrate
</pre>
<blockquote type="cite">
<pre wrap="">B than the sum of all the realizable simultaneous uplink
</pre>
</blockquote>
<pre wrap="">demand from
</pre>
<blockquote type="cite">
<pre wrap="">devices on the left, the outbound queue of the router is of
</pre>
</blockquote>
<pre wrap="">size M >
</pre>
<blockquote type="cite">
<pre wrap="">BT where T is the observed stable long delay, and the ROUTER does
nothing to signal congestion until the entire M bytes (now
</pre>
</blockquote>
<pre wrap="">very large) of memory are exhausted.
</pre>
<blockquote type="cite">
<pre wrap="">
Memory is now very cheap, and not-very-clueful network layer 2
designers (who don't study TCP or the Internet) are likely to throw
too much at the problem without doing the right thing in
</pre>
</blockquote>
<pre wrap="">their firmware.
</pre>
<blockquote type="cite">
<pre wrap="">
On 09/08/2009 06:47 PM, Dominik Kaspar wrote:
Hello David,
You mentioned the bimodal behaviour of your 3G connection.
</pre>
</blockquote>
<pre wrap="">I recently
</pre>
<blockquote type="cite">
<pre wrap="">noticed the same thing but have not yet been able to
</pre>
</blockquote>
<pre wrap="">explain why this
</pre>
<blockquote type="cite">
<pre wrap="">happens.
I also ran Ping tests over multiple days using an HSDPA modem (with
both the client and server located in Oslo, Norway). The
</pre>
</blockquote>
<pre wrap="">experienced
</pre>
<blockquote type="cite">
<pre wrap="">RTTs were very stable over short periods of time, but
</pre>
</blockquote>
<pre wrap="">sometimes they
</pre>
<blockquote type="cite">
<pre wrap="">averaged around 80ms, while at other times the average was at about
300ms.
A CDF illustration of the results is available here:
<a class="moz-txt-link-freetext" href="http://home.simula.no/~kaspar/static/cdf-hsdpa-rtt-00.png">http://home.simula.no/~kaspar/static/cdf-hsdpa-rtt-00.png</a>
What is the reason of these two modes? Is it caused by adaptive
modulation and coding on the physical layer? If so, why
</pre>
</blockquote>
<pre wrap="">does it affect
</pre>
<blockquote type="cite">
<pre wrap="">the delay so much? I would only expect a reduced bandwidth, but not
much change in delay...
Greetings,
Dominik
On Tue, Sep 8, 2009 at 7:56 PM, David P.
</pre>
</blockquote>
<pre wrap="">Reed<a class="moz-txt-link-rfc2396E" href="mailto:dpreed@reed.com"><dpreed@reed.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
I should not have been so cute - I didn't really want to
</pre>
</blockquote>
<pre wrap="">pick on the
</pre>
<blockquote type="cite">
<pre wrap="">operator involved, because I suspect that other 3G operators around
the world probably use the same equipment and same rough
</pre>
</blockquote>
<pre wrap="">configuration.
</pre>
<blockquote type="cite">
<pre wrap="">
The ping and traceroute were from Chicago, using an ATT
</pre>
</blockquote>
<pre wrap="">Mercury data
</pre>
<blockquote type="cite">
<pre wrap="">modem, the same channel as the Apple iPhones use, but it's
</pre>
</blockquote>
<pre wrap="">much easier
</pre>
<blockquote type="cite">
<pre wrap="">to run test suites from my netbook.
Here's the same test from another time of day, early Sunday
</pre>
</blockquote>
<pre wrap="">morning,
</pre>
<blockquote type="cite">
<pre wrap="">when things were working well.
Note that I ran the test over the entire labor day weekend
</pre>
</blockquote>
<pre wrap="">at intervals.
</pre>
<blockquote type="cite">
<pre wrap="">The end-to-end ping time was bimodal. Either it pegged at
</pre>
</blockquote>
<pre wrap="">over 5000
</pre>
<blockquote type="cite">
<pre wrap="">milliseconds, or happily sat at under 200 milliseconds.
</pre>
</blockquote>
<pre wrap="">Exactly what
</pre>
<blockquote type="cite">
<pre wrap="">one would expect if TCP congestion control were disabled by
overbuffering in a router preceding the bottleneck link
</pre>
</blockquote>
<pre wrap="">shared by many users.
</pre>
<blockquote type="cite">
<pre wrap="">
------------------------------
$ ping lcs.mit.edu
PING lcs.mit.edu (128.30.2.121) 56(84) bytes of data.
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=1 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=209 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=2 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=118 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=3 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=166 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=4 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=165 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=5 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=224 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=6 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=183 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=7 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=224 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=8 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=181 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=9 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=220 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=10 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=179 ms
64 bytes from zermatt.csail.mit.edu (128.30.2.121):
</pre>
</blockquote>
<pre wrap="">icmp_seq=11 ttl=44
</pre>
<blockquote type="cite">
<pre wrap="">time=219 ms
^C
--- lcs.mit.edu ping statistics ---
11 packets transmitted, 11 received, 0% packet loss, time
</pre>
</blockquote>
<pre wrap="">10780ms rtt
</pre>
<blockquote type="cite">
<pre wrap="">min/avg/max/mdev = 118.008/190.547/224.960/31.772 ms $ traceroute
lcs.mit.edu traceroute to lcs.mit.edu (128.30.2.121), 30
</pre>
</blockquote>
<pre wrap="">hops max, 60
</pre>
<blockquote type="cite">
<pre wrap="">byte packets
1 * * *
2 172.26.248.2 (172.26.248.2) 178.725 ms 178.568 ms 179.500 ms
3 * * *
4 172.16.192.34 (172.16.192.34) 187.794 ms 187.677 ms
</pre>
</blockquote>
<pre wrap="">207.527 ms
</pre>
<blockquote type="cite">
<pre wrap=""> 5 12.88.7.205 (12.88.7.205) 207.416 ms 208.325 ms 69.630 ms
6 cr84.cgcil.ip.att.net (12.122.152.134) 79.425 ms 89.227 ms
90.083 ms
7 cr2.cgcil.ip.att.net (12.123.7.250) 98.679 ms 90.727
</pre>
</blockquote>
<pre wrap="">ms 91.576
</pre>
<blockquote type="cite">
<pre wrap="">ms
8 ggr2.cgcil.ip.att.net (12.122.132.137) 72.728 ms 89.628 ms
88.825 ms
9 192.205.33.186 (192.205.33.186) 89.787 ms 89.794 ms
</pre>
</blockquote>
<pre wrap="">80.918 ms
</pre>
<blockquote type="cite">
<pre wrap="">10 ae-31-55.ebr1.Chicago1.Level3.net (4.68.101.158) 79.895 ms
70.927 ms
78.817 ms
11 ae-1-5.bar1.Boston1.Level3.net (4.69.140.93) 107.820
</pre>
</blockquote>
<pre wrap="">ms 156.892
</pre>
<blockquote type="cite">
<pre wrap="">ms
140.711 ms
12 ae-7-7.car1.Boston1.Level3.net (4.69.132.241) 139.638
</pre>
</blockquote>
<pre wrap="">ms 139.764
</pre>
<blockquote type="cite">
<pre wrap="">ms
129.853 ms
13 MASSACHUSET.car1.Boston1.Level3.net (4.53.48.98) 149.595 ms
154.366 ms
152.225 ms
14 B24-RTR-2-BACKBONE.MIT.EDU (18.168.0.23) 146.808 ms
</pre>
</blockquote>
<pre wrap="">129.801 ms
</pre>
<blockquote type="cite">
<pre wrap="">89.659 ms
15 MITNET.TRANTOR.CSAIL.MIT.EDU (18.4.7.65) 109.463 ms
</pre>
</blockquote>
<pre wrap="">118.818 ms
</pre>
<blockquote type="cite">
<pre wrap="">91.727 ms
16 trantor.kalgan.csail.mit.edu (128.30.0.246) 91.541 ms
</pre>
</blockquote>
<pre wrap="">88.768 ms
</pre>
<blockquote type="cite">
<pre wrap=""> 85.837 ms
17 zermatt.csail.mit.edu (128.30.2.121) 117.581 ms 116.564 ms
103.569 ms $
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
</body>
</html>