[rbridge] Back to ND/ARP optimization
Silvano Gai
sgai at nuovasystems.com
Mon May 7 07:31:14 PDT 2007
Donald,
Let's consider the following RBridge
+---------------------------------------------+
| RBRIDGE |
| +--------+ |
| | CPU | |
| +--------+ |
| | |
| +--------+ |
| | Fabric | |
| +--------+ |
| | | | |
+-----------------+---+---+-------------------+
| | |
A B C
It has three ports A, B, C, one internal fabric and one CPU.
In the following I will assume that it has won DR election on all the
ports.
WITH NO ARP/ND OPTIMIZATION
===========================
An ARP for Z is received on port A.
The fabric forwards it as a regular Broadcast (i.e. it sends it on all
the ports except the one where it came in). It goes out on B and C (but
not on A) and the CPU gets a copy. The CPU is not Z, so the ARP process
in the CPU discards it.
WITH ARP/ND OPTIMIZATION
========================
An ARP for Z is received on port A.
The fabric MUST be programmed to not forward ARP has a regular
broadcast, but to pass it to the CPU. Therefore a single copy of the ARP
frame is sent to the CPU. The CPU cannot do a simple test, as in the
previous case, but it needs to search Z in a cache (a bit more
expensive). It misses. What happens next is interesting.
You say that you want to "forwards it in native form out every other
port".
We cannot send out of port A, since we received the ARP from port A and
we MUST send it ONLY on B and C.
Therefore the CPU must:
- not use the classical broadcast mode (otherwise A will get a copy of
the frame);
- remember in SW from which port it received the frame;
- use some fabric specific mechanism to send out the frame only on B and
C.
The mechanism I have seen used in these cases is called "port direct" or
"index direct" in which basically the CPU has a way to send out any kind
of frame on a particular port.
If this mechanism is used, the CPU needs to replicate in SW the ARP
query.
Other mechanism can be invented and implemented in HW, but I don't think
they are worthwhile. See also the reply of Arien.
I hope it clarifies.
-- Silvano
> -----Original Message-----
> From: Eastlake III Donald-LDE008 [mailto:Donald.Eastlake at motorola.com]
> Sent: Sunday, May 06, 2007 8:40 PM
> To: Silvano Gai
> Cc: rbridge at postel.org
> Subject: RE: [rbridge] Back to ND/ARP optimization
>
> Silvano,
>
> Sorry, I still don't understand this.
>
> What does an Rbridge do if it receives a native broadcast frame with
an
> unknown Ethertype (unknown in the sense that it is not in any special
> table or comparison implemented in the Rbridge)? If the Rbridge isn't
> the DR for the port it received it on, it drops it. If it is the DR,
it
> forwards it in native form out every other port for which it is the DR
> and which is in the same VLAN. It also encapsulates it and sends it
off
> on a distribution tree pruned by VLAN.
>
> What does an Rbridge do if it receives a native broadcast ARP frame?
If
> it isn't the DR for the port it receive it on, it drops it. Otherwise,
> it learns the IP to MAC mapping for the sender fields in the ARP frame
> and it tries to find the target IP address in its database of IP to
MAP
> mappings. If it can't find an entry for the target IP address, it then
> proceeds as in the paragraph above to flood it.
>
> In other words, for an unknown target IP address, processing of a
> broadcast ARP frame is exactly the same as for an unknown Ethertype
> except for the learning step and the table look up attempt step. Why
> can't this be done in hardware? Even if I assume the learning and
table
> look up happen in software, if the IP address isn't found, why
couldn't
> the software just re-inject the frame into the hardware as if it had
> come in on its original receipt port with a flag saying "treat like an
> unknown Ethertype"?
>
> Thanks,
> Donald
>
> -----Original Message-----
> From: Silvano Gai [mailto:sgai at nuovasystems.com]
> Sent: Sunday, May 06, 2007 8:36 PM
> To: Eastlake III Donald-LDE008
> Cc: rbridge at postel.org
> Subject: RE: [rbridge] Back to ND/ARP optimization
>
>
> Donald,
>
> To provide a precise answer we need to consider which of the three
> schemes described in section 3.12 of
>
http://www.ietf.org/internet-drafts/draft-ietf-trill-rbridge-protocol-03
> .txt
> the RBridge will implement.
>
> The draft refers generically to "Some mix of these strategies".
>
> Lacking this information, let's assume the strategy you mention.
> The flood cannot be done in HW. In fact it is mandatory to not resend
> the ARP on the port it was received to avoid frame duplication.
>
> The flood must be implemented in SW has n-1 port transmissions; where
n
> is the number of ports.
>
> Therefore, the CPU of a 256 port RBridge:
> - in absence of ARP/ND proxy receives 1 ARP (and the HW does the
correct
> replication)
> - with ARP/ND proxy it receives 1 ARP (and the HW does nothing)and
sends
> out 255 ARP messages. Huge overhead
>
> -- Silvano
>
>
> > -----Original Message-----
> > From: Eastlake III Donald-LDE008
[mailto:Donald.Eastlake at motorola.com]
> > Sent: Sunday, May 06, 2007 2:30 PM
> > To: Silvano Gai
> > Cc: rbridge at postel.org
> > Subject: RE: [rbridge] Back to ND/ARP optimization
> >
> > Silvano,
> >
> > I don't understand what these Rbridge CPUs will be doing. If an
> Rbridge
> > receives a native ARP broadcast frame for an unknown IP address it
> would
> > just learn the source IP->MAC mapping and then flood the ARP
> broadcast.
> >
> > Donald
> >
> > -----Original Message-----
> > From: rbridge-bounces at postel.org [mailto:rbridge-bounces at postel.org]
> On
> > Behalf Of Silvano Gai
> > Sent: Friday, May 04, 2007 3:01 PM
> > To: Arien Vijn
> > Cc: rbridge at postel.org; Radia.Perlman at sun.com
> > Subject: Re: [rbridge] Back to ND/ARP optimization
> >
> > Arien
> >
> > Thanks for the real world data, which proves that ARP/ND proxy must
> not
> > be implemented on RBridges.
> >
> > In fact, since the queries toward unused addresses are dominant, the
> > ARP/ND cache will almost always miss and the RBridge CPU will be
busy
> > trying to remotely resolve queries that will never complete.
> >
> > -- Silvano
> >
> > > -----Original Message-----
> > > From: Arien Vijn [mailto:arien.vijn at ams-ix.net]
> > > Sent: Friday, May 04, 2007 10:04 AM
> > > To: Silvano Gai
> > > Cc: Arien Vijn; Radia.Perlman at sun.com; rbridge at postel.org
> > > Subject: Re: [rbridge] Back to ND/ARP optimization
> > >
> > > Hi,
> > >
> > > On May 3, 2007, at 10:55 PM, Silvano Gai wrote:
> > >
> > > [...]
> > >
> > > >> Does anyone have a handle on how much traffic is caused by
> ARP/ND?
> > > >>
> > > >
> > > > With an ARP cache of 30 minutes, typical in hosts today, even
with
> > > > a 100
> > > > K hosts in a VLAN we get at most 55 ARP seconds. Since not all
the
> > > > hosts
> > > > talk with each other, it is more typically like 5 ARP/sec.
> > > >
> > > > BASICALLY NOTHING.
> > >
> > > Well... that might be the case if all your hosts are actually
> > > answering. But query rates are pretty much determined by queries
for
> > > unused addresses. In other words by the query rates hosts
(routers)
> > > can achieve for addresses that are not in their caches.
> > >
> > > For what its worth. I am involved in a real network with 400+ BGP
> > > routers in one broadcast domain (/23 IPv4 subnet). The average
rate
> > > is little over 13 queries/second. That is with ARP mitigation and
> > > peak rates are much higher.
> > >
> > > -- Arien
> >
> >
> > _______________________________________________
> > rbridge mailing list
> > rbridge at postel.org
> > http://mailman.postel.org/mailman/listinfo/rbridge
More information about the rbridge
mailing list