[e2e] Are you interested in TOEs and related issues
Sunay.Tripathi at eng.sun.com
Wed Mar 3 20:55:58 PST 2004
> Well, the interesting thing is we went down this path in the early 1980s
> and found that TOEs didn't work.
> If I try to distill all that we learned in the 1980s into one question, I
> come out with:
> In the 1980s we discovered that communicating with a TOE over a bus
> about a TCP connection was as expensive or more expensive than
> simply handling the TCP connection in the main processor. What about
> today's TOE designs makes them different?
I don't have too much history (1980s predates my career) but there were
some attempts in mid 90s to do the same and compared to that, the
following reasons seem to be the driving force:
1) Extra processor(s) buried in the TOE for networking processing which is
hidden from the kernel and leaves the host CPU to do more application
related work. Saves the cost of licences for application which take
number of CPU into account (oracle is one such application cited).
2) On low end (1-2 CPU) x86 based machines, cost of adding a processor
is much higher than adding a TOE (I personally haven't verified this).
3) For the up and coming 10Gb NICs, TOE will help saturate the link. Some
vendors assert that TOE will be required to support 10Gb NICs.
4) Performance reasons. Just the LSO aspect of TOE (sending large chunks of
data and letting the TOE split it up in mss size pieces) and ack
coalescing gives a pretty good boost (our own prototypes indicates that
this is true). The gains are by optimizing data movement and not by
offloading protocol processing.
5) TOE is necessary for RDMA, iSCSI etc. for layering reasons. I am not
involved with RDMA so someone who is an expert can probably comment on
6) TOE based NIC are already making pretty good headway in embedded space.
The technology is already maturing so why not use it in broader market.
Note that the above claims are in no particular order of importance and made
my TOE vendors in general. Of these, I personally do agree with 1 and 4 but
that iteself doesn't mean that TOE will make it in general purpose networking.
It would be interesting to see if you and others in the list agree or
disagree with these claims.
Senior Staff Engineer,
Solaris Kernel Networking,
Sun MicroSystems Inc.
email: sunay at eng.sun.com Phone: 650-786-6007 (W)
More information about the end2end-interest