[e2e] Need help: setting winsock receive low watermark while using completion port and TCP
David P. Reed
dpreed at reed.com
Thu Aug 25 07:37:34 PDT 2005
Lars Eggert wrote:
> On Aug 24, 2005, at 0:21, Alex Krivonosov (alexkr) wrote:
>> I have a TCP connection handled by the completion port IO model.
>> What is happening is in case I specify a large buffer for receiving
>> (WSARecv), the operation completes only after the buffer is full,
>> not after receiving about 500 bytes (a packet), so a significant
>> delay is introduced. In case of small buffers, performance degrades.
>> Any advice on this? Completion port model is a must.
> Please understand that TCP doesn't deliver "packets" to the
> application, it provides a byte stream. You may want to look into
> using non-blocking I/O for the receive call. (I don't know what you
> mean by "completion port model.")
The definition of I/O completion in Winsock *is* buffer full. Size of
buffer on receive is not a major performance problem (system calls
aren't slow compared to processing), so if you want notification on 500
bytes, use 500 byte buffers.
A thought you might not have considered: Perhaps you are sending your
500 byte messages, one per call, on the sender with TCP_NODELAY set?
This could cause some performance problems if the source end has a fast
link, but the receiving node has a slow absorption rate (the packets on
the source will not combine into larger frames until the window fills
up.) Of course that is exactly what TCP_NODELAY is for (minimizing
message latency, but increasing network overhead) - if you don't care so
much about latency, don't set TCP_NODELAY.
(or you can get very complex by using I/O completion based app-level
output management on the send side to control the latency/efficiency
tradeoff, using WSASendMessage to gather mutliple frames adaptively and
"delaying" sends at the app level until precise conditions hold related
to your desired latency goal and trying to gather 1-3 of your messages
into single sends).
More information about the end2end-interest