Poco::Net::HTTPResponseStreamBuf performance issues

[petervn]

Hi,
I am migrating some of the legacy HTTP code to use the Poco::Net classes. The legacy code uses the TCP/IP sockets to send/receive HTTP requests and responses. While doing the performance testing, I've noticed that downloading data from a remote HTTP server using legacy code is on average three times fasters than downloading online content using the Poco libraries. The glowcode shows that most of the time is spent in BasicUnbufferedStreamBuf<char,std::char_trait<char> >::uflow and HTTPResponseStreamBuf::readFromDevice. BasicUnbufferedStreamBuf<char,std::char_trait<char> >::uflow is shown to have been invoked 164,445,204. times to service the total of 1023 HTTP responses. I wonder if the shear number of method invocations is the cause for the observable performance differences. Has anyone else run similar performance tests?

[guenter]

Since you're mentioning specifically HTTPResponseStreamBuf, I'm assuming you're using the URIStreamOpener class. Basically, HTTPResponseStreamBuf currently is somewhat of a performance disaster as it adds yet another stream on top the HTTP streams, and to make things worse, one that reads byte-by-byte from the underlying stream. This should definitely be optimized. You will get better results by using Poco::Net::HTTPClientSession directly.

[petervn]

Since you're mentioning specifically HTTPResponseStreamBuf, I'm assuming you're using the URIStreamOpener class. Basically, HTTPResponseStreamBuf currently is somewhat of a performance disaster as it adds yet another stream on top the HTTP streams, and to make things worse, one that reads byte-by-byte from the underlying stream. This should definitely be optimized. You will get better results by using Poco::Net::HTTPClientSession directly.

That was it, thanks a lot! I was using HTTPResponseStream to wrap the stream returned by the receiveResponse method call. I've changed my code around to avoid using HTTPResponseStream.