About upload file to httpformserver

[tiplip]

I run HTMLFormServer and upload file to this server at the same PC, say upload one file with size 10M, it'll take about more than 5s to finish this upload, that's about 2M/s for uploading.
My question is, is there some space to accelerate uploading through tuning some code?

[guenter]

Maybe something with your setup? On my three years old Mac Book Pro I constantly get 11 MB per seconds. There's still some potential for optimization in the Poco::Net::MultipartReader class, though.

[tiplip]

Maybe something with your setup? On my three years old Mac Book Pro I constantly get 11 MB per seconds. There's still some potential for optimization in the Poco::Net::MultipartReader class, though.

I'm not sure what 'setup' here means, list my test environment as follows:

System:
Microsoft Windows XP
Professional version 2002 Service Pack 2

Laptop:
DELL LATITUDE|6400

Intel(R) Core(TM)2 Duo CPU P8600 @2.40GHz
2.39GHz, 1.98GB of Ram Physical Address Extension

HTMLFormServer is built with VSExpress 2008

I don't know how to get your performance like 11M/s, can I get your copy of HTMLFormServer that I do test?
Thanks.

[guenter]

That's definitely an interesting issue. On Mac OS X and Linux, I can consistently get 11 MB/sec, while on Windows, on similar hardware, I only get 2 MB. Since it's the same code that runs on all platforms, there seems to be a huge inefficiency somewhere, possible in the IOstreams framework (this is used heavily in multipart decoding).

[tiplip]

hi guenter,

I agree with you, and I think we can attribute performance to windows' porting of iostream.
Of course, I don't do the same test on MS Windows server, maybe it's another story.

[sevenup]

I'm expecting the same issue when uploading large files on windows 7 64bits.
I also noticed that my CPU usage is very high during file upload (around 90% with 3 simultaneous ~500MB files)

That's all I do :
void handlePart(const MessageHeader& header, std::istream& stream)
{
std::ofstream ofs;

ofs.open("out", std::ios::binary);
ofs << stream.rdbuf();
ofs.close();
}

I didn't try on a linux system yet.

EDIT :
tested on my archlinux virtual machine and it's much faster (14MB/s vs 3MB/s on win7...) with lower cpu usage.

does anyone have a solution ?

[guenter]

This issue seems to be related: https://sourceforge.net/tracker/index.p ... tid=725709

Must be some strange behavior caused by the implementation of the Visual C++ iostreams library. I will look into it and see what can be done.

[guenter]

Okay, made some changes to MultipartReader. No longer reading characters from stream (get()), but directly from underlying streambuf (sbumpc()). Result: two times as fast now on Windows. That's at least something. But also 25 % faster with GCC now, so with MSVC it's still significantly slower than with GCC. New code is in 1.4.1 SVN branch: http://poco.svn.sourceforge.net/viewvc/ ... oco-1.4.1/

Could someone test with MSVC and STLport? Would be interesting.

[tiplip]

hi, All,

With poco 1.4.1, I can get almost 10M/s while uploading file to server if on the same computer, test environment similar to my first post.

[bstewart33]

I am new to POC 1.4.5 and have made a test program to upload a file to a php http server [http://posttestserver.com].

The following code I built under MAC OS X(10.8.2) and WIndows Server 2008 R2 Server. When I run the program with a 10MB file, under the MAC I get upload speeds around 10m/s, under windows I get 2m/s. I am using the HTMLForm part method to uoload the file. I also had to turn off chunked transfer encoding due the server not supporting chunked transfer encoding. Are there other parameters that can be set for windows in order to get speeds equivilant to the MAC?

Here is my code:
URI uri(argv[1]);
std::string path(uri.getPathAndQuery());
if (path.empty()) path = "/";

std::string uploadFilename(argv[2]);

HTTPClientSession &session(
*(new HTTPClientSession(uri.getHost(), uri.getPort())));

for (unsigned int ix( 0 ); ix < ITERATIONS; ix++)
{
HTTPRequest &request(
*(new HTTPRequest(
HTTPRequest::HTTP_POST, path, HTTPMessage::HTTP_1_1)));

HTMLForm &form (
*(new HTMLForm( HTMLForm::ENCODING_MULTIPART ) ) );

FilePartSource *partSource(
new FilePartSource (
uploadFilename, "application/octet-stream"));
//uploadFilename, "Text/Plain"));

std::istream &partStream(partSource->stream());

form.addPart("upload", partSource);

form.prepareSubmit(request);

//Determine content-length from form data
std::stringstream ss;
form.write(ss);
std::string formString(ss.str());

//Reset the form stream to begining of file
partStream.clear();
partStream.seekg(0);

request.setChunkedTransferEncoding(FALSE);
request.setContentLength(
static_cast<unsigned int> (formString.length()));

std::ostream &send = session.sendRequest(request);

form.write (send);

//----- other code to handle response here...

delete &form;
delete &request;
} //For Loop

delete &session;