Overview
Features
Download
Documentation
Community
Add-Ons & Services

About upload file to httpformserver

Please post support and help requests here.

About upload file to httpformserver

Postby tiplip » 13 Jan 2011, 09:42

I run HTMLFormServer and upload file to this server at the same PC, say upload one file with size 10M, it'll take about more than 5s to finish this upload, that's about 2M/s for uploading.
My question is, is there some space to accelerate uploading through tuning some code?
tiplip
 
Posts: 16
Joined: 13 Jan 2011, 09:21

Re: About upload file to httpformserver

Postby guenter » 14 Jan 2011, 09:08

Maybe something with your setup? On my three years old Mac Book Pro I constantly get 11 MB per seconds. There's still some potential for optimization in the Poco::Net::MultipartReader class, though.
guenter
 
Posts: 1110
Joined: 11 Jul 2006, 16:27
Location: Austria

Re: About upload file to httpformserver

Postby tiplip » 14 Jan 2011, 11:30

guenter wrote:Maybe something with your setup? On my three years old Mac Book Pro I constantly get 11 MB per seconds. There's still some potential for optimization in the Poco::Net::MultipartReader class, though.


I'm not sure what 'setup' here means, list my test environment as follows:

System:
Microsoft Windows XP
Professional version 2002 Service Pack 2

Laptop:
DELL LATITUDE|6400

Intel(R) Core(TM)2 Duo CPU P8600 @2.40GHz
2.39GHz, 1.98GB of Ram Physical Address Extension

HTMLFormServer is built with VSExpress 2008

I don't know how to get your performance like 11M/s, can I get your copy of HTMLFormServer that I do test?
Thanks.
tiplip
 
Posts: 16
Joined: 13 Jan 2011, 09:21

Re: About upload file to httpformserver

Postby guenter » 14 Jan 2011, 22:34

That's definitely an interesting issue. On Mac OS X and Linux, I can consistently get 11 MB/sec, while on Windows, on similar hardware, I only get 2 MB. Since it's the same code that runs on all platforms, there seems to be a huge inefficiency somewhere, possible in the IOstreams framework (this is used heavily in multipart decoding).
guenter
 
Posts: 1110
Joined: 11 Jul 2006, 16:27
Location: Austria

Re: About upload file to httpformserver

Postby tiplip » 17 Jan 2011, 07:03

hi guenter,

I agree with you, and I think we can attribute performance to windows' porting of iostream.
Of course, I don't do the same test on MS Windows server, maybe it's another story.
tiplip
 
Posts: 16
Joined: 13 Jan 2011, 09:21

Re: About upload file to httpformserver

Postby sevenup » 22 Jan 2011, 08:19

I'm expecting the same issue when uploading large files on windows 7 64bits.
I also noticed that my CPU usage is very high during file upload (around 90% with 3 simultaneous ~500MB files)

That's all I do :
void handlePart(const MessageHeader& header, std::istream& stream)
{
std::ofstream ofs;

ofs.open("out", std::ios::binary);
ofs << stream.rdbuf();
ofs.close();
}

I didn't try on a linux system yet.

EDIT :
tested on my archlinux virtual machine and it's much faster (14MB/s vs 3MB/s on win7...) with lower cpu usage.

does anyone have a solution ?
sevenup
 
Posts: 1
Joined: 22 Jan 2011, 08:11

Re: About upload file to httpformserver

Postby guenter » 22 Jan 2011, 12:48

This issue seems to be related: https://sourceforge.net/tracker/index.p ... tid=725709

Must be some strange behavior caused by the implementation of the Visual C++ iostreams library. I will look into it and see what can be done.
guenter
 
Posts: 1110
Joined: 11 Jul 2006, 16:27
Location: Austria

Re: About upload file to httpformserver

Postby guenter » 22 Jan 2011, 21:33

Okay, made some changes to MultipartReader. No longer reading characters from stream (get()), but directly from underlying streambuf (sbumpc()). Result: two times as fast now on Windows. That's at least something. But also 25 % faster with GCC now, so with MSVC it's still significantly slower than with GCC. New code is in 1.4.1 SVN branch: http://poco.svn.sourceforge.net/viewvc/ ... oco-1.4.1/

Could someone test with MSVC and STLport? Would be interesting.
guenter
 
Posts: 1110
Joined: 11 Jul 2006, 16:27
Location: Austria

Re: About upload file to httpformserver

Postby tiplip » 11 Feb 2011, 09:58

hi, All,

With poco 1.4.1, I can get almost 10M/s while uploading file to server if on the same computer, test environment similar to my first post.
tiplip
 
Posts: 16
Joined: 13 Jan 2011, 09:21

Re: About upload file to httpformserver

Postby bstewart33 » 10 Dec 2012, 18:13

I am new to POC 1.4.5 and have made a test program to upload a file to a php http server [http://posttestserver.com].

The following code I built under MAC OS X(10.8.2) and WIndows Server 2008 R2 Server. When I run the program with a 10MB file, under the MAC I get upload speeds around 10m/s, under windows I get 2m/s. I am using the HTMLForm part method to uoload the file. I also had to turn off chunked transfer encoding due the server not supporting chunked transfer encoding. Are there other parameters that can be set for windows in order to get speeds equivilant to the MAC?

Here is my code:
URI uri(argv[1]);
std::string path(uri.getPathAndQuery());
if (path.empty()) path = "/";

std::string uploadFilename(argv[2]);

HTTPClientSession &session(
*(new HTTPClientSession(uri.getHost(), uri.getPort())));

for (unsigned int ix( 0 ); ix < ITERATIONS; ix++)
{
HTTPRequest &request(
*(new HTTPRequest(
HTTPRequest::HTTP_POST, path, HTTPMessage::HTTP_1_1)));

HTMLForm &form (
*(new HTMLForm( HTMLForm::ENCODING_MULTIPART ) ) );

FilePartSource *partSource(
new FilePartSource (
uploadFilename, "application/octet-stream"));
//uploadFilename, "Text/Plain"));

std::istream &partStream(partSource->stream());

form.addPart("upload", partSource);

form.prepareSubmit(request);

//Determine content-length from form data
std::stringstream ss;
form.write(ss);
std::string formString(ss.str());

//Reset the form stream to begining of file
partStream.clear();
partStream.seekg(0);

request.setChunkedTransferEncoding(FALSE);
request.setContentLength(
static_cast<unsigned int> (formString.length()));

std::ostream &send = session.sendRequest(request);

form.write (send);

//----- other code to handle response here...

delete &form;
delete &request;
} //For Loop

delete &session;
bstewart33
 
Posts: 2
Joined: 10 Dec 2012, 17:30


Return to Support

Who is online

Users browsing this forum: No registered users and 4 guests

cron