Overview
Features
Download
Documentation
Community
Add-Ons & Services

Reading HTTP Multipart

Please post support and help requests here.

Reading HTTP Multipart

Postby michele_selea » 05 Nov 2008, 10:15

I notice a strange behavior while I make a multipart request. I'm getting a
multipart from a IP camera that send JPEG image in the body. The reading
function works, but if the multipart content-length goes over 50KBytes I have experience too much time consuming of the CPU. I think that the reading of the input stream is the problem: probably the problem is the length used for the allocation of the buffer. For some reason when I read the input stream of the Multipart I got this behavior for the reading function:
-read 1 byte and then some(for examples 64) bytes
-then again 1 byte and than some bytes.
I'm in a local network so the data transfer is fast. The reading function _Xsgetn_s of the streambuf is implemented with a loop that read all the input stream and this consume too much CPU time. Is there a way to tell to HttpInputStream to retrieve all the body, or all the part so that when I read the input stream I can read it in once? Or is there any way to specify the buffer to use in the MultipartInputStream so I can loop less times and waste less CPU time?
I notice that if I read a lot of bytes(even 100 or 500 kb) from HttpInputStream is not a problem (I think that the buffer in that case is 4000bytes). Instead if I read from MultipartInputStream I got cpu time consuming (I think that in this case the buffer is 1024bytes)
I look at the nextpart function of the multipart, and I think that is not so efficent to delete and reallocate the MultipartInputStream for every part.
Have someone experience this behavior?
Thanks
michele_selea
 
Posts: 6
Joined: 05 Nov 2008, 09:10
Location: Italy

Re: Reading HTTP Multipart

Postby michele_selea » 05 Nov 2008, 11:08

I check the deconstructor of the MultipartReader and ther's only:
delete _pMPI;
perhaps it's better to check if _pMPI is not NULL, to avoid possible leak problem.
Just an advice
michele_selea
 
Posts: 6
Joined: 05 Nov 2008, 09:10
Location: Italy

Re: Reading HTTP Multipart

Postby michele_selea » 05 Nov 2008, 16:32

I try to change the MultipartReader class that way:
-I change the method nextpart() this way:
void MultipartReader::nextPart(MessageHeader& messageHeader)
{
if(m_bFirstTime){
if (_boundary.empty())
guessBoundary();
else
findFirstBoundary();
m_bFirstTime=false;
}
/*
if (!_pMPI){
if (_boundary.empty())
guessBoundary();
else
findFirstBoundary();
}
else if (_pMPI->lastPart())
{
throw MultipartException("No more parts available");
}
*/
if(hasNextPart())
parseHeader(messageHeader);
else
throw MultipartException("No more parts available");

//delete _pMPI;
//_pMPI = new MultipartInputStream(_istr, _boundary);
}

-and method stream() this way:
std::istream& MultipartReader::stream() const
{
//poco_check_ptr (_pMPI);
//return *_pMPI;
return _istr;
}

For each part I have to read the exactly Content-Lenght+2 for read the
the rest of the use of multipart is the same: obviously I can make the parseHeader to extract the
so I don't have to read Content-Length+2. Now I can read part over 60Kbytes without time consuming and with cpu always at 0%. I think that this could be a way to optimize the code. Let me now if you agree this change.
michele_selea
 
Posts: 6
Joined: 05 Nov 2008, 09:10
Location: Italy

Re: Reading HTTP Multipart

Postby michele_selea » 05 Nov 2008, 16:57

Sorry for my previews post, was bad indented.
I try to change the MultipartReader class that way:
-I change the method nextpart() this way:
Code: Select all

void MultipartReader::nextPart(MessageHeader& messageHeader)
{
   if(m_bFirstTime){
      if (_boundary.empty())
         guessBoundary();
      else
         findFirstBoundary();
      m_bFirstTime=false;
   }
   /*
   if (!_pMPI){
      if (_boundary.empty())
         guessBoundary();
      else
         findFirstBoundary();
   }
   else if (_pMPI->lastPart())
   {
      throw MultipartException("No more parts available");
   }
   */
   if(hasNextPart())
      parseHeader(messageHeader);
   else
      throw MultipartException("No more parts available");

   //if(0!=_pMPI)
   //   delete _pMPI;
   //_pMPI = new MultipartInputStream(_istr, _boundary);
}


-and method stream() this way:
Code: Select all

std::istream& MultipartReader::stream() const
{
   //poco_check_ptr (_pMPI);
   //return *_pMPI;
   return _istr;
}


For each part I have to read the exactly Content-Lenght+2 for read the
the rest of the use of multipart is the same: obviously I can make the parseHeader to extract the
so I don't have to read Content-Length+2. Now I can read part over 60Kbytes without time consuming and with cpu always at 0%. I think that this could be a way to optimize the code. Let me now if you agree this change.
michele_selea
 
Posts: 6
Joined: 05 Nov 2008, 09:10
Location: Italy

Re: Reading HTTP Multipart

Postby michele_selea » 10 Nov 2008, 12:06

I'm feeling a bit lonely
michele_selea
 
Posts: 6
Joined: 05 Nov 2008, 09:10
Location: Italy

Re: Re: Reading HTTP Multipart

Postby guenter » 10 Nov 2008, 18:35

> I'm feeling a bit lonely

I haven't had the time yet to look into this. But if your workaround works for you, go with it. You just must be careful not to read more data than in the MIME part, otherwise you'll screw up the MIME parser (e.g., if you accidentally read into the next MIME header).


guenter
 
Posts: 1091
Joined: 11 Jul 2006, 16:27
Location: Austria


Return to Support

Who is online

Users browsing this forum: No registered users and 1 guest

cron