Reading HTTP Multipart

[michele_selea]

I notice a strange behavior while I make a multipart request. I'm getting a
multipart from a IP camera that send JPEG image in the body. The reading
function works, but if the multipart content-length goes over 50KBytes I have experience too much time consuming of the CPU. I think that the reading of the input stream is the problem: probably the problem is the length used for the allocation of the buffer. For some reason when I read the input stream of the Multipart I got this behavior for the reading function:
-read 1 byte and then some(for examples 64) bytes
-then again 1 byte and than some bytes.
I'm in a local network so the data transfer is fast. The reading function _Xsgetn_s of the streambuf is implemented with a loop that read all the input stream and this consume too much CPU time. Is there a way to tell to HttpInputStream to retrieve all the body, or all the part so that when I read the input stream I can read it in once? Or is there any way to specify the buffer to use in the MultipartInputStream so I can loop less times and waste less CPU time?
I notice that if I read a lot of bytes(even 100 or 500 kb) from HttpInputStream is not a problem (I think that the buffer in that case is 4000bytes). Instead if I read from MultipartInputStream I got cpu time consuming (I think that in this case the buffer is 1024bytes)
I look at the nextpart function of the multipart, and I think that is not so efficent to delete and reallocate the MultipartInputStream for every part.
Have someone experience this behavior?
Thanks

[michele_selea]

I check the deconstructor of the MultipartReader and ther's only:
delete _pMPI;
perhaps it's better to check if _pMPI is not NULL, to avoid possible leak problem.
Just an advice

[michele_selea]

I try to change the MultipartReader class that way:
-I change the method nextpart() this way:
void MultipartReader::nextPart(MessageHeader& messageHeader)
{
if(m_bFirstTime){
if (_boundary.empty())
guessBoundary();
else
findFirstBoundary();
m_bFirstTime=false;
}
/*
if (!_pMPI){
if (_boundary.empty())
guessBoundary();
else
findFirstBoundary();
}
else if (_pMPI->lastPart())
{
throw MultipartException("No more parts available");
}
*/
if(hasNextPart())
parseHeader(messageHeader);
else
throw MultipartException("No more parts available");

//delete _pMPI;
//_pMPI = new MultipartInputStream(_istr, _boundary);
}

-and method stream() this way:
std::istream& MultipartReader::stream() const
{
//poco_check_ptr (_pMPI);
//return *_pMPI;
return _istr;
}

For each part I have to read the exactly Content-Lenght+2 for read the
the rest of the use of multipart is the same: obviously I can make the parseHeader to extract the
so I don't have to read Content-Length+2. Now I can read part over 60Kbytes without time consuming and with cpu always at 0%. I think that this could be a way to optimize the code. Let me now if you agree this change.

[michele_selea]

Sorry for my previews post, was bad indented.
I try to change the MultipartReader class that way:
-I change the method nextpart() this way:

Code: Select all


void MultipartReader::nextPart(MessageHeader& messageHeader)
{
   if(m_bFirstTime){
      if (_boundary.empty())
         guessBoundary();
      else
         findFirstBoundary();
      m_bFirstTime=false;
   }
   /*
   if (!_pMPI){
      if (_boundary.empty())
         guessBoundary();
      else
         findFirstBoundary();
   }
   else if (_pMPI->lastPart())
   {
      throw MultipartException("No more parts available");
   }
   */
   if(hasNextPart())
      parseHeader(messageHeader);
   else
      throw MultipartException("No more parts available");

   //if(0!=_pMPI)
   //   delete _pMPI;
   //_pMPI = new MultipartInputStream(_istr, _boundary);
}


-and method stream() this way:

Code: Select all


std::istream& MultipartReader::stream() const
{
   //poco_check_ptr (_pMPI);
   //return *_pMPI;
   return _istr;
}


For each part I have to read the exactly Content-Lenght+2 for read the
the rest of the use of multipart is the same: obviously I can make the parseHeader to extract the
so I don't have to read Content-Length+2. Now I can read part over 60Kbytes without time consuming and with cpu always at 0%. I think that this could be a way to optimize the code. Let me now if you agree this change.

[michele_selea]

I'm feeling a bit lonely

[guenter]

> I'm feeling a bit lonely

I haven't had the time yet to look into this. But if your workaround works for you, go with it. You just must be careful not to read more data than in the MIME part, otherwise you'll screw up the MIME parser (e.g., if you accidentally read into the next MIME header).