Add-Ons & Services

Issue with DefaltingOutputStream/InflatingInputStream

Please post support and help requests here.

Issue with DefaltingOutputStream/InflatingInputStream

Postby jnewell » 10 Aug 2012, 23:04

I wrote a simple unit test to show the issue. The DefaltingOutputStream/InflatingInputStream sometimes fails with different sizes. I'll debug further but I'm posting to see if someone else has encountered this or maybe I'm just doing something wrong in the calling.

Code: Select all
# works with 64000
$ ./bccd2 -c -f -t comp
31950:TID=0:Main.cpp:93:I:logging enabled: level=6
31950:TID=0::0:I:Run Test: #11
31950:TID=0::0:I:original size=64000 compressed size=298

# doesn't work with 36828 and other odd ranges
$$ ./bccd2 -c -f -t comp
32098:TID=0:Main.cpp:93:I:logging enabled: level=6
32098:TID=0::0:I:Run Test: #11
32098:TID=0::0:I:original size=36828 compressed size=179
32098:TID=0::0:I:uncompress not equal to original

Code: Select all
static int test_compression2(std::ostream &out) {
  int size = 36828; // doesn't work
  //int size = 64000;   // works
  string orginalMessage(size, 'a');
  Buffer<char> buffer(size);
  MemoryOutputStream mos(buffer.begin(), size);
  DeflatingOutputStream dos(mos, -10, Z_BEST_SPEED); // use zlib with no header
  //d.write(orginalMessage.data(), orginalMessage.size());
  dos << orginalMessage;
  int csize = mos.charsWritten();
  out << "original size=" << size << " " << "compressed size=" << csize << endl;

  MemoryInputStream mis(buffer.begin(), csize);
  InflatingInputStream ii(mis, -10);
  string uncompressedMessage;
  ii >> uncompressedMessage;
  out << "size=" << uncompressedMessage.size() << endl;
  if (uncompressedMessage != orginalMessage) {
    out << "uncompress not equal to original" << endl;
  return 0;
Posts: 7
Joined: 10 May 2011, 19:55

Re: Issue with DefaltingOutputStream/InflatingInputStream

Postby jnewell » 20 Aug 2012, 00:55

**** Ignore this patch. It's not correct. See next message in thread. ****

I tracked this down. The charsWritten() in include/Poco/MemoryStream.h is sometimes returning an incorrect size. I believe the patch below fixes the issue.

Code: Select all
   std::streamsize charsWritten() const
      return static_cast<std::streamsize>(this->epptr() - this->pptr());
      //return static_cast<std::streamsize>(this->pptr() - this->pbase());
Last edited by jnewell on 20 Aug 2012, 14:23, edited 1 time in total.
Posts: 7
Joined: 10 May 2011, 19:55

Re: Issue with DefaltingOutputStream/InflatingInputStream

Postby guenter » 20 Aug 2012, 02:55

charsWritten() is correct. The issue is actually in the InflatingStreamBuf class, where under some circumstances the final call to inflate() is not done, resulting in an imcompletely inflated file. See <https://sourceforge.net/tracker/?func=detail&aid=3559665&group_id=132964&atid=725709>. Fix will be in 1.4.4.
Posts: 1168
Joined: 11 Jul 2006, 16:27
Location: Austria

Re: Issue with DefaltingOutputStream/InflatingInputStream

Postby jnewell » 20 Aug 2012, 14:22

I was way off. I was just getting lucky with my change with it covering more of the memory buffer. Sorry. My experience/ability with templates is poor. I never even saw the readFromDevice() being called in the debugger. I will remove the above patch so others don't get confused. Regards, Jim
Posts: 7
Joined: 10 May 2011, 19:55

Return to Support

Who is online

Users browsing this forum: No registered users and 1 guest