Overview
Features
Download
Documentation
Community
Add-Ons & Services

AsyncChannel memory leak?

Please post support and help requests here.

AsyncChannel memory leak?

Postby rekkai » 08 Feb 2008, 16:52

{edited to properly display code. use CODE tags for code}

I noticed a potential memory leak when using the async log channel. My attached unit test that does nothing but log messages to the channel ends up consuming about 5 MBs of memory that is never released back. This is in Windows and I see it in debug/release.

If I disable the Async channel and only do regular sync file logging, I don't see any usage of memory growing.

I have some memory leak detection code and it is complaining that the notification object allocated by calling new MessageNotification is not being freed.

~~#FF0000:
void AsyncChannel::log(const Message& msg)
{
open();
_queue.enqueueNotification(new MessageNotification(msg));
}
~~

Example test that reproduces this problem follows. I ran this in 1.31
Note that you need to open task manager view the memory usage before start and then at the end before exit.

Sorry about the formatting. I can get a better sample via email if you want.

Code: Select all

TEST(PocoLogLeak)
{
    if (UnitTest::gOnlyTest != GetName())
        return;

    std::cout << "Type something to start..Check app memory usage here." << std::endl;
    char ch;
    std::cin >> ch;

    Poco::PatternFormatter* pf = new Poco::PatternFormatter("%H:%M:%S:%i [%I] [%T]:%q:%t");
    pf->setProperty(Poco::PatternFormatter::PROP_TIMES, "local"); // option is UTC
    Poco::AutoPtr pFCConsole
        = new Poco::FormattingChannel(
        Poco::AutoPtr(pf));

    Poco::AutoPtr pFileChannel = new Poco::FileChannel();

    std::string logFilePathBase = "PocoLog.log";

    try
    {
        pFileChannel->setProperty(Poco::FileChannel::PROP_PATH,  logFilePathBase);
        Poco::AutoPtr pAsync = new Poco::AsyncChannel(pFileChannel);
        pFCConsole->setChannel(pAsync);
        pFCConsole->open();

        Poco::Logger::root().setChannel(pFCConsole);
        Poco::Logger::root().setLevel(Poco::Message::PRIO_TRACE);

    }
    catch(Poco::Exception& exc)
    {

        CHECK(0);
    }

    // ... Start pushing logs
    for (int idx = 0; idx < 15000; idx++)
    {
        std::ostringstream str;
        str << "This is a really long long long long Log Message : " << idx;
        Poco::Logger::root().trace(str.str());
    }

    UnitTest::waitForInput();
}

rekkai
 
Posts: 11
Joined: 02 Aug 2007, 05:53
Location: United_States

Re: AsyncChannel memory leak?

Postby guenter » 13 Feb 2008, 10:01

This is not a memory leak. The issue here is that your sample code fills up the NotificationQueue holding the log messages way faster than the AsyncChannel can process them. NotificationQueue internally uses a deque to hold the notification objects, and this deque grows very rapidly, to hold all the queued notifications. Unfortunately, the memory allocated by the deque won't be released until the deque is destroyed, so this is why the Task Manager reports rapidly growing memory usage.

At the time, your code reaches waitForInput, the AsyncChannel is still busy processing log messages.
guenter
 
Posts: 1121
Joined: 11 Jul 2006, 16:27
Location: Austria

Re: AsyncChannel memory leak?

Postby rekkai » 20 Feb 2008, 19:39

Thanks for the response.

Note that I usually wait for all the logging activity to stop and for the logging thread to become idle. At this point the memory appears yet not be released.

However, if you are saying that the queue does not release the allocated memory till it is destroyed (which will only happen at program exit) that could explain the memory consumption.

rekkai
 
Posts: 11
Joined: 02 Aug 2007, 05:53
Location: United_States

Re: AsyncChannel memory leak?

Postby axel » 05 Mar 2008, 19:09

> {edited to properly display code. use CODE tags for code}
>
> I noticed a potential memory leak when using the async log channel. My attached unit test that does nothing but log messages to the channel ends up consuming about 5 MBs of memory that is never released back. This is in Windows and I see it in debug/release.
>
> If I disable the Async channel and only do regular sync file logging, I don't see any usage of memory growing.
>
> I have some memory leak detection code and it is complaining that the notification object allocated by calling new MessageNotification is not being freed.
>
> ~~#FF0000:
> void AsyncChannel::log(const Message& msg)
> {
> open();
> _queue.enqueueNotification(new MessageNotification(msg));
> }
> ~~
>
> Example test that reproduces this problem follows. I ran this in 1.31
> Note that you need to open task manager view the memory usage before start and then at the end before exit.
>
> Sorry about the formatting. I can get a better sample via email if you want.
>
>
Code: Select all

> TEST(PocoLogLeak)
> {
>     if (UnitTest::gOnlyTest != GetName())
>         return;
>
>     std::cout      std::cin >> ch;
>
>     Poco::PatternFormatter* pf = new Poco::PatternFormatter("%H:%M:%S:%i [%I] [%T]:%q:%t");
>     pf->setProperty(Poco::PatternFormatter::PROP_TIMES, "local"); // option is UTC
>     Poco::AutoPtr pFCConsole
>         = new Poco::FormattingChannel(
>         Poco::AutoPtr(pf));
>
>     Poco::AutoPtr pFileChannel = new Poco::FileChannel();
>
>     std::string logFilePathBase = "PocoLog.log";
>
>     try
>     {
>         pFileChannel->setProperty(Poco::FileChannel::PROP_PATH,  logFilePathBase);
>         Poco::AutoPtr pAsync = new Poco::AsyncChannel(pFileChannel);
>         pFCConsole->setChannel(pAsync);
>         pFCConsole->open();
>
>         Poco::Logger::root().setChannel(pFCConsole);
>         Poco::Logger::root().setLevel(Poco::Message::PRIO_TRACE);
>
>     }
>     catch(Poco::Exception& exc)
>     {
>
>         CHECK(0);
>     }
>
>     // ... Start pushing logs
>     for (int idx = 0; idx < 15000; idx++)
>     {
>         std::ostringstream str;
>         str      }
>
>     UnitTest::waitForInput();
> }
>

>
axel
 
Posts: 2
Joined: 28 Feb 2008, 22:09

Re: AsyncChannel memory leak?

Postby axel » 05 Mar 2008, 19:16

I've had a similar problem but with a dead lock in AsyncChannel. The thread responsible for cleaning up queue was dying before having a chance to process messages and the main thread was hanging in close method, waiting for queue to be cleaned up which of course never happened. I've change close method to check if thread responsible for cleaning up the queue is still running. I've also included a change for Solaris compiler, since pthread_join is not detecting a dead thread.

void AsyncChannel::close()
{
if (_thread.isRunning())
{
while ( _thread.isRunning()
&& !_queue.empty() ){
Thread::sleep(100);
}
#if !defined(__SUNPRO_CC)
_queue.wakeUpAll();
_thread.join();
#endif
}
}

Since you don't have any control on thread created by async channel, you need to wait a couple of minutes at the end of your program, probably on Unitialize method.



axel
 
Posts: 2
Joined: 28 Feb 2008, 22:09


Return to Support

Who is online

Users browsing this forum: No registered users and 2 guests

cron