Overview
Features
Download
Documentation
Community
Add-Ons & Services

memory leak when inserting using ODBC connector

Please post support and help requests here.

memory leak when inserting using ODBC connector

Postby gdavis » 05 Apr 2010, 21:30

I modified the poco-1.3.6p2-all\Data\samples\Binding example to use the ODBC connector instead of SQlite's and put a loop around the insert statements within. The resulting application, compiled with Visual Studio 2005 under Windows XP, grows in memory considerably while it is doing these inserts. Has anyone else seen this problem and/or found a workaround? Thanks!
gdavis
 
Posts: 3
Joined: 05 Apr 2010, 19:43

Re: memory leak when inserting using ODBC connector

Postby alex » 06 Apr 2010, 14:19

Let's try to narrow it down: does the same thing happen with your loop modification when using SQLite connector?
alex
 
Posts: 1114
Joined: 11 Jul 2006, 16:27
Location: United_States

Re: memory leak when inserting using ODBC connector

Postby gdavis » 06 Apr 2010, 19:28

When using the SQLite connector the process grows ~150K over 2000 iterations (with 2 inserts per iteration). For ODBC over the same number of iterations the process grows ~3Mb.
gdavis
 
Posts: 3
Joined: 05 Apr 2010, 19:43

Re: memory leak when inserting using ODBC connector

Postby alex » 06 Apr 2010, 19:39

I'd say the size difference is likely due to the internals of the respective DB libraries. Can you post your code?
alex
 
Posts: 1114
Joined: 11 Jul 2006, 16:27
Location: United_States

Re: memory leak when inserting using ODBC connector

Postby gdavis » 06 Apr 2010, 22:49

I would expect a difference in memory usage depending on which connector is chosen, but I would hope the insert loop would use a constant amount of memory as I have a database client that needs to be able to insert an arbitrary number of rows over a long time period. Here is the loop (everything else is the same as the original Binding example, with ifdefs to select either ODBC or SQLite):

Code: Select all
   const int MAXITER = 10000;

   for(int i=0;i<MAXITER;i++) {
      Statement insert(session);

      // insert some rows
      Person person =
      {
         "Bart Simpson",
         "Springfield",
         12
      };

      insert << "INSERT INTO Person VALUES(?, ?, ?)",
         use(person.name),
         use(person.address),
         use(person.age);
         
      insert.execute();
      
      person.name    = "Lisa Simpson";
      person.address = "Springfield";
      person.age     = 10;
      
      insert.execute();
   }


I just verified that the above code requires a constant amount of memory under Linux (Ubuntu 8.04 LTS).

I tried using a prepared statement and found the resulting application also uses memory proportional to the number of loop iterations. Looking into this a bit more deeply, I noticed that ODBC::Binder's _lengthIndicator vector grew for every iteration of the loop:
Code: Select all
   
Statement insert = ( session << "INSERT INTO Person(name,address,age) VALUES(?,?,?)",
                               use(person.name), use(person.address), use(person.age) );
while(i--) {   
    insert.execute();
}
gdavis
 
Posts: 3
Joined: 05 Apr 2010, 19:43

Re: memory leak when inserting using ODBC connector

Postby saurabhdave2000 » 13 Dec 2010, 14:51

Hi Alex/GDavis,

Have you guys been able to find the solution for the leak?

I am facing the same issue, basically my code connects of PGSQL via POCO+psqlodbc and every insert/update statement is causing memory leak.

Appreciate any help from your side.

Thanks,
Saurabh
saurabhdave2000
 
Posts: 2
Joined: 13 Dec 2010, 14:47

Re: memory leak when inserting using ODBC connector

Postby Giovani » 12 Apr 2013, 16:53

Im using poco 1.5.1 and postgresql odbc driver version 09.00.0310-1
Still getting leaks..
Any ideas?
Giovani
 
Posts: 5
Joined: 26 Nov 2012, 19:45

Re: memory leak when inserting using ODBC connector

Postby alex » 12 Apr 2013, 23:24

alex
 
Posts: 1114
Joined: 11 Jul 2006, 16:27
Location: United_States

Re: memory leak when inserting using ODBC connector

Postby alex » 22 Apr 2013, 07:50

The only thing I was able to find is leak in PostgreSQL ODBC driver, see https://github.com/pocoproject/poco/issues/152 for details.
alex
 
Posts: 1114
Joined: 11 Jul 2006, 16:27
Location: United_States

Re: memory leak when inserting using ODBC connector

Postby Giovani » 24 Apr 2013, 19:13

Here are some results of my odbc tests.

Postgresql odbc driver version: 09.00.0310-1
O.S: Ubuntu 32 bits 3.0.0-27-generic kernel

Pure ODBC. Leaks nothing.
Notice that im using SQLDescribeCol(supposed to leak).

Code: Select all
#include <sql.h>
#include <sqlext.h>
#include <stdio.h>
#include <string.h>
#include <string>

int main()
{
   SQLHENV env;
   SQLHDBC dbc;
   SQLHSTMT stmt;
   SQLRETURN ret; /* ODBC API return status */
   SQLCHAR outstr[1024];
   SQLSMALLINT outstrlen;

   /* Allocate an environment handle */
   SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &env);
   /* We want ODBC 3 support */
   SQLSetEnvAttr(env, SQL_ATTR_ODBC_VERSION, (void *) SQL_OV_ODBC3, 0);
   /* Allocate a connection handle */
   SQLAllocHandle(SQL_HANDLE_DBC, env, &dbc);
   /* Connect to the DSN mydsn */
   ret = SQLDriverConnect(dbc, NULL, (SQLCHAR*)"DSN=test;UID=bry_admin;PWD=bry_admin123;ByteaAsLongVarBinary=1;", SQL_NTS,
         outstr, sizeof(outstr), &outstrlen,
         SQL_DRIVER_COMPLETE);
   if (SQL_SUCCEEDED(ret)) {
      printf("Connected\n");
      printf("Returned connection string was:\n\t%s\n", outstr);
   } else {
      fprintf(stderr, "Failed to connect\n");
   }

   /* Allocate a statement handle */
   ret = SQLAllocHandle(SQL_HANDLE_STMT, dbc, &stmt);
   std::string sql = "insert into test(cod, cod_value) values(0, 'aa')";
   ret = SQLExecDirect(stmt, (SQLCHAR*)sql.c_str(), SQL_NTS);
   if(ret == SQL_SUCCESS || ret == SQL_SUCCESS_WITH_INFO) {
      printf("Insert data OK!!. SQL was:\n\n%s\n\n");
   }

   SQLSMALLINT columnCount;
   SQLSMALLINT colNumber;
   SQLCHAR colName[80];
   SQLSMALLINT nameLen;
   SQLSMALLINT dataType;
   SQLULEN colSize;
   SQLSMALLINT decimalDigits;
   SQLSMALLINT nullable;

   sql = "SELECT cod FROM test";
   ret = SQLExecDirect(stmt, (SQLCHAR*)sql.c_str(), SQL_NTS);
   SQLNumResultCols(stmt, &columnCount);
   for (int i=0; i<columnCount; i++ )
   {
       SQLDescribeCol(stmt, i+1, colName, sizeof(colName),
                      &nameLen, &dataType,
                      &colSize, &decimalDigits, &nullable);
   }

   SQLFreeHandle(SQL_HANDLE_STMT, stmt);
   SQLDisconnect(dbc);
   /* free up allocated handles */
   SQLFreeHandle(SQL_HANDLE_DBC, dbc);
   SQLFreeHandle(SQL_HANDLE_ENV, env);
}


SQLAPI with odbc support. Leaks nothing(on insert).
Code: Select all
#include "SQLAPI.h"
#include "odbcAPI.h"
int main()
{
SAConnection conn;
conn.Connect("YOUR CONNECTION STRING HERE", "USERNAME","PASSWORD", SA_ODBC_Client);
SACommand cmd(con, "INSERT INTO test( cod, cod_value ) VALUES (:cod, :cod_value)");
int i = 0;
cmd.Param("cod").setAsLong() = i;
cmd.Param("cod_value").setAsString() = link.c_str();
cmd.Execute();
}

Poco ODBC. Leaks.
Code: Select all
   ODBC::Connector::registerConnector();
   Session ses(SessionFactory::instance().create("ODBC", "YOUR CONNECTION STRING HERE"));
   int value = 0;
   std::string aa("aa");
   ses << "insert into test(cod, cod_value) values(?, ?)", use(value), use(aa), now;
   ODBC::Connector::unregisterConnector();

Table used:
CREATE TABLE test
(
cod integer,
cod_value character varying
)

But i also could not find any problem in poco odbc...
Giovani
 
Posts: 5
Joined: 26 Nov 2012, 19:45

Next

Return to Support

Who is online

Users browsing this forum: No registered users and 1 guest

cron