Re: [libdbi-users] Found and fixed: dbd_mysql thread unsafe problem
/libdbi.so.1.1.0) ==2828==by 0x134BFA: dbSetupConnection (database.c:211) ==2828==by 0x13D2F4: GetItemMetaData (database.c:2960) ==2828== ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Lib DBI windows version
Hi, "dead" is relative here. It is unfortunately pretty much true that I was not able to put much of an effort into further development of libdbi, although there were a couple of bug fixes in the past couple of years. I don't mean to whine, but sometimes life takes unexpected turns, and in my case they're 6 and 3 years old now. Needless to say, this, and a new position with extended responsibilities, caused a shift of priorities that won't do my software projects any good. I'd hate to see libdbi die of inactivity, but it is hardly possible to sleep less than I did in the past couple of years just to do some extra programming. And I apologize if I was not able to tend to questions on the lists in a timely fashion. Regarding the original posters question: you can certainly use libdbi on Linux and just about any unixish platform. This includes Cygwin (http://www.cygwin.com), a library which provides a Unix-like interface to Windows. To the best of my knowledge there is no native Windows port available which would allow you to use it in native Windows applications. Source code is available from a git repository at SourceForge: https://sourceforge.net/p/libdbi/libdbi/ci/master/tree/ Documentation of the dbi interface is here (the Programmer's Guide is what you want): http://libdbi.sourceforge.net/docs.html regards, Markus On 2018-12-29 03:25, Simon Walter was heard to say: > Hi Pabitra, > > I asked a question on this list in 2015 and there has been no activity since > then. I didn't get an answer to my question. So I assumed that the project is > pretty dead. There is support for prepared statements. So I ended up using > APR DBD. The documentation is alright. Though I did find myself digging into > the source to understand a few things. > > I haven't used it on Windows, but the whole point of the APR is cross > platform usage. > > https://apr.apache.org/docs/apr-util/1.6/group___a_p_r___util___d_b_d.html [2] > > Best wishes, > > Simon > > On December 28, 2018 4:28:42 PM UTC, Pabitra Dash via libdbi-users > wrote: > >> Hello, >> >> Good day. >> I have following queries. >> >> -As libdbi provides standard C abstraction, Can I use it in Windows and >> Linux platforms? >> -From where can I get DbiXX detail documentation and source code of the >> same? >> >> Regards, >> Pabitra > > ___ > libdbi-users mailing list > libdbi-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/libdbi-users [1] -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 Links: -- [1] https://lists.sourceforge.net/lists/listinfo/libdbi-users [2] https://apr.apache.org/docs/apr-util/1.6/group___a_p_r___util___d_b_d.html___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Problem getting SQLite last sequence value
lled on string
> field %s", int_string);
> }
>
> dbg('s', LOG_INFO, "Returning integer: %d", *buffer);
>
> return (0);
> }
>
> Could this behavior relate to the dbi_conn reference passed? But then,
> why
> it works with MySQL and not with SQLite?
>
> Or maybe it's the table schema? Then why everything works fine from the
> sqlite3 shell?
>
> Thank you very much.
> Ciao!
Hi,
unfortunately I didn't find the time to actually reproduce your problem.
However, I've found the following conspicuous line in dbd_sqlite3.c
(line 1236 in my copy):
|| strstr(curr_type, "INTEGER PRIMARY KEY") /* BAD BAD HACK */
It is quite possible that the "bad bad" part of this hack came to life
in your example. Could you please test your code again with a slightly
modified table definition like this:
inode integer primary key not null autoincrement,
Please let me know if this helps.
regards,
Markus
--
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38
--
___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] prepared statements
At 2015-06-11 09:12, Simon Walter was heard to say: > Hi all, > > I'm new to using libdbi and C in general. I am wondering why libdbi has > no support for prepared statements. > > Is it that because the way libdbi is written, a prepared statement > would > not provide any benefit? Or is it just lacking that feature? If it's > just lacking the feature, are there plans to add that? > > If the application I'm writing can benefit from prepared statements, I > would like to eventually either add that feature to libdbi or I may > have > to use something else in the future. > > I haven't taken a deep look at the libdbi code. So any insight is > appreciated. > > Kind regards, > > Simon > > PS > > If I should have sent this to the development list, please let me know. > Thanks. Hi, I can't tell if prepared statements would speed up things when accessing databases through libdbi. As in many free software projects, the main reason for the lack of prepared statements is that nobody invested his time in implementing them. This includes myself, although I can't recall many requests for this feature. However, I think libdbi could support prepared statements if someone takes care of it. My commitments to my dayjob and to my family do not leave much dev time currently. If you think about helping to implement this feature, you need to consider both libdbi and libdbi-drivers. libdbi needs a generic interface to prepared statements that covers all database backends. The drivers have to implement the feature using the functions of their respective client libraries. Therefore, it is best to design and discuss the interface first, and then to move on from there. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Valgrind error with pgsql: Conditional jump or move depends on uninitialised value(s)
At 2014-11-04 11:41 Alex Bligh was heard to say: > On 3 Nov 2014, at 22:54, Markus Hoenicka > wrote: > >> I wouldn't bet from valgrind's output that it is libdbi variables >> which are uninitialized. Can you re-run your test with a different >> database engine? I'd suggest using the sqlite3 driver as this engine >> has few if any external dependencies. > > I reran with mysql and it doesn't appear. I presume it's the pgsql dbi > driver. This is one explanation. Another explanation is that the PostgreSQL client library or one of the libraries it depends on causes these messages. Could you please fire up the psql command line utility under valgrind and do what your test program does, i.e. establish a connection? If this test does not report unitialized variables, we'll have to revisit the pgsql driver. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Valgrind error with pgsql: Conditional jump or move depends on uninitialised value(s)
At 2014-11-02 19:05 Alex Bligh was heard to say: > I have a very simple program (below) only marginally adapted from the > example here: > > http://libdbi.sourceforge.net/docs/programmers-guide/quickstart-code.html > > which works, but when run through valgrind produces an enormous number > of errors. > The program in its entirety is below. It makes a connection to pgsql, > does no > queries, and immediately shuts the connection. > > I'm not so much worried about the alleged memory leaks but the 85 > instances of > "Conditional jump or move depends on uninitialised value(s)". I don't > know if this is a real problem, but even if not the challenge here > is that /any/ query via dbi is now tainted as uninitialised, so it's > impossible > to see 'real' errors. > > I've not attached a full valgrind dump as it's enormous. However, you > can see > the first two below. > > Any ideas? > Hi, I wouldn't bet from valgrind's output that it is libdbi variables which are uninitialized. Can you re-run your test with a different database engine? I'd suggest using the sqlite3 driver as this engine has few if any external dependencies. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] DB2 support
At 2014-09-09 23:50 S M was heard to say: > I installed the 0.8.3 libdbi/libdbi-devl packages from the yum repos. I did > clone the latest version from git but there was configure file to compile > it.-- If you work straight from a repo you need to bootstrap your local sources. Both libdbi and libdbi-drivers contain an autogen.sh script in the top-level directory which will generate an appropriate configure script for you. regards Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Want excitement? Manually upgrade your production database. When you want reliability, choose Perforce. Perforce version control. Predictably reliable. http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] DB2 support
At 2014-09-08 15:50 S M was heard to say: > Hi, > > I'm trying compile libdbi with DB2 support. I've downloaded the latest > version (libdbi-drivers-0.9.0) and exec'ed > > ./configure --with-db2 --with-db2-incdir=/opt/ibm/db2/V10.5/include > --with-db2-dir=/opt/ibm/db2/V10.5 > > That part worked. But when running 'make' it fails with: > [...] > dbd_db2.c: At top level: > dbd_db2.c:267: error: conflicting types for 'dbd_goto_row' > /usr/include/dbi/dbd.h:41: note: previous declaration of 'dbd_goto_row' was > here Hi, this smells like a mismatch between your libdbi headers/libraries and the driver version you're trying to build. The dbd_goto_row() prototype and driver implementations were changed in early 2013. Make sure you have a matching libdbi 0.9 for your libdbi-drivers 0.9, or better yet, use the git versions of both if you're fiddling with experimental drivers. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Want excitement? Manually upgrade your production database. When you want reliability, choose Perforce Perforce version control. Predictably reliable. http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi and SQL Injection
At 2014-01-10 15:56, Rick Robinson was heard to say:
> I have tried the following, which I believe is on the right track but the
> program crashes as soon as it gets to dbi_conn_quote_string_copy. I try to
> use dbi_conn_quote_string_copy to keep the bad string locked down, and then
> use dbi_conn_queryf to read the new locked down value into the db. Sadly, I
> am not the greatest C coder so I am probably overlooking something relatively
> simple to make this work.
>
> char *pTitle=NULL;
>
> char *pTitle2=NULL;
>
> if ((ch)->player.title)
> {
> sprintf (pTitle, (ch)->player.title);
> }
> else
> {
> sprintf (pTitle, "None");
> }
>
> dbi_conn_quote_string_copy(conn, pTitle, &pTitle2);
>
> sprintf (sql_columns, "name, title");
> sprintf (sql_string, "REPLACE into data (%s) VALUES ("%s", "%s")",
> sql_columns,
> GET_NAME (ch),
> pTitle2 );
>
> result = dbi_conn_queryf (conn, sql_string);
Well, this is not a list about C coding, but it looks like you sprintf()
some string to a non-allocated buffer. I'm just wondering why the code
doesn't crash in sprintf() then.
In any case, it is far simpler to use dbi_conn_quote_string_copy() on
the assembled SQL query string rather than on single items. E.g. the
quoted string is surrounded by escape characters, and you subsequently
wrap another pair around the title. This does not seem to generate valid
SQL. Also, if you don't use the printf()-like capabilities of
dbi_conn_queryf(), there is no point in using this function. Use
dbi_conn_query() instead.
regards,
Markus
--
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38
--
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi and SQL Injection
Am 2014-01-10 14:17, schrieb Markus Hoenicka: > if I understand you correctly, you attempt to insert a value containing the > string "%s Saints going down tonight!" using the libdbi function > dbi_conn_queryf(). Thing is, dbi_conn_queryf() is intended to make > dbi_conn_query() behave somewhat like sprintf() in that you can specify a > formatting string containing placeholders like "%s", followed by parameters > that are filled in. If you want to preserve the "%s" literally, you either > need to escape or quote the values properly, or you should rather use > dbi_conn_query() which sends the string parameter to the db engine literally. > You still need to watch out for proper quoting and escaping as per the > language specs of your db engine. I forgot to mention that the dbi_conn_quote_string() and dbi_conn_escape_string() functions and their relatives can do the quoting and escaping for you, please check the manual. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi and SQL Injection
Am 2014-01-09 18:49, schrieb Rick Robinson:
> I am new to both DBs and this library. I am attempting to save a string and
> insert it into my MYSQL DB.
>
> An example of the data being read into the db is here:
>
> The name is Krolps, and title is everything after it:
> Krolps %s Saints going down tonight!
>
> The title entry in the MYSQL DB is turned into
> name, title
>
> Here is my small program below:
>
> void
> website_pfile_phase_one (struct char_data *ch)
> {
> dbi_conn conn;
> dbi_result result;
> dbi_inst instance;
>
> dbi_initialize_r (NULL, &instance);
> conn = dbi_conn_new_r ("mysql", instance);
>
> dbi_conn_set_option (conn, "host", "box");
> dbi_conn_set_option (conn, "username", "username");
> dbi_conn_set_option (conn, "password", "password");
> dbi_conn_set_option (conn, "dbname", "dnbname");
> dbi_conn_set_option (conn, "encoding", "UTF-8");
>
> if (dbi_conn_connect (conn) < 0)
> {
> puts ("Could not connect. Please check the option settingsn");
> }
> else
> {
> char sql_string[MAX_STRING_LENGTH], sql_columns[MAX_STRING_LENGTH];
>
> sprintf (sql_columns,
> "name, title");
> sprintf (sql_string,
> "REPLACE into data (%s) VALUES ("%s", "%s")",
> sql_columns, GET_NAME (ch), ch->player.title ? ch->player.title : "None") );
>
> result = dbi_conn_queryf (conn, sql_string);
>
> if (result)
> {
> puts ("SUCCESS Website Data");
> }
> dbi_result_free (result);
> }
> dbi_conn_close (conn);
> puts ("END of website data");
> dbi_shutdown_r (instance);
> }
>
> Any suggestions on what I am doing wrong? My guess is I need to chanfge
> dbi_conn_queryf to something else?
Hi,
if I understand you correctly, you attempt to insert a value containing
the string "%s Saints going down tonight!" using the libdbi function
dbi_conn_queryf(). Thing is, dbi_conn_queryf() is intended to make
dbi_conn_query() behave somewhat like sprintf() in that you can specify
a formatting string containing placeholders like "%s", followed by
parameters that are filled in. If you want to preserve the "%s"
literally, you either need to escape or quote the values properly, or
you should rather use dbi_conn_query() which sends the string parameter
to the db engine literally. You still need to watch out for proper
quoting and escaping as per the language specs of your db engine.
hope this helps
Markus
--
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38
--
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] (no subject)
Am 2013-06-11 10:53, schrieb Flur Blubr: > ./configure --prefix=/apps/cactiprod/syslog-ng/libdbi-drivers --with-dbi-incdir=/apps/cactiprod/syslog-ng/libdbi/include/dbi --with-dbi-libdir=/apps/cactiprod/syslog-ng/libdbi/lib --with-mysql --with-mysql-libdir=/apps/cactiprod/mariadb/lib --with-mysql-incdir=/apps/cactiprod/mariadb/include/mysql --disable-docs > > [...] dbd_mysql.c:50:21: error: dbi/dbi.h: No such file or directory Hi there, I haven't tested this yet, but you should get better results if you specify the libdbi include directory as "--with-dbi-incdir=/apps/cactiprod/syslog-ng/libdbi/include" (no trailing "/dbi"). The sources reference the include files as "dbi/foo.h", so the trailing "dbi" in your specification confused the preprocessor. NB the --XX-incdir and --XX-libdir options should be avoided anyway and should probably have been removed or at least be deprecated. For a cleaner solution, please define CFLAGS and LDFLAGS appropriately before running configure and make. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] libdbi and libdbi-drivers versions 0.9.0 released, finally
Hi, unless you've already noticed I'd like to draw your attention to the fact that after years of heavy development :-) libdbi and libdbi-drivers are available at version 0.9.0. Although development definitely took way too long due to my enormous workload in my dayjob during the past couple of years (plus some extra burdens in my private life), the releases are finally out in the wild. I hope that all who have contributed code, ideas, and bug reports can be proud of the result. I'd like to specifically point out the new test kit, the concept of libdbi instances, transaction support, and the enormous speedup of reading MySQL result sets sequentially, all of which were either coded or requested by avid users. This holds true even if most of the cvs checkins carry my name - I've checked in lots of things originally provided by others. I'll update the web pages asap (see above). regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] firebird driver segfault
markus.hoeni...@mhoenicka.de writes: > Christoph Kottke writes: > > Am 19.02.2013 00:16, schrieb markus.hoeni...@mhoenicka.de: > > > Did you ever succeed in running the new test > > > kit? Or the old one shipped with 0.8.3 fwiw? > > > > hi, > > > > i can't remeber on old kit passes and the new one has never pass. > > > ... even more findings After perusing the available documentation I concluded that it is not possible to drop tables from within a C program. Apparently you have to resort to isql/isql-fb in order to get rid of them. I modified our test program to do just that. Things work ok until just after dropping a trigger and a generator (both of which are created when creating the test table. I don't know what these are good for, someone else coded that). The next step would be to actually drop the table using isql. However, at this point the firebird client library segfaults, although I can't see why. It is not being called at this point. The test table is still present after the crash. In any case I concluded that firebird is above my head. I've spent an entire week trying to get a simple test case running, to no avail. I'm going to release libdbi-drivers without official firebird support unless someone steps up and takes a look at the firebird driver and the tests. best regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] firebird driver segfault
Christoph Kottke writes: > Am 19.02.2013 00:16, schrieb markus.hoeni...@mhoenicka.de: > > Did you ever succeed in running the new test > > kit? Or the old one shipped with 0.8.3 fwiw? > > hi, > > i can't remeber on old kit passes and the new one has never pass. > Hi, maybe you can shed some light on my latest findings: - isql-fb can connect from a FreeBSD box to a firebird server running on Debian - the libdbi-drivers test_dbi program (cvs revision) connects to the server, successfully creates a database, but then immediately hangs forever - the *old* test program as of approx. 0.8.3 runs most tests ok when run locally on Debian. However, trouble starts when the program tries to get rid of the table that was used during the tests - the "DROP TABLE" command causes an error saying "unsuccessful metadata update" - I took from several web resources that it may be necessary to explicitly commit a transaction before dropping a previously used table. However, this causes a "Dynamic SQL Error" - this, in turn, seems to be fixed by enclosing the table name in double quotes. However, this now causes a "invalid transaction handle (expecting explicit transaction start)" kind of error - if I try to fix this by issuing a "SET TRANSACTION" command, we're back to "Dynamic SQL error", full circle Can you confirm, from your experience, that firebird is kind of picky about dropping tables which were previously used? This is kind of weird, as any serious database should support dropping tables. How do you deal with this in your own programs? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] firebird driver segfault
Christoph Kottke was heard to say: > Am 19.02.2013 00:16, schrieb markus.hoeni...@mhoenicka.de: >> Did you ever succeed in running the new test >> kit? Or the old one shipped with 0.8.3 fwiw? > > hi, > > i can't remeber on old kit passes and the new one has never pass. > Hi, I've tried a few things with the new testkit in the past couple of days. However, it seems we're asking too much of firebird if we intend to create and drop tables with each test - this results in lock file errors which I was not yet able to resolve. gdb isn't cooperative at least on Linux, so I can't even tell exactly which SQL command kills the server. My tried and true "printf debugger" wasn't helpful either. I built the firebird client on FreeBSD yesterday. I'll try and see if gdb is more up to the task on that platform. In any case, going back to your original problem report: could you please provide two simple but complete test cases, one which is supposed to succeed (i.e. which retrieves a single row of data) and one which fails (which retrieves three or more rows, according to what you mentioned). Both test cases should be complete in that they create the database, create the table, insert the data, and clean up after themselves when the test succeeds. It would be a major step forward towards debugging if I was able to run these test cases on my box. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] firebird driver segfault
Christoph Kottke writes: > some more hits, > > if you use xinet.d you must enabled by hand in > /etc/xinet.d/firebird.conf and restart xinetd. > and new created db can only access by SYSDBA before the are activated... > Hi, one reboot later things have improved considerably. I tried to start the superserver process manually yesterday. There were no error messages, but the commands apparently did not succeed. After I booted the box tonight, the server was started correctly, and I was at least able to connect to the test database. I started to fiddle with tests/test_dbi in order to get some tests running. The test database is created ok using isql-fb, and I can connect to that (empty) database manually. I also found out the hard way that you have to run ./test_dbi as root to avoid error messages like: Can't access lock files' directory /tmp/firebird However, running the test as root results in error messages like (printed using printf using a modified debug firebird driver): Fatal lock manager error: invalid lock id (19892), errno: 11 --Resource temporarily unavailable I'll have to investigate which commands exactly cause this error messages, but at this time the test program does not run to completion using this driver. Did you ever succeed in running the new test kit? Or the old one shipped with 0.8.3 fwiw? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials, tech docs, whitepapers, evaluation guides, and opinion stories. Check out the most recent posts - join the conversation now. http://goparallel.sourceforge.net/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] firebird driver segfault
Christoph Kottke was heard to say: > Hi, > > thanks for your investigating :) > > i've run an debian 6.0.6 on i386 and install the lastest firebird-1.5 > from http://firebirdsql.org/ > because libdbi-driver depends on old include files (gds.h, etc). but you > can use the tarball only > to compile and use the server from debian repository. > > i 've never had problems to install an firebird server on debian, it > works always out of the box. > > the server is: firebird2.5-classic > Hi, I've tried both classic and super. In order to get the latter up and running, you're supposed to configure the package like this (as root): dpkg-reconfigure firebird2.5-super Even that command fails, saying it cannot connect. As this is still part of the installation, I have to consider this package broken. Classic does not require this step, but it doesn't work either. I'm sure that I miss some critical step in the procedure. What about accounts: do you run your queries from a regular user account? root? firebird? Also, where do you create databases? What are the permissions of that directory, and who owns it? Do you use localhost in the database specifier, do you use any other host information, or none at all? I'll give it another try tonight, but of course there is no point in delaying libdbi-drivers 0.9 any further if I don't have a chance to fix the firebird driver anyway. We'd have to move that driver to the "code present but unsupported" vault. BTW I'm running Debian Wheezy i386 on the test box. I've installed it from scratch on Saturday, so it shouldn't be screwed up in any way that affects firebird. If I include Debian's weird non-standard library directories, I don't have any problems building the firebird driver from the headers and libraries provided by the Debian packages. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials, tech docs, whitepapers, evaluation guides, and opinion stories. Check out the most recent posts - join the conversation now. http://goparallel.sourceforge.net/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] firebird driver segfault
Christoph Kottke writes: > hi, > > i've installed the latest cvs revisions for libdbi and libdbi-driver and > when ever i try to fetch more than > 3 row from an firebird table it's end in an segfault. > > but when i revert the "bull patch" in dbi_result.c it works like a charm. > Hi, I'd be happy to look at this issue. However, I haven't been able to set up firebird on Debian either. I've followed the excellent Ubuntu instructions which should work for Debian as well: https://help.ubuntu.com/community/Firebird2.5 However, no luck. isql-fb never manages to make a connection. I cannot create database, and I can't access the example database either. This database drives me nuts, I haven't seen it work on any platform here in the past couple of years. In order to have a chance to debug the driver, would you please share some information how you succeeded in running firebird at all. Again, I'm not talking about libdbi or libdbi-drivers problems. I just can't get the engine itself to work. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials, tech docs, whitepapers, evaluation guides, and opinion stories. Check out the most recent posts - join the conversation now. http://goparallel.sourceforge.net/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] firebird driver segfault (c/o Christian???)
Christoph Kottke writes: > hi, > > i've installed the latest cvs revisions for libdbi and libdbi-driver and > when ever i try to fetch more than > 3 row from an firebird table it's end in an segfault. > > but when i revert the "bull patch" in dbi_result.c it works like a charm. > Hi, thanks for the bug report. This one barely came in time to stop me from packaging the long-awaited 0.9 releases :-/ Unfortunately, I haven't been able to make firebird operational on my FreeBSD box for at least two years so I cannot do any real debugging here. I plan to set up a Debian box shortly which may be a better platform for this database engine. It took me a couple of hours to more or less understand what's going on in theory. In any case, I've got a hunch. I assume the "bull patch" was correct as it fixed a memory leak shown by valgrind, without causing any problems with the drivers that I can test over here. What if the firebird driver just happened to work ok as long as libdbi prevented one particular bordercase to occur, at the expense of some bits of leaked memory? In that case it would be wrong to revert the patch. Instead, someone in the know (Christian please ???) would have to review the result fetching code of the firebird driver with an eye on numrows_matched. According to what I've seen in the driver code, firebird makes it particularly hard to get things right. Also, Christoph, did you run make check? If that didn't crash, we need extra tests to discover problems like these. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] pgsql problems
markus.hoeni...@mhoenicka.de writes: > Hi, > > I've moved libdbi to 0.9.0 in CVS and I've hoped to get libdbi-drivers > release-ready as well. I've installed PostgreSQL 9.2 today to see if > things worked as smoothly as they did a while back. I had to learn > that the test harness doesn't manage to get through the database > encoding test. After disabling that one, I still get 47 failures, > while both mysql and sqlite3 succeed without a hitch. Is anyone using > PostgreSQL with libdbi, and if yes, which versions? kinda talking to myself, but maybe someone's still listening... Turns out that there were both encoding and binary data issues. PostgreSQL >= 9.0 uses hex-encoded binary data which screwed up all tests related to BYTEA data. I had to rewrite the entire BYTEA decoding stuff in the pgsql driver to handle this new format. We're now back at 0 errors, but I can test this against PostgreSQL 9.2 only. Would someone be kind enough to test the code on older versions of this engine? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] pgsql problems
Hi, I've moved libdbi to 0.9.0 in CVS and I've hoped to get libdbi-drivers release-ready as well. I've installed PostgreSQL 9.2 today to see if things worked as smoothly as they did a while back. I had to learn that the test harness doesn't manage to get through the database encoding test. After disabling that one, I still get 47 failures, while both mysql and sqlite3 succeed without a hitch. Is anyone using PostgreSQL with libdbi, and if yes, which versions? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Toby Thain writes: > Yes, you just found out the hard way that it's wise to include ENGINE > specifications in MySQL CREATE TABLE's. > True indeed. The problem with MySQL's storage engines is that the transaction_support and savepoint_support driver capabilities cannot be relied upon because they depend on a couple of factors - how the database server was compiled and configured, how particular tables were created and so on. The driver caps may get things wrong in quite a few cases. BTW I've finished updating the test kit which should now cover all aspects of transactions and savepoints as well. I'd greatly appreciate if everyone could give the current cvs revisions a try and report success and failures. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Markus Hoenicka writes: > Another possible reason for problems is the table type. If someone > uses MyISAM tables, the result I got would be expected. However, MySQL > uses InnoDB tables as default these days, and I double-checked that > they were used in my tests. Turns out that InnoDB is the default engine only in versions 5.5.5 and later. On my box, MyISAM tables were created by default. The test correctly found out that the latter don't support transactions :-( I've changed the CREATE TABLE statement on my box to ask for InnoDB tables instead. This eliminates the failure of the rollback test. I'll try and finish the test suite asap. The transaction tests require some polish as they should run only if the driver supports transactions. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Rainer Gerhards was heard to say: > we have given it a try today and things look pretty good :-). Sounds good :-) > Unfortunately, we can reproduce the problem with MySQL. I barely > remember that MySQL by default has implicit commits enabled, what > needs to be turned off if you need real ones. > As Olivier already mentioned, any eplicit START TRANSACTION overrides autocommit, so this shouldn't interfere here. Even if an application using libdbi turns autocommit off or on deliberately, START TRANSACTION should still work as expected. Another possible reason for problems is the table type. If someone uses MyISAM tables, the result I got would be expected. However, MySQL uses InnoDB tables as default these days, and I double-checked that they were used in my tests. I'll try to test the pgsql driver on the weekend to see if that causes problems too. I'll also check the MySQL logs to see if I find something weird, and I'll run some transactions in MySQL's plain ol' command line interface just to make sure it isn't MySQL playing tricks on us. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Rainer Gerhards writes: > > Zitat von Rainer Gerhards : > > > > > sorry for the long silence, we got sidetracked ourselfs. Finally, we > > > yesterday tried to write some test programs to get started. To do so, > > > I did a cvs checkout for both libdbi and libdbi-drivers (according to > > > instructions on the site). I can see the new transaction functions > > > inside the sgml files as well as the headers. > > > However, I do not find any implementation (.c files). > > > > > > Am I overlooking something? > > > > > > > No. Apparently my bad. I'll fix that tonight. > > Thanks a lot! > Rainer I've checked in new versions of libdbi/src/dbi_main.c containing the implementations of the transaction functions and libdbi-drivers/tests/test_dbi.c with a first shot at the required test functions. MySQL seems to fail to rollback transactions on my box whereas SQLite3 succeeds. I'm too tired now to track this down, maybe someone else can have a look at this too. I hope I'll get back to this on the weekend. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Zitat von Rainer Gerhards : > sorry for the long silence, we got sidetracked ourselfs. Finally, we > yesterday tried to write some test programs to get started. To do > so, I did a cvs checkout for both libdbi and libdbi-drivers > (according to instructions on the site). I can see the new > transaction functions inside the sgml files as well as the headers. > However, I do not find any implementation (.c files). > > Am I overlooking something? > No. Apparently my bad. I'll fix that tonight. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi+mysql speed issues
Olivier Doucet writes: > Performance are now correct :) I checked with valgrind / callgrind, > and function mysql_data_seek is not called anymore. > > You can now patch other drivers, and maybe release a new version ? I've updated the remaining drivers and the driver documentation. Did anyone else have a chance to test the code lately? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi+mysql speed issues
Olivier Doucet was heard to say: >> Do you have any numbers? Would be nice to know what speed gain this >> (almost) one-line optimization effected. > > I've done a benchmark with rrdtool that uses libdbi. > http://tof.canardpc.com/view/31953cab-4329-45e9-9e30-eed2d9148ee1.jpg > max value on X-Axis is equivalent to ~ 35k rows retrieved from MySQL. > > Quite impressive, right ? ;) > Looks like my 30 min effort was well spent then. Thanks for kicking my butt to get this done. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi+mysql speed issues
Olivier Doucet was heard to say: Hi, > I finally succeeded in compiling both component for my system (I added > flag -lpthread when both linking / compiling or else I have following > error : '/usr/lib64/dbd/libdbdmysql.so: undefined symbol: > pthread_mutex_trylock'). libdbi-drivers uses the mysql_config script shipped with MySQL to obtain the list of required libraries to link against, *unless* you use the --with-mysql-libdir configure switch. Please try and see if libdbi-drivers compiles fine without that switch. Otherwise try manually what "mysql_config --libs" reports. > > Performance are now correct :) I checked with valgrind / callgrind, > and function mysql_data_seek is not called anymore. Do you have any numbers? Would be nice to know what speed gain this (almost) one-line optimization effected. > > You can now patch other drivers, and maybe release a new version ? > Thank you ! > I think I'll be able to finish migrating the drivers tonight as this requires changes in just one line per driver. New version is a different issue as I have to update the documentation and make sure everything is ready for prime time. It would be great if we could find a couple of kind souls who test the current CVS revisions on as many platforms as possible to find showstoppers, if any. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi+mysql speed issues
Markus Hoenicka writes: > Hi, > > I'll have to read the code again a little more thoroughly, but to the > best of my knowledge libdbi emulates MySQL's approach to retrieving > rows from result sets. In order to walk through the rows of e.g. a > PostgreSQL result set you have to retrieve the rows by index > sequentially, so libdbi has to maintain an internal pointer anyway. We > do not have to add one, so there are no extra changes. Also, libdbi > internally already mixes the cursor style and next_row style calls, > because we have to cater for database engines which use either of > these methods without exposing these differences to the libdbi user. > > As for the API change, we have extensive driver API changes between > 0.8.x and the upcoming(TM) 0.9 release anyway, think of the recent > addition of the transaction stuff. You won't be able to keep your > 0.8.x drivers once you switch to libdbi 0.9. You'll probably notice > problems only if you build from cvs regularly (and only if you update > one but not the other), but I expect those users to know what they're > doing. > Hi, I've implemented the suggested changes and checked in the updated files. If you want to speed test the code, please check out, build, and install the current cvs revisions of both libdbi and libdbi-drivers. I've updated the mysql, pgsql, and sqlite3 drivers at this time, but making the remaining drivers compile is trivial. I've added the suggested check to speed up sequential row fetching in mysql. I do see a small but significant decrease in the time required to run gmake check (11.78 vs. 12.51 s), although the test code does not test retrieving boatloads of rows specifically. I'd appreciate if someone with a nice testcase (Olivier?) could give the changes a try and report some numbers. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi+mysql speed issues
Mike Rylander was heard to say: > Markus, > > Would it be worth the ugliness of a module-global variable to bury the > value of the row index used in the previous call to > dbd_mysql.c::dbd_goto_row() > inside the MySQL driver itself, side-stepping the function signature change > for other drivers? The benefit of avoiding the API change may not be > outweighed by the potential fragility of naively tracking the state > internally, of course, since in practice folks install and upgrade to new > versions of drivers and the libdbi core at the same time, but then again, > it may. I'm a little nervous about the potential for problems when mixing > direct goto_row() (cursor style) and next_row() calls, but I haven't looked > at the code to see if there's actually an issue there... > Hi, I'll have to read the code again a little more thoroughly, but to the best of my knowledge libdbi emulates MySQL's approach to retrieving rows from result sets. In order to walk through the rows of e.g. a PostgreSQL result set you have to retrieve the rows by index sequentially, so libdbi has to maintain an internal pointer anyway. We do not have to add one, so there are no extra changes. Also, libdbi internally already mixes the cursor style and next_row style calls, because we have to cater for database engines which use either of these methods without exposing these differences to the libdbi user. As for the API change, we have extensive driver API changes between 0.8.x and the upcoming(TM) 0.9 release anyway, think of the recent addition of the transaction stuff. You won't be able to keep your 0.8.x drivers once you switch to libdbi 0.9. You'll probably notice problems only if you build from cvs regularly (and only if you update one but not the other), but I expect those users to know what they're doing. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS and more. Get SQL Server skills now (including 2012) with LearnDevNow - 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only - learn more at: http://p.sf.net/sfu/learnmore_122512 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi+mysql speed issues
Olivier Doucet was heard to say: > Hi Markus, > > 2013/1/8 Markus Hoenicka : >> We could modify the driver function >> dbd_goto_row() by passing both the wanted row index rowidx and the >> current row index currowidx(which libdbi keeps track of anyway). > > This is one way to fix the problem, I agree. Unfortunately my level in > C is too low to make such huge changes without breaking everything > else :) Anyone willing to create the patch for this ? > I'll commit these changes asap (may take a day or two). Can you build libdbi and libdbi-drivers from the cvs sources? You seem to have a good test case to see if these changes help. > Is there a way, outside libdbi, to fix this problem ? For example, go > over dbd_goto_row() and call fetch_row directly() ? Or maybe the > behaviour is different between database engines ? > I'm afraid you can't do that except if you bypass the abstraction layer altogether and use libmysqlclient natively. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS and more. Get SQL Server skills now (including 2012) with LearnDevNow - 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only - learn more at: http://p.sf.net/sfu/learnmore_122512 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi+mysql speed issues
Olivier Doucet was heard to say: > Hello everyone, > > I'm following a quite old topic about libdbi speed issues. > I was able to track the cause of these issues : The major problem is > how libdbi goes from one row to another. > > RRDTool (the tool that used libdbi and that I was inspecting) is using > dbi_result_next_row() function (as stated in libdbi documentation > btw). > > This function moves from one row to another with function > dbi_result_seek_row(), incrementing currentRow index each time. This > gives a call to dbd_mysql.c::dbd_goto_row() that uses > mysql_data_seek() each time... > > That's why for a query result of 34k rows (yes it happens. No it is > not a problem in the query itself), we have tens of thousands of call > to this function (which is very low), and this is definitely not > needed, because as we use fetch_row(), we automatically move from one > row to another. Seeking is just a useless task (as internal driver > does not know where we are, and needs to start from row 0 and seek to > the given row - where we already were). > > I'm absolutely not a libdbi user, and I don't know what could be done > outside libdbi to not use dbi_result_next_row() and use directly > RESULT->onn->driver->functions->fetch_row() directly. Is it possible ? > > And/or patching dbi_result.c : > just check RESULT->currowidx near line 102 before calling doing > goto_row() function and call it only if we are not on the good row. Am > I right ? > Hi, your analysis is pretty much correct. If you look at the comments in dbd_mysql.c::dbd_goto_row(), the original author of the mysql driver was well aware of the limitations of his implementation. The reason is that other database APIs, e.g. PostgreSQL, allow to fetch rows from a result set by index, whereas the MySQL API assumes that you step through the rows sequentially. The original design of libdbi appears to somewhat favor PostgreSQL in this respect. Anyway, without having thought about the issue in too much detail, one possible solution comes to mind. We could modify the driver function dbd_goto_row() by passing both the wanted row index rowidx and the current row index currowidx(which libdbi keeps track of anyway). This would allow drivers to decide whether they have to actually seek the position. pgsql doesn't have to anyway, and mysql doesn't have to if rowidx = currowidx+1. This API change would not mandate changes to existing drivers as they may ignore the additional parameter and keep working as before, but it may offer options to speed up queries in some drivers. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS and more. Get SQL Server skills now (including 2012) with LearnDevNow - 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only - learn more at: http://p.sf.net/sfu/learnmore_122512 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] Can't connect to MS SQLServer,
Daniel Hilst writes: > I'm trying to setup collectd to monitor a MS SQLServer, collectd uses > libdbi, witch uses freetds driver.. I can connectd with tsql, but > test_dbi can't.. Here is its output: > > http://pastebin.com/6kfzfPjw > Hi, I'm not familiar with the freetds driver, and I do not have a SQLServer or Sybase installation to fiddle with. Can you check the server logs to find out why the driver was unable to connect? > > Also I have a question, what should I put on version?. I tried 7.0 which > is the tds version used with freetds, but without successs. > The driver source suggests that the version should use the format X.XX.XX. If you're using version 7.x, only the first character is checked though, so "7.0" should have worked here as well. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
markus.hoeni...@mhoenicka.de writes: > I've stolen some time from myself to provide a first shot at > transaction and savepoint support, see the current cvs revisions of > libdbi and libdbi-drivers. The code is entirely untested except that > the drivers which I use myself compile and don't crash upon loading. I > didn't get round to adding the documentation and the tests, but feel > free to test the current code yourself. Usage should be pretty obvious > if you look at the diffs. I'm sure some rough edges remain, but > then... it's a start. > FYI I've also updated the docs in cvs to reflect the latest changes. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Rainer Gerhards writes: > > This should be easy to retrofit. I'lls see if I find > > some time but feel free to beat me at it. > > I am quite busy myself at the moment, but I could try and see if I > could craft something along that path... > Hi all, I've stolen some time from myself to provide a first shot at transaction and savepoint support, see the current cvs revisions of libdbi and libdbi-drivers. The code is entirely untested except that the drivers which I use myself compile and don't crash upon loading. I didn't get round to adding the documentation and the tests, but feel free to test the current code yourself. Usage should be pretty obvious if you look at the diffs. I'm sure some rough edges remain, but then... it's a start. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Mike Rylander was heard to say: > I think I'd add three calls to the proposed API, though, for support > of savepoints. They're part of the SQL standard, and supported by > several SQL RDBMS' including Postgres (the db my project uses, and we > use libdbi to connect to PG). The relevant PG documentation is > available at http://www.postgresql.org/docs/9.1/static/sql-savepoint.html > . I have wrappers for savepoints in my implementation, and they are > very handy for complex DB interactions. > Sounds like a no-brainer if it is as simple as you say. I don't keep the SQL standard underneath my pillow, but is it safe to assume that database engines are responsible to deal with any pending savepoints if a transaction is committed (i.e. no extra work for libdbi)? This brings up another question. Applications should of course check the "transaction_supported" and "savepoints_supported" driver capabilities and act responsibly. But if a database engine does not support savepoints, or transactions altogether, should libdbi just go ahead when asked, or should it throw an error instead? I recall that MySQL "supported" transactions in MyISAM tables using no-ops. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Rainer Gerhards was heard to say: > That's what I would expect. However, I think it would be useful to > have a call like "drvrTXSupport()" returning 1 if the driver supports > transaction and 0 otherwise. That way, an app could either fall back > to non-transcation mode OR tell the user that the database system is > not suitable for that kind of application. That would probably also > provide a smooth migration path, were only the primary TX interface > needs to be implemented and drivers can be upgraded as time permits > (assuming that some drivers are maintained by external entities). In fact, we already have a (not well documented) infrastructure for things like this in place. Every driver keeps a list of boolean "capabilities" which is merely a key-value list of whatever we think is useful or necessary. To date, only the "safe_dlclose" capability seems to be used, see e.g. dbd_initialize in libdbi-drivers/drivers/mysql/dbd_mysql.c. There is an internal function _dbd_register_driver_cap() which can be used by the drivers to announce a capability. And there is a public function dbi_driver_cap_get() which allows applications to query the list of capabilities at runtime. So all we'd have to do is modify the drivers and have them add a "transaction_support" capability set to 1 for those who support transactions. > > For TX support, I'd expect calls for > > - begin transaction > - commit > - rollback > > that's it, so all in all 4 calls. Would you agree to that? yes, except that we don't need the drvrTXSupport() function as we can use dbi_driver_cap_get() instead. > >> This should be easy to retrofit. I'lls see if I find >> some time but feel free to beat me at it. > > I am quite busy myself at the moment, but I could try and see if I > could craft something along that path... > > Rainer regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
Rainer Gerhards was heard to say: > Hi all, > > sorry if this is an exceptionally dumb question, but: does libdbi > support transactions (and, if so, via which functions)? I tried to > find the transaction support functions, but did not come up with > anything. Also a (granted, quick) look at the source did not tell me > anything. > > Any help is deeply appreciated, > Rainer > Hi, not exceptionally dumb, but rather touching a sensitive issue. I've used simple wrappers in one of my applications that could easily be moved into libdbi. I think a transaction interface has been missing from libdbi because some database engines that were supported in the days of yore did not have transaction support. I think most database engines which are supported or have experimental support these days support transactions by sending queries along the lines of "begin" or "begin work" etc. This should be easy to retrofit. I'lls see if I find some time but feel free to beat me at it. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Memory leak when using dbi_initialize in loop
Shakthi Kannan writes: > Which is the current cvs revision, and which file is it? I am not able > to see any commit logs at: > > http://sourceforge.net/mailarchive/forum.php?forum=libdbi-cvs > Hi, I'm afraid that the cvs log at sourceforge has been broken for a while. I always get back error messages related to the mailing lists when I check in changes. In any case, you can review the changes through the web cvs frontend: http://libdbi.cvs.sourceforge.net/viewvc/libdbi/libdbi/src/ The leak was fixed in dbd_helper.c which is now at 1.44. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Memory leak when using dbi_initialize in loop
Shakthi Kannan writes: > The API has been changed in the latest source. So, I updated the > following code snippet, and I still see the memory leak: Hi Shakthi, I've reproduced the leak that you detected in the current cvs sources. According to my analysis the drivers pass error messages through dbd_geterror() as allocated strings. _dbd_internal_error_handler() created another copy with strdup() which isn't necessary. The latter copy was freed, the original copy was left in place. I've fixed this leak in the current cvs revision. This is what I get now on my box when I run your test program: ==3717== HEAP SUMMARY: ==3717== in use at exit: 161,359 bytes in 5,251 blocks ==3717== total heap usage: 20,903 allocs, 15,652 frees, 16,739,701 bytes allocated ==3717== ==3717== LEAK SUMMARY: ==3717==definitely lost: 0 bytes in 0 blocks ==3717==indirectly lost: 0 bytes in 0 blocks ==3717== possibly lost: 0 bytes in 0 blocks ==3717==still reachable: 157,263 bytes in 5,250 blocks ==3717== suppressed: 4,096 bytes in 1 blocks ==3717== Reachable blocks (those to which a pointer was found) are not shown. ==3717== To see them, rerun with: --leak-check=full --show-reachable=yes ==3717== ==3717== For counts of detected and suppressed errors, rerun with: -v ==3717== Use --track-origins=yes to see where uninitialised values come from ==3717== ERROR SUMMARY: 289 errors from 1 contexts (suppressed: 48 from 8) regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Memory leak when using dbi_initialize in loop
Shakthi Kannan was heard to say: > I came across this memory leak post: > > http://comments.gmane.org/gmane.comp.db.libdbi.devel/309 > > Is this fixed in the sources and am I using old version of the software? > > libdbi-devel-0.8.3-4.fc14.i686 > libdbi-0.8.3-4.fc14.i686 > libdbi-drivers-0.8.3-6.fc14.i686 > libdbi-dbd-mysql-0.8.3-6.fc14.i686 > Hi, I had almost forgotten that bug. That's indeed the most likely reason that your test app leaks memory. Unfortunately you're not using an old version as 0.8.3 is the latest official release. The bug is fixed in the cvs version which, by all means, ist tested, stable, and reliable. I just won't have the time to finish a new release until September or so. So if that leak really hurts you, you'd have to build libdbi and libdbi-drivers from cvs. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA The must-attend event for mobile developers. Connect with experts. Get tools for creating Super Apps. See the latest technologies. Sessions, hands-on labs, demos & much more. Register early & save! http://p.sf.net/sfu/rim-blackberry-1 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Memory leak when using dbi_initialize in loop
Shakthi Kannan was heard to say:
> int main() {
> dbi_conn conn;
>
> while (1) {
> dbi_initialize(NULL);
>
[...]
>
> dbi_shutdown();
> }
>
> return 0;
> }
Hi,
I'll test your code on my dev box as soon as time permits. libdbi is
not supposed to lose memory under any circumstances. I'll have to see
the valgrind output as sometimes the leak is not within libdbi itself,
but within a driver or the client library (if the driver forgets to
free memory allocated by the library).
In any case, I'd suggest to modify your code. dbi_initialize() is
supposed to be run once per process, so the following is more
appropriate:
int main() {
dbi_conn conn;
dbi_initialize(NULL);
while (1) {
[...]
}
dbi_shutdown();
return 0;
}
If it is indeed libdbi that leaks memory, it won't do so in every
cycle of your loop.
regards,
Markus
--
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38
--
BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] libdbi+mysql speed issues
Adam Jacob Muller writes: > Hi, > I'm developing an application that utilizes rrdtool (and the lidbdi > interface into rrdtool) heavily and having some performance issues that seem > to center around libdbi, > Hi Adam, I can't see any obvious issues with your C code, so I assume the speed differences really stem from libdbi, one way or another. Unfortunately I currently lack the time for any deeper analysis, but I'm somewhat surprised about your results. libdbi has originally been developed as a wrapper for MySQL and PostgreSQL and should work fairly well with these database engines (other engines are a *lot* harder to fit). The only overhead that the mysql driver contains is the column type analysis and data conversion stuff. Some other engines have to buffer the data and thus create way more overhead. Did you have a chance to verify your results with any of the other database drivers? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] PATCH: sqlite3 parser bugs
Vikram Ambrose writes: > And here is an update to the patch for a third bug. Better late than never ... I've applied your patch, and things seem to work ok. Thanks for being this patient. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] sqlite3 multiple tables
[Vikram's original mail from 2010-12-25 got lost from my inbox, so I have to apologize for (1) being darn late and for (2) not keeping the thread intact] > 1. Artist (id,name) > 2. Album (id,VA,name) > 3. Album_Artist (artist_id,album_id) [...] > no tables in statement ! > no tables in statement ! How did you create the tables? Did you specify types for each column? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] Hello
Vikram Ambrose writes: > I've been emailing the list for about a month now with my patches and I > have got no responses. Can someone please confirm that these messages > are going through? Hi, they do, and I sincerely apologize that I haven't been able to take care of your messages and patches yet. My dayjob currently eats all available time (lest I stop sleeping entirely). I can't make any promises, but I'll do my very best to return to my libdbi duties asap. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dynamic loading
Rainer Gerhards was heard to say: > I don't remember if you use libtool. If not, Samuel may consider modifying > libdbi to utilize libtool, which what should not be too hard. As of my > understanding (never actually tried it), libtool provides full flexibility in > deciding if dynamic or static linking is used. > Dynamic vs. static linking is a different issue. At this time, libdbi is not linked against the drivers at all, for the reasons mentioned in my previous mail. Instead it uses dlopen() to load the drivers at runtime. libtool is involved here only on platforms which do not provide their own dlopen() implementation. However, building libdbi as a dynamic library uses libtool anyway, and it should be possible to statically link a project against libdbi if desired without further modifications. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dynamic loading
Quoting "DeLuca, Samuel Louis" : > Hello, > > Is there any way of compiling libdbi without a dynamic loader? If > not, is there any reason why it would be a bad idea for me to write > a patch to make it possible to dynamically link in drivers instead > of using dynamic loading? Hi, at this time there is no such option. The benefit of dlopen()ing the drivers instead of linking them in is that you don't have to recompile libdbi in order to install drivers. This makes it convenient for package or port systems to ship individual drivers separately from the library. I can't judge though whether your idea is technically feasible. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] sqlite and binary
Vikram Ambrose writes: > I can't seem to get BLOBs to work with SQLite. > > Here is some test code. > http://en.pastebin.ca/2026862 > > Output on my machine is; > > dbi_conn_connect: rv = 0: > data = ERROR > > What am I doing wrong? Probably nothing, at least not in your own code. Your test code prints the following on my box: $ ./sqlite_binary_test dbi_conn_connect: rv = 0: dbi_conn_query: data = close your eyes, this is binary All I changed is the driver. I've used sqlite3 as I currently don't have sqlite installed. However, as far as I can see there is no difference between those drivers in terms of blob handling. Could you test sqlite3 as well just to see whether it is a problem of your box or indeed a problem of the driver. Did you run make check in libdbi-drivers after building and installing the drivers? The test program includes checks for blob handling, so if there is a problem in the driver it should report errors. Runs fine here. Also, could you send a few specs of your OS and libdbi/libdbi-drivers versions? regards, Markus P.S. your code contains a strlen() call on binary data. I assume you are aware that this wouldn't work with arbitrary binary data. -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Learn how Oracle Real Application Clusters (RAC) One Node allows customers to consolidate database storage, standardize their database environment, and, should the need arise, upgrade to a full multi-node Oracle RAC database without downtime or disruption http://p.sf.net/sfu/oracle-sfdevnl ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] maintenance release 0.8.4 available
Hi all, after a longer hiatus there is a new libdbi release available for download. This is merely a maintenance release which was requested by the Debian and Ubuntu packagers (see Bug#592447 in the Debian bug tracker). There are no end-user-visible changes compared to the previous release, so if you have 0.8.3 installed there are no benefits of installing the new version. The following changes were implemented: - bumped the package version number to 0.8.4 - updated some autotools-related things in configure.in, autogen.sh, and Makefile.am (recent versions need AC_CONFIG_MACRO_DIR set properly, otherwise I wouldn't be able to bootstrap the build on my FreeBSD development box) - LIB_CURRENT=1, LIB_REVISION=0, LIB_AGE=0 - backported the --disable-docs behaviour fix in configure from HEAD Bumping up LIB_CURRENT was necessary due to backwards-incompatible changes in constants. We had planned to handle the library versioning thing appropriately starting with 0.9.0, but the Debian folks were eager to upgrade their (even more outdated) 0.8.2 packages to 0.8.3. This would have screwed up dozens of their packages unless the so name issue was fixed. Our greatly enhanced, all new and shiny 0.9.0 will be shipped real soon now (TM). However, the Debian folks have suggested a bunch of upstream patches which I'm inclined to include in the next release, so don't hang on the edges of your seats. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Timeout / Reconnect
Vikram Ambrose writes:
> There is indeed enough information to find this scenario. Here is a very
> simple callback I use at the moment:
>
> void DB_error_callback(dbi_conn conn, void *udata){
> int rv;
> const char *errormsg = NULL;
>
> rv = dbi_conn_error(DB_conn,&errormsg);
> if(strstr(errormsg,"gone away")){
> rv = dbi_conn_connect(DB_conn);
> if(rv){
> printf("ERROR: DB_error_callback: Tried to reconnect -
> failed\n");
> }else {
> printf("ERROR: DB_error_callback: Reconnected\n");
> }
> }
> }
>
Hi,
I just found some time to fiddle with this. As it seems, your solution
is pretty much focused on MySQL. However, I assume that the server
going away is a problem of all database engines, except maybe the
embedded ones. Adding some code to the MySQL driver to fix your
original problem thus might be the wrong solution. Don't you
experience the same problems with other DB engines? In that case, we'd
have to fix libdbi, rather than a driver.
regards,
Markus
--
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38
--
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Timeout / Reconnect
Vikram Ambrose was heard to say: > Is there a way of having the mysql driver, reconnect and then complete > the query without erroring out, in the case of driver/server timeout? > This way I do not lose my query to a timeout or need to have some sort > of global command buffer that the callback would use to re-execute the > failed query. Hi, does the MySQL API provide enough information to tell from a failure of mysql_query that the connection has stalled? I've found a log entry in cvs which added a timeout option to the driver, so with that in place and a useful error message from libmysqlclient I'd say it's doable. We'd just have to loop over a limited number of retries, trying to reconnect each time before resending the query. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] New release
Phil Longstaff writes: > any idea on when the next release of libdbi will be? It's been a long > time since the last one and there are some good changes in svn. Hi, I wouldn't place any bets. I've been discussing things with João lately, he has put an enormous amount of effort into updating our test harness which is way better than the old one. We ran into problems with firebird which both of us were unable to solve. Our latest guess is that the test code triggers a documented firebird bug which we can't work around. If we drop firebird support for now, we could release a new version after doing some minor cleanup. Unfortunately I'm haunted by a couple of deadlines in the next couple of weeks, so it is very uncertain whether I'll be able to finalize a release. Needless to say, test reports using the current cvs version are always welcome. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] New API?
Toby Thain was heard to say: > I think the deprecation is progressive because all in all it is > better if users always specify their instance explicitly. Hidden > global data has proven to be a liability in many such instances > (strtok(), errno, ...) This is certainly the rationale behind the decision to deprecate the old versions. Also, most applications will only have to define an additional structure and pass an additional argument in three function calls. Sanity doesn't always come that cheap. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] New API?
Vikram Noel Ambrose writes: > I just built libdbi and libdbi-drivers from svn. When recompiling my > application, I got a warning saying, dbi_initialize, dbi_conn_new and > dbi_shutdown has been deprecated. > > I think the docs online are out of data: > http://libdbi.sourceforge.net/docs/programmers-guide/reference-core.html > > What are those functions being replaced with? The docs reflect the situation as of the latest official release. This should probably be spelled out somewhere to make things clear. If you fiddle with the current svn version, please build the programmers guide from the sources. The current svn version implements a new feature called "instances" which required us to change the API slightly. In a nutshell, a libdbi instance allows you to initialize libdbi, load and unload drivers, and shut down the library independently from other parts of your program which also use libdbi. The problem came up with some piece of software which was designed to load modules at runtime. Some of these modules used libdbi. In order to unload such a module properly, you'd have to shut down the library. But that would have made libdbi unavailable to the remaining modules. Therefore the new interface uses instance handles to manage the library. A single process can open as many instances as it needs, and initialize and shut them down individually. The new initializer requires a pointer to a dbi_inst structure: int dbi_initialize_r(const char *driverdir, dbi_inst *pInst); You'll need this handle later to create connections and to shut down the instance: dbi_conn dbi_conn_new_r(const char *name, dbi_inst Inst); void dbi_shutdown_r(dbi_inst Inst); The older versions (dbi_initialize, dbi_conn_new and dbi_shutdown) are now implemented on top of these functions, using a single static instance handle. HTH Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Vikram Noel Ambrose was heard to say: > By the way, the url for the source code repository is dead on the website. Thanks for the heads up. I may have missed a SourceForge announcement about restructuring their CVS servers. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Vikram Noel Ambrose writes: > Yes of course. VARCHAR should be treated as NULL terminated character > strings and not binary. As long as libdbi is consistent as to what it > thinks a "VARCHAR(256) COLLATE 'binary'" refers to, then I'll be happy. > I've checked in a fixed version of dbd_mysql.c. I've also pasted in the patch below. It should apply cleanly to previous versions as well, as that part of the code hasn't been touched for a while. Vikram, could you please check whether this fixes your problem? regards, Markus --- dbd_mysql.c 2009/10/13 21:56:18 1.101 +++ dbd_mysql.c 2010/01/19 23:35:08 1.102 @@ -21,7 +21,7 @@ * Copyright (C) 2001-2002, Mark Tobenkin * http://libdbi.sourceforge.net * - * $Id: dbd_mysql.c,v 1.101 2009/10/13 21:56:18 mhoenicka Exp $ + * $Id: dbd_mysql.c,v 1.102 2010/01/19 23:35:08 mhoenicka Exp $ */ #ifdef HAVE_CONFIG_H @@ -675,8 +675,6 @@ _attribs |= DBI_DATETIME_DATE | DBI_DATETIME_TIME; break; - case FIELD_TYPE_VAR_STRING: - case FIELD_TYPE_STRING: case FIELD_TYPE_TINY_BLOB: case FIELD_TYPE_MEDIUM_BLOB: case FIELD_TYPE_LONG_BLOB: @@ -688,6 +686,8 @@ _type = DBI_TYPE_BINARY; break; } + case FIELD_TYPE_VAR_STRING: + case FIELD_TYPE_STRING: #ifdef FIELD_TYPE_NEWDECIMAL case FIELD_TYPE_NEWDECIMAL: // Precision math DECIMAL or NUMERIC field (MySQL 5.0.3 and up) #endif -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Markus Hoenicka writes: > information. This check simply looks spurious to me: > > if(field->charsetnr == 63) > It probably isn't as wrong as I thought initially. MySQL gurus, please correct me if I'm wrong, but I take the following from the manual: The purpose of the field->charsetnr==63 is to tell TEXT and BLOB fields apart (and their relatives of other sizes). Both are represented by a FIELD_TYPE_BLOB (or MYSQL_TYPE_BLOB) type. The former is a NULL-terminated string with a character set and a collation, the latter is a binary object without character set or collation. There is apparently no way to tell these apart other than by their field->charsetnr value. Therefore I tend to think the proper fix is to rearrange the case statements in the mysql driver (see the _translate_mysql_type function). The VAR_STRING and STRING types should be moved below the default: entry to never mark those as binary. Vikram, do I understand correctly that you don't expect to be able to store binary strings in a "VARCHAR(256) COLLATE 'binary'" column? If we fix the driver as I suggested above, you'd get back only the part of the string up to the first NULL. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Toby Thain was heard to say: > I have discussed this issue with Vikram on #mysql and IMHO it is > wrong in concept for a collation change to affect column type: As it > is not concerning the contents of the field but merely > interpretation. Naturally he didn't expect to have to change his code. > > But then perhaps the driver writer held a different opinion. I totally agree here, collation should not affect column type. I still suspect that MySQL reports the column type correctly, but that the driver uses incorrect or insufficient means to interpret this information. This check simply looks spurious to me: if(field->charsetnr == 63) This looks more to me like a crutch than like an author having a different opinion. Toby, isn't there a more foolproof check available in the MySQL C API? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Toby Thain was heard to say: > This is probably driver dependent. IMHO the first check would be to > see if the unexpected reversion to binary type is occurring within > libdbi or in libmysqlclient (or mysqld). > I bet it is. The mysql driver uses some weird check (in _translate_mysql_type) to find out whether or not a field is binary. Unless I'm dense the code in question may actually cause other problems as well, so I'd like to ask everyone with some MySQL knowledge to have a look at it. Also, the existing check may be version dependent. Vikram, which MySQL version are you running? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Vikram Noel Ambrose writes: > Either mysql is telling libdbi that the field is binary or libdbi is > getting confused somewhere. > Could you please insert a call to dbi_result_get_field_type_idx() to see what mysql thinks this is? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Vikram Noel Ambrose writes: > When I invoke dbi_result_get_string_idx on a column declared > "VARCHAR(256) COLLATE 'binary'", libdbi complains that: "The requested > variable type does not match what libdbi thinks it should be". > > I then tried using _get_binary_idx, but that seems to cause the next dbi > call to segfault (_get_short_idx). I wasn't sure if dbi was giving me > the '\0' character, so I called _get_field_length_idx and that just > returns 0. I don't have any idea whats going on now. > > Has anyone else run into this? I've never tried things like this. Which driver do you use? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_ulonglong overflow
Devin Reade writes: > I'm going to have to take a step back for a moment: Currently the only > thing that I have that is using libdbi is one legacy application. I'm > not saying that I won't use it in future projects, just that I have no > active projects at the moment depending on it. Therefore other users > may have more of an opinion. (I originally stepped into this conversation > to answer the "what's wrong with ato*?" question on a more theoretical > basis.) > I'm sorry about that, I've left you in the To: field unintentionally - I wanted to solicit some opinions from the list instead. But thanks anyway for providing your thoughts on this matter. regards, Markus > Also, your userbase may be wary of anything that results > in an API change, at least between minor versions. Even if you can't > propagate an error, you should be able to log the problem, though. > > That being said, I do not think that it is unreasonable for dbd_fetch_row > to error out if there is a problem parsing any of the result columns. > One should be able to determine (from log files, if not programatically) > the set of all problems in that case, though. > > Devin > -- > I'm not a complete idiot, some parts are missing. > > > -- > This SF.Net email is sponsored by the Verizon Developer Community > Take advantage of Verizon's best-in-class app development support > A streamlined, 14 day to market process makes app distribution fast and easy > Join now and get one step closer to millions of Verizon customers > http://p.sf.net/sfu/verizon-dev2dev > _______ > libdbi-users mailing list > libdbi-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/libdbi-users -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_ulonglong overflow
Devin Reade writes: > In the particular case of this use in libdbi, I've not looked at the > code to see to what extent it is an *actual* problem; however as a > generalization, I would advise against using ato*() functions and instead > use strto*() functions because the latter provides input error checks > that the former does not. I've had a look at libdbi-driver's current ato*() usage. As far as I can see it is used only to turn raw row data into integers by functions which do not return values, i.e. without an easy way to indicate conversion errors. I haven't checked all drivers yet, but I see two options: 1) make the functions in question return error codes. This would allow dbd_fetch_row() to return an error code as well. However, an error in a single field would make the operation fail. 2) use a field flag to tag the affected field only. This would require users to check integer fields manually, in a similar fashion as you have to check for NULL values if this is relevant. What do you think? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_ulonglong overflow
Devin Reade writes: > They should at least be as portable as the ato* functions, unless > perhaps someone is still using a K&R C compiler somewhere, which > I doubt. If there's any portability issue, I would expect it to > rear its head more for the 'long long' versions of both rather than > ato* vs strto*. > I've checked the repository. Obviously atoll() was missing from MinGW (for the more or less native Windows port of libdbi). Does anyone happen to know if MinGW still lacks atoll()/strtoll() support these days? In any case, we stole an atoll() implementation from MySQL back then, which upon close inspection also turns out to be implemented via its own strtoll(). Therefore switching from ato*() to strto*() should not hurt. > That being said, the only real test is empirical. > True indeed. I'll look into this. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_ulonglong overflow
Devin Reade writes: > The ato* are indeed often implemented in terms of strto*. The problem > is that the ato* signature just doesn't lend itself to error > checks, so even if they wrap strto*, error information gets lost in > the wrapper layer. What about the portability of the strto* functions? I recall we had to add an implementation of atoll() to the libdbi sources to compile it on at least one platform (I can't quite recall which one). Can we expect that all platforms which are currently supported provide all required strto*() functions? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_ulonglong overflow
Devin Reade was heard to say: > Markus Hoenicka wrote: > >> To better deal with your shock: Could you please elaborate what's >> wrong with atoll()? > > In the particular case of this use in libdbi, I've not looked at the > code to see to what extent it is an *actual* problem; however as a > generalization, I would advise against using ato*() functions and instead > use strto*() functions because the latter provides input error checks > that the former does not. For example, if someone provides "073Ah" as > input (thinking of the old DOS style hex stuff), what do they get? > > atol: 0x49 with no indication that the whole string wasn't parsed > strtol: 0x3B, with an indication that the "Ah" wasn't parsed (assuming > a zero base) or 0x73a with an indication that 'h' wasn't parsed > (assuming a base of 16). > > In code inspections, ato*() is something my eye is drawn to as a > potential source of problems, just as gotos are. I don't remember if > ato*() were mentioned in the classic paper > "Can't Happen or /*NOT REACHED*/ or Real Programs Drop Core" > but if not they should have been. > Thanks for elaborating on this. I guess I have incorrectly generalized from FreeBSD that ato*() are wrappers around the corresponding strto*() functions. If they are indeed separately implemented in other OSes, you have a good point in avoiding ato*() functions in libdbi. A quick look at the code suggests indeed that there is insufficient error checking on the return of the string to number conversions. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_ulonglong overflow
Quoting Vikram Noel Ambrose : > I was pretty shocked to see an atoll call in libdbi. But there is no > point on going on about that. > To better deal with your shock: Could you please elaborate what's wrong with atoll()? I recall from my home platform (FreeBSD) that atoll() is merely a wrapper around strtoll() anyways. So do you mind the use of atoll() per se or the lack of a potentially necessary discrimination between strtoll() and strtoull()? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Error -7 When retrieving NULL flag
Vikram Noel Ambrose uttered: > Example query: > > SELECT id,ISNULL(data) FROM > or > SELECT id,data IS NULL FROM > or > SELECT id,CASE WHEN data IS NULL THEN 0 ELSE 1 END AS is_data_null FROM > > dbi_result_get_int_idx(result,2); > > Gives me an error: > -7: The requested variable type does not match what libdbi thinks it > should be > > I'm pretty sure the last two queries are ANSI SQL. > > Any ideas how I can retrieve the result of those queries? I think this is not a problem of the queries themselves. If they failed, you'd get a different error. It is just a matter of how the database engine returns your values. The libdbi error indicates that it obviously doesn't use INT as return type. There are a couple of options to deal with this: 1) use dbi_result_get_field_type() or dbi_result_get_field_type_idx() to check the return type before retrieving the value. Different database engines may use different types. 2) use explicit return types like text as they might cause less problems, something along the lines of: SELECT id,CASE WHEN data IS NULL THEN 'null' ELSE 'notnull' END AS is_data_null FROM... 3) use libdbi metadata functions to check for NULL values instead of using SQL. See dbi_result_field_is_null() and dbi_result_field_is_null_idx() 4) the upcoming libdbi release will provide XXX_as_string() and XXX_as_longlong() functions which attempt to cast the return values regardless of their original types. These might serve as last resorts as well as the casting is done in a driver-specific manner. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] out of memory
Vikram Noel Ambrose writes: > The malloc inside _binary_copy_idx was failing. And I have no idea why. > But a few system restarts and libdbi installations later, its started > working. Must have had one too many segfaults that day. Poor computer. > > Sorry for the false alarm. Thanks for the all-clear. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] out of memory
Hi Vik, Vikram Noel Ambrose was heard to say: > Ubuntu-9.10 x86_64, lidbi and mysql driver from the distro repos (0.8.3.1). > > I can insert a 40kb MEDIUMBLOB into a table with no problem, but when I, > dbi_conn_queryf, dbi_result_first_row and then > dbi_result_get_binary_copy_idx, I get an out of memory error. > > The blob seems to be consistent from an md5sum I took through the > mysqlclient. Could you provide a stripped-down testcase to reproduce this problem? I'd like to see if it fails on 32bit systems as well. > > I tried to debug the problem, by downloading the libdbi/-driver sources > and peppering it a little. But I cannot run the compiled code on my > machine. I get a general protection fault from libdbi.so.0.1.0 when I > try to run my home built dbi. > Did you check the distro repo for any patches that they applied? They might have fixed 64bit issues, if any. If they build from stock sources, at least an unmodified homebuilt library should work ok. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] UNIX Timestamp - DATE to UTC conversion
Quoting Ken Ramsay <[EMAIL PROTECTED]>: > > Thanks guys, > > MySQL defines the field as a DATETIME , so the return type is time_t. As > far as I could find out, there is no helper funtion to convert this into > secs from 1 Jan 1970 , so I got around it by using the difftime() call > and subtracting the returned value from 1/1/1970. difftime() returns the > difference in seconds. > > Not sure it is the most elegant way to do it but it works ...;-} > Ah, I see. libdbi provides a dbi_result_get_datetime() function to retrieve these time_t values. Unless you have to do it in SQL, you could then post-process this value using gmtime() or any other suitable Unix time function. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] UNIX Timestamp - DATE to UTC conversion
Ken Ramsay writes: > time = dbi_result_get_uint(result, > "UNIX_TIMESTAMP(HIST_Timestamp)"); Without actually trying the code: one rough guess is that MySQL does not return the timestamp as an uint value but as something longer. Did you try using dbi_result_get_field_type() and dbi_result_get_field_attrib() to find out which type is returned? regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] sqlite and sqlite3 drivers CHAR as TEXT or INTEGER
Quoting João Henrique Freitas <[EMAIL PROTECTED]>: > Ok > > I can't do a atoi() because is out of my control. But in change the > struct of table to include a CHAR(1) than CHAR. And I can get CHAR(1) > like string. > I can't claim that this was an ingenious design decision back then as it is too long ago to remember for sure. But in retrospect I feel what you describe here offers users a choice of how to treat a single char. Either as a tiny integer (CHAR) or as a one-byte text [CHAR(1)]. It is probably best to keep this behaviour. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] Using field_type and field_attribs
Hi,
I'm a bit confused about your question as you seem to provide the
correct answer along with your question?
regards,
Markus
João Henrique Freitas writes:
> Hello,
>
> This is a short question.
>
> What are the methods to differentiate a char or int from a query?
>
> Like: SELECT the_int, the_char .
>
> I can use:
>
> column_number = 2;
>
> dbitype = dbi_result_get_field_type_idx(result,column_number);
> dbitype_attrb = dbi_result_get_field_attribs_idx(result,column_number);
>
> switch (dbitype) {
> case DBI_TYPE_INTEGER: < int and char are integer
> if(dbitype_attrb == DBI_INTEGER_SIZE1) {
>// I have char
>break;
> } else if(dbitype_attrb == DBI_INTEGER_SIZE2) {
>// I have short
> } else if(dbitype_attrb == DBI_INTEGER_SIZE4) {
>// I have int
> } else if(dbitype_attrb == DBI_INTEGER_SIZE8) {
>// I have long long
> } else {
>// Nothing
> }
>
>
>
> Thanks
>
>
> --
> ---
> João Henrique Freitas - joaohf_at_gmail.com
> Campinas-SP-Brasil
> BSD051283
> LPI 1
> http://www.joaohfreitas.eti.br
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ___
> libdbi-users mailing list
> libdbi-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/libdbi-users
--
Markus Hoenicka
[EMAIL PROTECTED]
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] newline in query causes all field types to appear as DBI_TYPE_STRING
Quoting Toby Thain <[EMAIL PROTECTED]>: > > On 13-Oct-08, at 7:15 AM, Markus Hoenicka wrote: > >> Hi, >> >> just to clarify: it is sqlite, not the sqlite driver which does not >> support newlines in SQL statements. > > Wow, that's inconvenient. I imagine that must break very many ORM/SQL > generating layers. > I googled a bit on this issue, but apparently no one complained about this restriction yet. The mailing list does not seem to have any comments on this. I'd suggest the OP picks up his entry in SQLite's CVSTrac again and adds the simple test case that I've provided. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] newline in query causes all field types to appear as DBI_TYPE_STRING
Hi, just to clarify: it is sqlite, not the sqlite driver which does not support newlines in SQL statements. Some other SQL engines like MySQL do support newlines, so this appears to be a design decision (I don't know what the standard says). To verify, use these SQL scripts: - select version() - and - select sqlite_version() - write the scripts to a file using Unix line endings, and pipe them into mysql and sqlite, respectively. The former succeeds, whereas the latter complains about incomplete SQL. I think it is easier to avoid newlines on the input side rather than trying to "fix" sqlite's behaviour in the driver. regards, Markus Quoting Kris Groves <[EMAIL PROTECTED]>: > The parsing of the sql statement will fail at the first new line char as > this indicates the end of the c-string. > > This is the first time I have seen newline characters explicitly > embedded in an sql statement. Is there a reason that the newline > characters are needed ? -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi 1.0, plan to launch
Quoting João Henrique Freitas <[EMAIL PROTECTED]>: > Hello, > > Have we plan to launch libdbi 1.0 release? > Not yet. I've found a serious issue with an application using libdbi when running on Windows/Cygwin. Using the latest code of both the app and libdbi, the app crashes when it forks. I was not able to debug this problem as the crash happens inside the fork() call (which is implemented in real strange ways in Cygwin due to the lack of a native fork in Windows). A simple ten-liner using libdbi and fork() does not crash though. I'm reluctant to release libdbi before I can make sure this problem is not related to libdbi instances in any way. Unfortunately I'm currently not able to work on this issue as I'm being absorbed by my dayjob and a relocation next week. In order to schedule a release later this year (late October, November?) I'd like to know if anyone else would like to see additional features in 1.0. What about your rewritten tests? regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi, driver sqlite3 and 64 bits
Quoting João Henrique Freitas <[EMAIL PROTECTED]>: > If I compiling and install libdbi and libdbi-drivers in a Linux > 64bits, the library will go to /usr/local/lib. Is more confortable put > it on /usr/local/lib64? Because we has compiled and installed in a > 64bits plataform? > /usr/local/lib is the default directory chosen by the autotools, this is not specific to libdbi. I guess the autotools would use a different directory on 64bit OSes if there was a standard which says so. It is probably going to cause more grief than benefit to fiddle with this location. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi, driver sqlite3 and 64 bits
Quoting João Henrique Freitas <[EMAIL PROTECTED]>: > /usr/local/lib/dbd/libdbdsqlite3.so: undefined symbol: > _dbi_internal_error_handler Hi, this typo was fixed in revision 1.22 on March 4, 2008. If you don't want to work with the current cvs revisions, simply change "_dbi_internal_error_handler" to _dbd_internal_error_handler" in dbd_sqlite3.c, around line 733. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] static link to libdbi
Hi, thanks for your inquiry. For future posts please use an appropriate mailing list so others can benefit or chime in too: libdbi-users@lists.sourceforge.net [EMAIL PROTECTED] I've cc'ed the former, please address your replies to that list too. As for the linking problem, I suppose you have to link the drivers statically too. For some reason the driver (libmysql.so) cannot access the libdbi library functions if they are statically linked into your project. Try linking the drivers statically against libdbi and try again. regards, Markus Quoting A C <[EMAIL PROTECTED]>: > Hi, > >I try to link statically to libdbi (either link to libdbi.a or w/ > the linker flag "gcc -g -static -ldbi) but every time I run just a > simple app, I keep getting the following error: > /usr/lib64/dbd/libmysql.so: undefined symbol: > _dbd_result_set_numfields libdbi: Failed to load driver: > /usr/lib64/dbd/libmysql.so > >The same app works fine if linked dynamically w/ the linker flag > "gcc -ldl -ldbi. > >Would you please provide some instructions how to do it? Thanks > > Peter Long. > _ > See how Windows Mobile brings your life together—at home, work, or on the go. > http://clk.atdmt.com/MRT/go/msnnkwxp1020093182mrt/direct/01/ -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] cannot get int from second table in a query
Daniel Moreno writes: > Markus, > I'm new using libdbi so we didn't talk before so I want to say good job > to you and all the people that made it possible. > Why thank you. A fair share of all praise should always go to David Parker and Mark Tobenkin who started the project with an excellent design which still works as intended today. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] cannot get int from second table in a query
Quoting Daniel Moreno <[EMAIL PROTECTED]>: > Markus, > We could add queries like these to the tests or if is a known issue > should be documented that didn't work with sqlite3 driver. > I absolutely agree. I have always tried in the past to use bug reports as an excuse to add yet another test... Someone else on the list is currently rewriting the test program to make it more modular. This will allow us to add further tests without adding further complexity. I'll add the alias problem to the driver documentation unless we manage to fix the problem before the next release. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] cannot get int from second table in a query
Quoting Kris <[EMAIL PROTECTED]>: > I am currently trying to fix the find_result_field_types function so > that it can work with aliases as well as multiple table selects without > the table prefix (e.g. select * from one,two). I'll see if I can use > anything from daniels "dirty patch" on the cvs version. > > Kris. I've had a look at the driver code yesterday, and I'm confident that I can migrate the code to the prepare/step/finalize interface which is now considered the default in sqlite3. This allows us to use the sqlite3_column_type() function which is preferable over the guessing code in find_result_field_types(). The only downside is that sqlite3_column_type() won't distinguish between integer types, so we'll probably have to keep some of the weird logic currently in use. If you find a way to make the current code work with aliases and such, I'll most likely be able to reuse this code with the new interface as well. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] cannot get int from second table in a query
Kris Groves writes: > Hi List, > > I'm seeing strange behavior here : > > This is a sqlite/sqlite3 driver problem, not a generic libdbi problem. The sqlite driver has to guess the column types from the "CREATE TABLE" statement that was used to build the tables involved in a query. The reason is that sqlite is essentially typeless. It is obvious that aliases strain the column type guessing code. The situation has improved somewhat in sqlite3, now that there is limited support for data types. However, the internals of the sqlite3 driver do not yet make good use of the improved type support. I'll try to rewrite the sqlite3 driver to take advantage of the type support, but I haven't checked yet whether this is possible within the constraints of libdbi's current design. In any case it'll take some time to implement. For the time being you may have a look at the functions long long dbi_result_get_as_longlong(dbi_result Result, const char *fieldname); long long dbi_result_get_as_longlong_idx(dbi_result Result, unsigned int fieldidx); char *dbi_result_get_as_string_copy(dbi_result Result, const char *fieldname); char *dbi_result_get_as_string_copy_idx(dbi_result Result, unsigned int fieldidx); These functions are available in the latest cvs version. The first two functions return any value as a long long integer (which you can cast to any other integer type) using reasonable conversions if necessary and if possible at all. The last two functions return just about anything as strings. Obviously these are merely workarounds in your case, but they still may be helpful to get things done. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Multithreading question
Quoting Rainer Gerhards <[EMAIL PROTECTED]>: > You need to be very careful. Not just libdbi's own thread safety must be > considered, but the thread safety of the drivers. In short, IMHO it is > not possibly to correctly assume that a libdbi call, at least on the > same connection handle, is thread safe. For example, this will always > fail with MySQL. To the best of my knowledge libdbi does not use mutexes > itself to avoid threading pitfalls on the driver layer (and I think this > is a good design-decision, as most apps do not need these and they cost > performance). > > So your app must make sure that it calls into the drivers only in a > thread safe way. In short: you must mutex-protect these cases yourself. I have to admit that I do not have any experience in terms of thread-safe programming, but as always I'm trying to understand. Is the problem you mentioned MySQL specific? Is there anything we can (or should?) do about it in libdbi or in the driver? If I understand correctly, the best way to avoid problems is to use one connection handle per thread. Is this solution applicable to most programs, or does it have a performance penalty that most apps want to avoid? Maybe it's worthwhile to add a section about threading issues to the programmer's guide? Being a non-expert myself, I'd greatly appreciate contributions from those "in the know". regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] PTHREAD_CANCEL_ASYNCHRONOUS safe?
Hi, I've checked in the required changes to make timeout a generic option. Currently the mysql, pgsql, sqlite, and sqlite3 drivers support this option. I couldn't find a way to set the timeout on the client side for the firebird driver. I'd like to ask the maintainers of the remaining drivers to check whether they can support this option too. The timeout option specifies the timeout in seconds. The sqlite/sqlite3 drivers used to specify the timeout in milliseconds though. Both drivers now support the generic option which is in seconds, although sqlite_timeout/sqlite3_timeout override the generic option if both are set. Please be aware of this change. regards, Markus Ethan Funk writes: > That would be really great! I believe the SQLite3 driver currently > uses the "sqlite3_timeout" property: not very generic. To make this > generic, you would need to use something like "timeout" across all > the drivers and have the SQLite3 driver alias this to the old > property to maintain back compatibility. > -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] PTHREAD_CANCEL_ASYNCHRONOUS safe?
Ethan Funk writes: > That would be really great! I believe the SQLite3 driver currently > uses the "sqlite3_timeout" property: not very generic. To make this It does so on purpose, as there were no plans back then to make it a generic option. Older versions of SQLite had a very coarse locking mechanism which made timeouts likely as soon as several users access the same database. This has been improved since, but the option remained in place, using the naming scheme for non-generic options which are always prefixed with the driver name . > generic, you would need to use something like "timeout" across all > the drivers and have the SQLite3 driver alias this to the old > property to maintain back compatibility. > Yup. Should be available real soon now. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] PTHREAD_CANCEL_ASYNCHRONOUS safe?
Ethan Funk writes: > This may be a suggestion more for the libdbi_driver folks, but it > sure would be nice if I could set a Timeout property for connection > settings on the postgress and mysql drivers. The SQLite drivers have > this. > The libdbi_driver folks isn't far either :-) Both the pgsql and mysql drivers can be trivially retrofitted with a timeout option. I'll see to it tomorrow. I'll also check if the other engines support a timeout setting too, so we could use a general option to keep things simple. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] PTHREAD_CANCEL_ASYNCHRONOUS safe?
Hi, thanks for the excellent explanation. With that in mind, I'd say it depends. libdbi does not use mutexes. Also, the libdbi library does not allocate any memory *before* the database client query function returns. However, some client libraries require the allocation of memory before the query can performed (e.g. firebird), so these drivers will have allocated some memory in case you cancel a query. Everything else is beyond the control of libdbi/libdbi-drivers. If the query implementation of one of the supported client libraries uses mutexes or allocates memory which is not freed if you cancel a query, you'll experience the same problems as if you didn't use libdbi as an abstraction layer. It is probably advisable to ask the lists of the database engines you plan to support if their clients safely handle cancelling queries. regards, Markus Rainer Gerhards writes: > Well, the base requirement is that you need to make sure no resource is > left after you are cancelled. Let's say you lock a mutex (A), than do > some work (B), and later unlock the mutex (C). Then you get cancelled > during B). In that case the mutex will remain locked if you do not > explicitly unlock it. That, of course, will cause trouble when you come > back the next time - you'll then block on A) because the mutex is still > locked. > > Less severe problems occur if you malloc() in A and free() in C). If you > are cancelled in B, the result is a mem leak. > > To prevent these situations, you can either block cancellation before A > and enable it again after C or you can work with so-called cancel > cleanup handlers, which in essence do what you would do in C. > > The bottom line is that an app must be aware of cancellation. I'd > personally find this a bit dangerous, given the third party nature of > libdbi. IMHO the best option would be the caller handle the complexity. > At least this is what I am doing in rsyslog (but, granted, cancellation > happens very infrequently there, so I may be overlooking something > obvious). > > HTH, > Rainer -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] PTHREAD_CANCEL_ASYNCHRONOUS safe?
Hi, I have to admit that I'm not exactly familiar with this kind of stuff. Is there any documentation available about the requirements to call a code PTHREAD_CANCEL_ASYNCHRONOUS safe? regards, Markus Ethan Funk writes: > Does any one know if the dbi_conn_query function (and the underlying > drivers) are PTHREAD_CANCEL_ASYNCHRONOUS safe? > I would like to be able to cancel a query if it is taking too long. > > Ethan... > > - > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ > ___ > libdbi-users mailing list > libdbi-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/libdbi-users -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Use dynamic dbi_result_bind_* functions
João Henrique Freitas writes: > "Name" is type binary has a string like "xorg.conf". > > You see any problem if binary does not return a string? Yes, binary > data could be a image. It will only return a without sense string. > Well, I figured that if you expect a field to contain null-terminated strings you wouldn't use BLOB or BINARY in the first place. From a practical point of view, BLOB or BINARY fields are allowed to contain sequences of characters which are not zero-terminated. We could always add a NULL byte just in case, but I somehow feel you should rather use the correct field type if you expect to store strings. But then, this is just my opinion and I'm open for discussions. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Use dynamic dbi_result_bind_* functions
Markus Hoenicka writes: > João Henrique Freitas writes: > > How we will automatically convert the types into strings formats, > > inside dbi_result_fget_string*? > > > > Yes, this should happen within libdbi to make it independent of the > drivers. We should use standard C library conversion functions as far > as possible. > I've got a first implementation available for testing in CVS. There are only get_as* functions at this time as I'm still struggling with the bind* interface. The new functions can be admired by running "make check" in libdbi-drivers. That is, you'll need the CVS versions of both libdbi and libdbi-drivers. The conversions are currently done as follows: get_as_string_copy/get_as_string_copy_idx (returns allocated buffer) integers: rendered by sprintf float and double: rendered by sprintf strings: as is. NULL strings are returned as empty strings though. binary strings: returned as empty strings date and time: pretty-printed as -MM-DD HH:MM:SS. Date-only values will show an allballs time, and time-only values will claim 1970-01-01 (the start date of the Unix epoch) as date. get_as_longlong/get_as_longlong_idx (returns long long integer value) integers: typecast to long long float and double: typecast to long long. Remember that float and double values easily exceed the range of long long. Currently there is no such thing as an overflow error flag. strings: rendered by strtoll(). If the string contains a number as understood by that function, its value is returned. NULL strings, empty strings, and strings that cannot be parsed as a number return 0. binary strings: returned as 0 date and time: returned as number of seconds elapsed since the Unix epoch. Time-only values will be calculated as of 1970-01-01, and date-only values will indicate the number of seconds elapsed at midnight (the first second of the given date). All in all, these new functions are certainly no "one-size-fits-all" solution for retrieving field values without knowing the types of the fields, but they may be convenient in many cases. Please let me know if these defaults don't seem reasonable, or if we should attempt to report overflows, unconvertible values etc. with error flags. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Use dynamic dbi_result_bind_* functions
João Henrique Freitas writes: > How we will automatically convert the types into strings formats, > inside dbi_result_fget_string*? > Yes, this should happen within libdbi to make it independent of the drivers. We should use standard C library conversion functions as far as possible. > > Meanwhile, the dbi_result_get_as_longlong* return integers. What will > be the retur If this get a string? > I'd suggest to use strtoll() for the conversion. If the string happens to be the representation of an integer which makes sense to strtoll(), we'll return this value. If the string does not represent a number, strtoll() returns 0 and sets errno. We should return 0 as well and report the conversion failure by setting the libdbi error status appropriately which you can query by calling dbi_conn_error() if necessary. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Use dynamic dbi_result_bind_* functions
Markus Hoenicka writes: > char* dbi_result_fget_string_copy(dbi_result Result, const char* > fieldname); > char* dbi_result_fget_string_copy_idx(dbi_result Result, unsigned int > fieldidx); > int dbi_result_fbind_string_copy(dbi_result Result, const char* > fieldname, char** bindto); > Come to think of it, the following names may be more appropriate: dbi_result_get_as_string_copy() dbi_result_get_as_string_copy_idx() dbi_result_bind_as_string_copy() dbi_result_get_as_longlong() dbi_result_get_as_longlong_idx() dbi_result_bind_as_longlong() regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Use dynamic dbi_result_bind_* functions
João Henrique Freitas writes: > char *dbi_get_value(dbi_result *result, int row_number, unsigned int > column_number); > > int dbi_bind_value(dbi_result *result, int row_number, unsigned int > column_number, char *value); > I'd prefer to model these functions after the existing interface. The names are open for discussion (the f prefix means "force"), but in terms of the functionality I thought about something like: char* dbi_result_fget_string_copy(dbi_result Result, const char* fieldname); char* dbi_result_fget_string_copy_idx(dbi_result Result, unsigned int fieldidx); int dbi_result_fbind_string_copy(dbi_result Result, const char* fieldname, char** bindto); These resemble the dbi_result_get_string_copy() et al. functions. As all values which are not strings will have to be provided in an allocated buffer anyway, this seems the most logical approach to such a function. Obviously, the calling function has to free the returned buffer eventually. In addition, I'd suggest another group of functions that retrieves integers regardless of the storage type: long long dbi_result_fget_longlong(dbi_result Result, const char* filename); long long dbi_result_fget_longlong_idx(dbi_result Result, unsigned int fieldidx); int dbi_result_fbind_longlong(dbi_result Result, const char* fieldname, long long* bindto); These functions would convert and return all numerical values (integers, floats, datetime, even strings if they happen to contain a number) as long long integers which the calling function can cast to whatever it needs. Let me know if that would meet your needs. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] Use dynamic dbi_result_bind_* functions
João Henrique Freitas writes: > Hello, > > Following, I have a function my_dbi_get_value. I use dbi_result_get > and tranform the value received in a char *buf to return. > [...] > How I can use dbi_result_bind_* and not need to allocate a char *buf? > What is the advantage of using bind* instead of get* here? As a side note, your code is close to an alternative interface which we've discussed in the past. Instead of using strongly typed return values, the alternative interface would allow to retrieve data as a string or an integer, regardless of the storage type. For obvious reasons, the results would not always be meaningful, but at least you'd be able to retrieve a string representation of each and every value. Your request indicates that such an interface would be a useful addition. Would you be interested to integrate such an interface into libdbi? I'd be happy to help. regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] libdbi and bind functions, testing
Hi,
João Henrique Freitas writes:
> Hello,
>
> I need to work with some bind functions in libdbi. And I will test all
> and put on dbi_test.c.
This is greatly appreciated, as the test program so far does not cover
the bind* functions.
> while (dbi_result_next_row(result)) {
> error = dbi_result_bind_uint(result, "id", &idnumber);
>
> dbi_result_bind_string(result, "name", &fullname);
> printf("%i. %s\n", idnumber, fullname);
> }
I have to admit that I've never used the bind* functions myself. But I
recall from the programmers manual that you have to set up bindings
after a successful query, but before retrieving any rows. I don't know
whether this explains the error you mention, but you may want to try
to reverse the order of your code, just in case.
I'll be happy to review any problems once you have integrated the
tests into dbi_test.c. BTW as you seem to contribute more and more
code to libdbi, would you like to sign up as a developer? This way you
could check in your changes directly and make them available for other
developers.
regards,
Markus
--
Markus Hoenicka
[EMAIL PROTECTED]
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de
-
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
