Re: [libdbi-users] libdbi+mysql speed issues
Markus, Would it be worth the ugliness of a module-global variable to bury the value of the row index used in the previous call to dbd_mysql.c::dbd_goto_row() inside the MySQL driver itself, side-stepping the function signature change for other drivers? The benefit of avoiding the API change may not be outweighed by the potential fragility of naively tracking the state internally, of course, since in practice folks install and upgrade to new versions of drivers and the libdbi core at the same time, but then again, it may. I'm a little nervous about the potential for problems when mixing direct goto_row() (cursor style) and next_row() calls, but I haven't looked at the code to see if there's actually an issue there... --miker On Tue, Jan 8, 2013 at 4:46 AM, Markus Hoenicka < markus.hoeni...@mhoenicka.de> wrote: > Olivier Doucet was heard to say: > > > Hello everyone, > > > > I'm following a quite old topic about libdbi speed issues. > > I was able to track the cause of these issues : The major problem is > > how libdbi goes from one row to another. > > > > RRDTool (the tool that used libdbi and that I was inspecting) is using > > dbi_result_next_row() function (as stated in libdbi documentation > > btw). > > > > This function moves from one row to another with function > > dbi_result_seek_row(), incrementing currentRow index each time. This > > gives a call to dbd_mysql.c::dbd_goto_row() that uses > > mysql_data_seek() each time... > > > > That's why for a query result of 34k rows (yes it happens. No it is > > not a problem in the query itself), we have tens of thousands of call > > to this function (which is very low), and this is definitely not > > needed, because as we use fetch_row(), we automatically move from one > > row to another. Seeking is just a useless task (as internal driver > > does not know where we are, and needs to start from row 0 and seek to > > the given row - where we already were). > > > > I'm absolutely not a libdbi user, and I don't know what could be done > > outside libdbi to not use dbi_result_next_row() and use directly > > RESULT->onn->driver->functions->fetch_row() directly. Is it possible ? > > > > And/or patching dbi_result.c : > > just check RESULT->currowidx near line 102 before calling doing > > goto_row() function and call it only if we are not on the good row. Am > > I right ? > > > > Hi, > > your analysis is pretty much correct. If you look at the comments in > dbd_mysql.c::dbd_goto_row(), the original author of the mysql driver > was well aware of the limitations of his implementation. The reason is > that other database APIs, e.g. PostgreSQL, allow to fetch rows from a > result set by index, whereas the MySQL API assumes that you step > through the rows sequentially. The original design of libdbi appears > to somewhat favor PostgreSQL in this respect. > > Anyway, without having thought about the issue in too much detail, one > possible solution comes to mind. We could modify the driver function > dbd_goto_row() by passing both the wanted row index rowidx and the > current row index currowidx(which libdbi keeps track of anyway). This > would allow drivers to decide whether they have to actually seek the > position. pgsql doesn't have to anyway, and mysql doesn't have to if > rowidx = currowidx+1. This API change would not mandate changes to > existing drivers as they may ignore the additional parameter and keep > working as before, but it may offer options to speed up queries in > some drivers. > > regards, > Markus > > > > -- > Markus Hoenicka > http://www.mhoenicka.de > AQ score 38 > > > > > -- > Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS > and more. Get SQL Server skills now (including 2012) with LearnDevNow - > 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. > SALE $99.99 this month only - learn more at: > http://p.sf.net/sfu/learnmore_122512 > ___ > libdbi-users mailing list > libdbi-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/libdbi-users > -- Mike Rylander | Director of Research and Development | Equinox Software, Inc. / Your Library's Guide to Open Source | phone: 1-877-OPEN-ILS (673-6457) | email: mi...@esilibrary.com | web: http://www.esilibrary.com -- Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS and more. Get SQL Server skills now (including 2012) with LearnDevNow - 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only - learn more at: http://p.sf.net/sfu/learnmore_122512___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi transaction support
On Tue, Nov 20, 2012 at 7:59 AM, Rainer Gerhards wrote: > On Tue, Nov 20, 2012 at 1:02 PM, Markus Hoenicka > wrote: >> In fact, we already have a (not well documented) infrastructure for things >> like this in place. Every driver keeps a list of boolean "capabilities" >> which is merely a key-value list of whatever we think is useful or >> necessary. To date, only the "safe_dlclose" capability seems to be used, see >> e.g. dbd_initialize in libdbi-drivers/drivers/mysql/dbd_mysql.c. There is an >> internal function _dbd_register_driver_cap() which can be used by the >> drivers to announce a capability. And there is a public function >> dbi_driver_cap_get() which allows applications to query the list of >> capabilities at runtime. So all we'd have to do is modify the drivers and >> have them add a "transaction_support" capability set to 1 for those who >> support transactions. > > ok, that makes sense and is excellent to have! > >>> For TX support, I'd expect calls for >>> >>> - begin transaction >>> - commit >>> - rollback >>> >>> that's it, so all in all 4 calls. Would you agree to that? >> First, I'm glad to so movement on this! The code in the project I'm involved with handles transactions on its own already using something along the lines of the wrapper pattern Markus mentioned, so it won't make a big difference to me immediately, but having a sanctioned API is almost always better, IMO. I think I'd add three calls to the proposed API, though, for support of savepoints. They're part of the SQL standard, and supported by several SQL RDBMS' including Postgres (the db my project uses, and we use libdbi to connect to PG). The relevant PG documentation is available at http://www.postgresql.org/docs/9.1/static/sql-savepoint.html . I have wrappers for savepoints in my implementation, and they are very handy for complex DB interactions. I can't offer many properly shaped tuits toward the development effort right now, but supporting savepoints should amount to essentially adding cut-and-paste duplicates of begin, commit and rollback. Would it be objectionable to ask for savepoint support to piggy-back on transaction support? -- Mike Rylander | Director of Research and Development | Equinox Software, Inc. / Your Library's Guide to Open Source | phone: 1-877-OPEN-ILS (673-6457) | email: mi...@esilibrary.com | web: http://www.esilibrary.com -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] [Libdbi-drivers-devel] unresolved issues of envisaged next releases
On Feb 5, 2008 5:55 PM, Markus Hoenicka <[EMAIL PROTECTED]> wrote: > Hi all, > > I haven't seen much feedback on the latest development version of > libdbi and libdbi-drivers lately. I don't mean to complain - if we > think positively, no mail on this list must mean that everything is > running smoothly... > > However, I don't like to rush out releases that happen to work on the > box of a single developer. I'd like to solicit at least some feedback > about the development version, whether it works on other boxes too, > whether the dbi_conn_error_flag() vs. dbi_conn_error() change is the > right way to go, whether all drivers that anyone may want to use works > in its current state, and so on. We've tested the pre-release tarballs with Evergreen* and all is well. * http://open-ils.org/ > > Also, I haven't heard much of the requested escape-only vs. escape-and > quote functions? If no one complains, I'll implement these as a > libdbi-only patch which will leave both libdbi and libdbi-drivers at > 0.8.3. > As long as the existing quote API is unchanged, I think the addition would be good. Thanks Markus! -- Mike Rylander | VP, Research and Design | Equinox Software, Inc. / The Evergreen Experts | phone: 1-877-OPEN-ILS (673-6457) | email: [EMAIL PROTECTED] | web: http://www.esilibrary.com - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] libdbi-0.8.2, libdbi-drivers 0.8.2: ready for prime time?
On 2/19/07, Markus Hoenicka <[EMAIL PROTECTED]> wrote: > Hi all, > > as previously announced here I've checked in a couple of minor changes that > make > the CVS versions of both libdbi and libdbi-drivers ready for release. All I > need > to move on is some feedback from platforms that I can't test. I ran a few > tests > on FreeBSD and on Windows XP/Cygwin which showed no regressions. Could anyone > else please confirm that the current CVS versions work ok on major platforms > (Linux, OSX, Solaris, other *BSDs)? I'd like to get this release off my desk. I'm very happy to report that everything builds and works fine on Linux (as far as I've tested). Specs for the setup: Debian Etch w/ custom kernel (2.6.16 SMP) Postgres 8.1.[14] and 8.2 Evergreen CVS HEAD (our project) Obviously, this is no surprise since it's basically the code that we've been using all along. Let me know if there's any other detail you'd like. Everything is working well, thanks all! -miker > > regards, > Markus > > -- > Markus Hoenicka > [EMAIL PROTECTED] > (Spam-protected email: replace the quadrupeds with "mhoenicka") > http://www.mhoenicka.de > > > - > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > ___ > libdbi-users mailing list > libdbi-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/libdbi-users > -- Mike Rylander [EMAIL PROTECTED] GPLS -- PINES Development Database Developer http://open-ils.org - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] driver issues and timezones
On 2/14/07, Markus Hoenicka <[EMAIL PROTECTED]> wrote: > Mike Rylander <[EMAIL PROTECTED]> was heard to say: > > > > hmmm... The libtool interface version number is about compatibility, > > where as the release number is about human-understandable "freshness". > > These are orthogonal concepts, though the former does deserve > > attention. It seems to me that the LIB_CURRENT and LIB_AGE should > > just both be 0, since there is no previously versioned interface. > > Then LIB_CURRENT just gets bumped when new functions are added, and > > LIB_AGE gets bumped as old functions are changed, deprecated or > > removed. Trying to make the version number match the interface number > > seem both counter productive (a new interface version for each > > release?) and against libtool best practices, no? ... I may be > > misunderstanding the proposal, though. > > > > Maybe *I* am all dense about the libtool versioning stuff, but I see it like > this: LIB_AGE is for recording the backwards-compatibility and thus is not > relevant for the version number. All we have to consider is LIB_CURRENT (the > number of the interface) and LIB_REVISION (loosely, the number of released > source code changes that did not alter the interface). libdbi version numbers > were picked rather randomly at the beginning, but as we had the drivers in a > separate project, we wanted to avoid having, say, libdbi-drivers 8.3 depend on > libdbi 7.5. We therefore decided to have all 0.8.x releases of libdbi work > with > all 0.8.x releases of libdbi-drivers and so on. While this schema does not > strictly prevent changes to the application vs. libdbi interface (it prevents > changes to the libdbi vs. libdbi-drivers interface), it is unlikely to change > one interface without changing the other. Therefore we currently have "0.8" as > a rather randomly picked interface number and "1" as the revision number. Sorry, I should have investigated much more closely before speaking... I was basing all that on the SF bug discussion, which was more about the abstract use of the LIB_* stuff, I believe -- or at least pre-use. If there's already a procedure in place then I don't want to suggest we change it. > > If we follow your request to release the current CVS code (more on that > below), > libtool would require no change of LIB_CURRENT, but LIB_REVISION would need to > be bumped up. This is just what happens if we name the new release 0.8.2 (the > interface remains at "0.8", the revision moves to "2". > > What I suggested in the discussion of the feature request back then is to > make a > clean start with the version and interface numbers next time we need to change > the interface. The interface number, and thus the major version, should be > "1", > and the revision (= minor version) should start at "0". This is equivalent to > stating that we'll bump up the major version whenever the interface changes, > and the minor version whenever we add features that do not require a change of > the interface. If we agree on that, there is no reason not to use the libtool > numbers as the version number (LIB_CURRENT.LIB_REVISION). It is one set of > numbers less that need the attention of the developers. > Sure, and for a library package that definitely makes sense. In fact the more I think about it the more sense it makes... :) [sinp] > > > On a related note, we should also consider adding the "link libdbi > > into drivers" patch that's been floating around into CVS. Our project > > requires that we do this, as it uses the > > > > Parent => dl_open(app-logic) => dl_open(libdbi) => dl_open(libdbi-driver) > > > > pattern. When libdbi goes to load the driver it can't due to missing > > symbols that the driver requires in the loading binary. Linking > > libdbi into the drivers fixes this, it's a one line change in the > > Makefile, and it does not break the more traditional setup. In fact, > > this would help with the above mentioned libdbi <-> driver interface > > checks. > > > > Note that any apache module that attempts to use libdbi will also > > suffer from this. > > > > I recall this discussion, but I didn't have enough time to look into this. > We'll > have to test whether the traditional setup is not affected by this on any > platform supported so far. I'm especially concerned about Cygwin and Mingw. To > speed up things, do you have a link to that patch handy? > It's actually the libdbi-drivers Makefile that needs patching, and it's currently specific to linux builds for us. Mingw
Re: [libdbi-users] driver issues and timezones
On 2/12/07, Markus Hoenicka <[EMAIL PROTECTED]> wrote: > Mike Rylander <[EMAIL PROTECTED]> was heard to say: > > > Continuing this long-dormant thread, I want to thank the libdbi core > > team for the help and support in getting basic ISO timezone support > > into the code base. It has helped the Evergreen[1] project immensely, > > as we wouldn't have a fast and stable database connector without it. > > > > It is nice to see such a high-profile project use libdbi successfully! > > > Is it time for 0.8.2? > > > > libdbi has a history of being developed in batch mode. Development usually is > dormant as long as no one complains about bugs or missing features. Maybe it > is Entirely understandable, especially given that current stability of the code. > time again for a wake-up call to the hibernating developers to collect all > issues which should be addressed in an upcoming new release. From my POV, the > following things come to mind (they have been requested previously, but I > couldn't find the mails right away as I'm away from my development box): > > - change the release version numbers to the library interface numbers. The > issue > has been discussed before > (http://sourceforge.net/tracker/index.php?func=detail&aid=1578577&group_id=65979&atid=512948). > This change will allow better version checks by the runtime linker and thus > will > avoid frustrations when using applications built against an incompatible > libdbi > version. > hmmm... The libtool interface version number is about compatibility, where as the release number is about human-understandable "freshness". These are orthogonal concepts, though the former does deserve attention. It seems to me that the LIB_CURRENT and LIB_AGE should just both be 0, since there is no previously versioned interface. Then LIB_CURRENT just gets bumped when new functions are added, and LIB_AGE gets bumped as old functions are changed, deprecated or removed. Trying to make the version number match the interface number seem both counter productive (a new interface version for each release?) and against libtool best practices, no? ... I may be misunderstanding the proposal, though. > - add a "late binding" interface to libdbi. Currently you have either to know > in > advance which type a particular query is going to return, or you have to add > type checks after each query in order to find out. A "late binding" interface > would allow to request e.g. a string or an integer from any type of field, > with > libdbi performing all required conversions. > That actually seems fairly trivial for all types except DBI_TYPE_BINARY -- that might require base64 encoding or the like. I'll look into doing this. > - the drivers should be overhauled too where necessary. I've fiddled with some > experimental code to improve concurrent accesses with sqlite3. Also, we should > check at least the MySQL and PostgreSQL drivers with the latest versions of > these database engines to work out any compatibility issues. > I've been using the Postgres driver against an 8.2 database in testing, and that seems to work just fine. On a related note, we should also consider adding the "link libdbi into drivers" patch that's been floating around into CVS. Our project requires that we do this, as it uses the Parent => dl_open(app-logic) => dl_open(libdbi) => dl_open(libdbi-driver) pattern. When libdbi goes to load the driver it can't due to missing symbols that the driver requires in the loading binary. Linking libdbi into the drivers fixes this, it's a one line change in the Makefile, and it does not break the more traditional setup. In fact, this would help with the above mentioned libdbi <-> driver interface checks. Note that any apache module that attempts to use libdbi will also suffer from this. > The above issues are just suggestions. If anything else seems worth to be > included in the next release, feel free to add your favourites. > Having said all everything above, I'd personally prefer to see a new version out right now with the current code. It is tested and stable, and does exactly what I need (other than the driver-libdbi linking issue, but that's separate). There seems to have been a descent amount of code added since 0.8.1 was released, and now that this codebase is in use in a large production project (it's been tested stable, etc) my immediate concern is stamping the code as "good" and being able to point others at a tarball to say "that is tested and works." As it stands, I get funny looks (or worse, no second look at all) when I say, "Grab libdbi from CVS. Yes, it's stable and safe, it's jus
Re: [libdbi-users] driver issues and timezones
Continuing this long-dormant thread, I want to thank the libdbi core team for the help and support in getting basic ISO timezone support into the code base. It has helped the Evergreen[1] project immensely, as we wouldn't have a fast and stable database connector without it. We are now relying on the code in CVS for our production services at PINES[2], and we're getting a good deal of outside interest and test implementations. As such, we're pulling libdbi from CVS when we need to build a new server, and we're advising others to do the same. As you can imagine, this is less than optimal for sites that are short on time or expertise, and gives a feeling (at least to those uninitiated with open source) of bleeding-edge requirements that can turn less experimental potential users off. So, with that, I'd like to see what the general feeling of the community would be to releasing a new tarball for the current codebase. As I mentioned, we're currently using this heavily in production and we are extremely happy with the performance and stability. I'm not sure if any other projects have adopted or tested the current code, but as far as the timezone support goes, it is very stable in our use. Is it time for 0.8.2? TIA [1] http://open-ils.org [2] http://gapines.org and http://www.georgialibraries.org UPDATE: While looking at the code I notices that one of my original debugging fprintf's managed to hang around. Attached is a patch that comments out that one line, and does nothing else. It's probably just as easy for someone to go in and remove all the fprintf lines, commented out or not, since they're not needed at all, AFAICS. Thanks again! -- Mike Rylander [EMAIL PROTECTED] GPLS -- PINES Development Database Developer http://open-ils.org On 5/16/06, Mike Rylander <[EMAIL PROTECTED]> wrote: On 5/16/06, Markus Hoenicka <[EMAIL PROTECTED]> wrote: > Mike Rylander writes: > > I've attached a patch against my local copy of the new file, so it > > should patch CVS with out too much of a fight. I've added a little > > logic to look for a ':' minute separator in the -8:00 format. It > > compiles and runs for me, so please give it a whirl and make sure it > > passes muster. > > > > The patch works as advertized. However, I felt the same could be > achieved with less code, and I started to reimplement the timezone > handling by reversing the logic. Checking for a separator first allows > a cleaner implementation imho. I've checked in the code, a patch > against the pre-timezone dbd_helper.c is attached. Please check > whether the current implementation works ok for your testcases. The patch works perfectly, and the logic is definitely more readable now. Thanks again! > > regards, > Markus > > > > > -- > Markus Hoenicka > [EMAIL PROTECTED] > (Spam-protected email: replace the quadrupeds with "mhoenicka") > http://www.mhoenicka.de > > > -- Mike Rylander [EMAIL PROTECTED] GPLS -- PINES Development Database Developer http://open-ils.org libdbi-fprintf-comment.patch Description: Binary data - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Finding inserted rows
xample-auto-increment.html > > The moral of this story is it's pretty difficult to get a *portable* > usage pattern for incrementing INSERT ids, but each database tends to > have a successful workaround. > > --Toby > > > > > > > regards, > > > Markus > > > > > > -- > > > Markus Hoenicka > > > [EMAIL PROTECTED] > > > (Spam-protected email: replace the quadrupeds with "mhoenicka") > > > http://www.mhoenicka.de > > > > > > > > > > > > ------- > > > All the advantages of Linux Managed Hosting--Without the Cost > > and Risk! > > > Fully trained technicians. The highest number of Red Hat > > certifications in > > > the hosting industry. Fanatical Support. Click to learn more > > > http://sel.as-us.falkag.net/sel? > > cmd=lnk&kid=107521&bid=248729&dat=121642 > > > ___ > > > libdbi-users mailing list > > > [EMAIL PROTECTED] > > > https://lists.sourceforge.net/lists/listinfo/libdbi-users > > > > > > > > > -- > > Mike Rylander > > [EMAIL PROTECTED] > > GPLS -- PINES Development > > Database Developer > > http://open-ils.org > > > > > ___ > libdbi-users mailing list > libdbi-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/libdbi-users > -- Mike Rylander [EMAIL PROTECTED] GPLS -- PINES Development Database Developer http://open-ils.org ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] Finding inserted rows
OT from the original post, but pertinent to the discussion...
On 5/27/06, Markus Hoenicka <[EMAIL PROTECTED]> wrote:
Martin Kutschker writes:
> Have the systems which use sequences some nameing schemes, which build the
sequence name from the table name?
>
SQLite and MySQL don't use sequences, so I can speak only of
PostgreSQL (don't know about Oracle, Sybase et al.). If you use the
Oracle does use sequences, and they look quite a bit like Pg sequences
but require a little more work to use AFAICT.
Sybase (well, the MSSQL version) has a "magical" column property that
feels more like MySQL's autoincrement.
http://troels.arvin.dk/db/rdbms/#mix-identity
Serial type in PostgreSQL, a sequence is automatically created with a
predictable name (tablename_colname_seq). However, you can do this
just as well:
CREATE SEQUENCE whatever_seq;
CREATE TABLE tablename (
colname integer DEFAULT nextval('whatever_seq') NOT NULL
A mechanism that relies on a particular naming scheme of sequences
would fail in the second case.
While it would take a little work in the Pg libdbi driver to detect
the version (if that's not already done), 8.1 has a function called
lastval() that returns the last value produced by the last sequence
used. This falls down if you use more than one sequence on a table
(say, as a default on a non-SERIAL column, as I do in some cases), but
it works well for the common case. The driver could use the sequence
name if provided, otherwise, under 8.1, it would 'SELECT lastval()'
and return the expected value in either case.
Applications would have to take care to be coded to Pg 8.1+ in order
to use the "magic" sequence stuff, but it's more portably future
proof, IMHO. Of course, older apps that provide a sequence name would
continue to function as they do now.
> And perhaps dbi couild handle the problem that Mysql doesn't store the id by
table or sequence but by connection by automatically calling
mysql_last_insert_id() on every insert and storing the ids in a dbi datastructure
per table.
>
I don't see a real problem here. If an application makes sure to store
relevant id values after INSERT commands, you'll get a portable code
without causing dbi to interfere with tables behind the users back.
regards,
Markus
--
Markus Hoenicka
[EMAIL PROTECTED]
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de
---
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
--
Mike Rylander
[EMAIL PROTECTED]
GPLS -- PINES Development
Database Developer
http://open-ils.org
---
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] driver issues and timezones
On 5/16/06, Markus Hoenicka <[EMAIL PROTECTED]> wrote: Mike Rylander writes: > I've attached a patch against my local copy of the new file, so it > should patch CVS with out too much of a fight. I've added a little > logic to look for a ':' minute separator in the -8:00 format. It > compiles and runs for me, so please give it a whirl and make sure it > passes muster. > The patch works as advertized. However, I felt the same could be achieved with less code, and I started to reimplement the timezone handling by reversing the logic. Checking for a separator first allows a cleaner implementation imho. I've checked in the code, a patch against the pre-timezone dbd_helper.c is attached. Please check whether the current implementation works ok for your testcases. The patch works perfectly, and the logic is definitely more readable now. Thanks again! regards, Markus -- Markus Hoenicka [EMAIL PROTECTED] (Spam-protected email: replace the quadrupeds with "mhoenicka") http://www.mhoenicka.de -- Mike Rylander [EMAIL PROTECTED] GPLS -- PINES Development Database Developer http://open-ils.org --- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid0709&bid&3057&dat1642 ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] driver issues and timezones
On 5/16/06, Markus Hoenicka <[EMAIL PROTECTED]> wrote:
Mike Rylander writes:
> I'd be interested to see what the optimization was. :)
>
I've replaced
if (strlen(*cur)) {
with
if (*cur) {
Cool. Not sure why I didn't do that originally... :)
Saves us one function call. Whoohoo.
Besides, I've changed index() calls to strchr() calls just to make
the code consistent.
OK. I've continued with strchr() ... more on that below.
> > 04:05:06-08:00 (time)
> > 1999-01-08 04:05:06 -8:00 (timestamp)
>
> It may just be a typo, but this is incorrect based on my original
> patch. I followed ISO8601 format for using numeric timezone
> extentions on a timestamp, which requires the format of
>
> {+|-}hh[:mm]
>
According to my tests both '-8:00' and '-08:00' work. '-0800' is
another option, whereas '-800' fails (correctly so). I did not change
the logic of your string parsing. Your patch is probably a bit more
forgiving than you thought. In any case, according to the PostgreSQL
docs all of the above notations that work conform to ISO8601.
Ahh.. I see. Unfortunately, I believe that there is a bug here in my
original patch. I'm looking at the total length of the timezone
string sans direction marker, and only if it is greater than 4 do I
assume there is a ':' minute separator. If the length is exactly 4
then I am assuming the zero padded, four digit format. This works
fine with complete hour offsets, but some timezones require half or
quarter hours, and the original patch won't work with those timezones
in the -8:00 format -- they will be split like this: -5:45 --> (5:,
5), instead of (5, 45).
I've attached a patch against my local copy of the new file, so it
should patch CVS with out too much of a fight. I've added a little
logic to look for a ':' minute separator in the -8:00 format. It
compiles and runs for me, so please give it a whirl and make sure it
passes muster.
Thanks again for taking quick action on this, and being open to
patches from complete outsiders. That's very encouraging!
regards,
Markus
--
Markus Hoenicka
[EMAIL PROTECTED]
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de
--
Mike Rylander
[EMAIL PROTECTED]
GPLS -- PINES Development
Database Developer
http://open-ils.org
libdbi-time_t-timezone-correction.patch
Description: Binary data
Re: [libdbi-users] driver issues and timezones
On 5/15/06, Markus Hoenicka <[EMAIL PROTECTED]> wrote:
Mike Rylander writes:
> [UPDATE]While my coworker was investigating the problem
> described above further, I've developed (and attached) a patch to
> dbd_helper.c to parse ISO8601 timezone offsets inside
> _dbd_parse_datetime() if they happen to occur. I use Postgres as my
> main database, and this allows me to use TIMESTAMP WITH TIME ZONE
> columns and get the correct date and time out, without having to
> mangle the timezone on the system (or inside the database) to assume
> UTC. It is untested with other databases, but it's fairly protective,
> looking for specific timezone markers, and should be able to handle
> any standard ISO timezone offset.
>
I've applied this patch (with one micro-optimization). I'll have to
I'd be interested to see what the optimization was. :)
add some information to the driver docs in order to explain how
temporal types are handled. For the time being, the following
description must suffice:
- you can pass time and datetime values with a numeric timezone
suffix. The suffix starts with '-' or '+'. The timezone can be given
either as hours only ('-8'), as hours and minutes ('-800'), or as
hours and minutes separated by a colon ('-8:00'). Other forms,
e.g. strings like 'PST', are currently not supported. Examples:
04:05:06-08:00 (time)
1999-01-08 04:05:06 -8:00 (timestamp)
It may just be a typo, but this is incorrect based on my original
patch. I followed ISO8601 format for using numeric timezone
extentions on a timestamp, which requires the format of
{+|-}hh[:mm]
Postgresql seems to drop the ':' from timezones from within the psql
shell, so I am checking for that permutation as well.
- some database engines support timezones by default and provide no
special column type. sqlite and sqlite3 use a string notation
internally and therefore have no problems with timezones.
- some database engines use specific column types to honor a timezone
suffix. This is true for pgsql. Use TIMESTAMP WITH TIME ZONE
instead of TIMESTAMP and TIME WITH TIME ZONE instead of TIME if you
want to use timezone suffixes.
- some database engines ignore the timezone suffix. This is true for
mysql.
- I can't tell how the other supported database engines and their
drivers handle timezones. Do the driver maintainers listen?
- due to the differences between the implementations, the use of
timezone data creates a portability problem. If your application is
supposed to support all available libdbi drivers, it may be
necessary to do the math inside your application before you store
temporal data.
regards,
Markus
--
Markus Hoenicka
[EMAIL PROTECTED]
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de
--
Mike Rylander
[EMAIL PROTECTED]
GPLS -- PINES Development
Database Developer
http://open-ils.org
---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid0709&bid&3057&dat1642
___
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users
[libdbi-users] driver issues and timezones
I'm a new user of libdbi, and so far it's great. I have no problem building simple test programs, and, other than timezone support for DBI_DATETIME types (which I plan to submit a patch for), it does exactly what I need. However, I'm having a problem using libdbi from within a shared object. In much the same way that Apache loads DSOs, we have built a server infrastructure that allows loading of application implementation shared libraries. The problem is that in order for libdbi to load a driver on demand we must link both libdbi proper (normal) any driver we may want to use (eh?) against our implementation libraries. Not only that, but we have to add an LDFLAGS setting to the libdbi-drivers configure.in to tell it to link against libdbi.so. I'm hoping that we're just overlooking something simple here, but we've been pulling our hair out for the last couple days trying to figure this out, and that's the only solution we could come up with. Thanks in advance for any help you can provide. [UPDATE]While my coworker was investigating the problem described above further, I've developed (and attached) a patch to dbd_helper.c to parse ISO8601 timezone offsets inside _dbd_parse_datetime() if they happen to occur. I use Postgres as my main database, and this allows me to use TIMESTAMP WITH TIME ZONE columns and get the correct date and time out, without having to mangle the timezone on the system (or inside the database) to assume UTC. It is untested with other databases, but it's fairly protective, looking for specific timezone markers, and should be able to handle any standard ISO timezone offset. Thanks again, in advance. -- Mike Rylander [EMAIL PROTECTED] GPLS -- PINES Development Database Developer http://open-ils.org libdbi-time_t-timezone.patch Description: Binary data
