Ah, and to think I just fixed 4.4.20 ;-)
Removing the dependency on db_static.vcproj and merging the relevant source
code files into _bsddb.vcproj did the trick -- all x64 bsddb-related tests now
pass. The only issue with this approach is that it locks _bsddb.vcproj into
4.4.20. However, considering that this approach (i.e. bringing their source
files into our build instead of linking against a static lib compiled with
wildly incompatible flags) only took me about two minutes to implement and
immediately fixed every bsddb problem I was encoutering, I'm convinced it's the
right way to go. (I can separate the dependencies easily enough.)
Woeful PyCon/hotel connectivity is preventing me from getting to
bugs.python.org at the moment; I'll raise a ticket later to capture this stuff
and we can move the discussion there once I've attached some patches.
Trent.
From: Gregory P. Smith [EMAIL PROTECTED]
Sent: 14 March 2008 00:23
To: Trent Nelson
Cc: [email protected]; Jesus Cea
Subject: Re: Windows x64 & bsddb 4.4.20 woes
On 3/13/08, Trent Nelson <[EMAIL PROTECTED]> wrote:
Hey Greg,
I'm at PyCon indeed, staying through the sprints 'til next Thursday. I'll drop
you a note offline re catching up.
The other query I had was whether or not I should try a later version of
BerkeleyDB -- are we committed to 4.4.20 (or 4.4.x?) for 2.6/3.0 or is it worth
investigating newer versions? Martin/Jesus, any thoughts on this?
Python 2.6/3.0 should be built on Windows using BerkeleyDB 4.5.x for now.
4.6.x is out but has some bugs on some platforms so i don't recommend shipping
our release using it; 4.7.x is in beta and some bugs are being worked on; if
its out and shows no signs of obvious issues before the 2.6/3.0 beta period is
over I recommend we build our binary releases using it. Otherwise 4.5 it will
be. There is no reason to use 4.4.x.
Regarding the db_static build and conflicting compile/link options -- I'm going
to bring the db_static source directly into the _bsddb project (for now) which
should make this a lot easier to debug.
Trent.
From: Gregory P. Smith [EMAIL PROTECTED]
Sent: 13 March 2008 22:00
To: Trent Nelson
Cc: [email protected]; Jesus Cea
Subject: Re: Windows x64 & bsddb 4.4.20 woes
I haven't built the bsddb stuff on windows myself in a few years and have never
had access to a windows x64 system so I'm no silver bullet. Making the
BerkeleyDB compile and link options match with those of python is the first
place I'd start. Also you should be able to make a debug build of BerkeleyDB
(though it sounds like you may have tried that already?). Next off in the
debugging i'd take a look at the assembly to see what exactly it was failing to
do.
If you're at PyCon right now we should meet up and try to figure it out (I just
arrived).
On 3/13/08, Trent Nelson <[EMAIL PROTECTED]> wrote:
I've been trying to give the Windows x64 builds a bit of TLC the past few
evenings. I managed to get a successful build with all external modules last
night (Tcl/Tk required about a half a dozen code/configuration changes each in
order to build in a Windows x64 environment with Visual Studio 9, I'll deal
with that in a separate thread or roundup issue).
Unfortunately, though, we're back to more bsddb issues. I got about 15 tests
in without error before test_whichdb ran, which results in the following being
called in dbhash.py:
return bsddb.hashopen(file, flag, mode)
I can trace that call to DBEnv_open() in _bsddb.c:
static PyObject*
DBEnv_open(DBEnvObject* self, PyObject* args)
{
int err, flags=0, mode=0660;
char *db_home;
if (!PyArg_ParseTuple(args, "z|ii:open", &db_home, &flags, &mode))
return NULL;
CHECK_ENV_NOT_CLOSED(self);
MYDB_BEGIN_ALLOW_THREADS;
err = self->db_env->open(self->db_env, db_home, flags, mode);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Placing a breakpoint at the line above and stepping in results in Visual Studio
reporting: " A buffer overrun has occurred in python_d.exe which has corrupted
the program's internal state. Press Break to debug the program or Continue to
terminate the program.". FWIW, the exception is being raised as part of the
/GS buffer overflow checks (implemented in gs_result.c, which is provided in my
VS9 installation).
This has been annoyingly awkward to debug. I can't break down that call into
multiple steps in order to try place breakpoints in the db_static module. The
callstack isn't that useful either:
_bsddb_d.pyd!__crt_debugger_hook()
_bsddb_d.pyd!__report_gsfailure(unsigned __int64 StackCookie=2211040)
_bsddb_d.pyd!__GSHandlerCheckCommon(void * EstablisherFrame=0x000000000021bce0,
...)
_bsddb_d.pyd!__GSHandlerCheck(_EXCEPTION_RECORD *
ExceptionRecord=0x000000000021bbc0, ...)
ntdll.dll!00000000773ae13d()
[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]
ntdll.dll!00000000773aea57()
ntdll.dll!00000000773b59f8()
_bsddb_d.pyd!__os_strdup() + 0x18 bytes
_bsddb_d.pyd!__os_tmpdir() + 0x281 bytes
You'd think placing breakpoints in db 4.4.20's __os_strdup and __os_tmpdir
methods would do something, but alas, the bufferoverflow exception is raised
before any breakpoints are set. This makes me suspect there's something funky
going on with the entire build and linking of db_static (VS should honour those
breakpoints if the code is being entered, I even added db_static to pcbuild.sln
and rebuilt but no dice). I've noticed that they're not using consistent
compiler flags by default (we use /GS, they use /GS-, we allow function level
linking, they don't -- note that I did change db_static's options to align with
_bsddb's but the bufferoverflow exception is still being thrown).
Greg, Jesús, I'm CC'ing you guys as stfw'ing seems to bring back you two the
most when it comes to bsddb issues. I've still got a list of things to try
with regarding to debugging this x64 issue, but I wanted to reach out now to
see if anyone else had encountered it before. Has bsddb ever been built
successfully on Win64 and passed all tests or am I venturing into new ground?
Martin, you've changed externals/bsddb-4.4.20 with regards to x64 builds
recently -- have you been able to get things working in your x64 environments?
Regards,
Trent.
_______________________________________________
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
