Just to let you know. I merged the fix manually onto LMDB_0.9.16 and it works.
https://github.com/deephacks/lmdbjni/commit/54a6ed3f1e0896a29e4bd10594bf9c80b0b25c65 On Fri, Nov 20, 2015 at 10:34 AM, Kristoffer Sjögren <[email protected]> wrote: > The last build I know worked for lmdbjni on windows used LMDB_0.9.14. > > I was going to try a patch release on LMDB_0.9.16 but the fix doesn't > merge cleanly. > > On Fri, Nov 20, 2015 at 10:26 AM, Howard Chu <[email protected]> wrote: >> Kristoffer Sjögren wrote: >>> >>> Yep, the commit fixes the problem. Thanks Howard! >> >> >> The only question now - what is "modified" in the Java UTF-8 encoding? >> >>> >>> On Fri, Nov 20, 2015 at 2:14 AM, Howard Chu <[email protected]> wrote: >>>> >>>> Kristoffer Sjögren wrote: >>>>> >>>>> >>>>> Changing CharacterSet in the build file doesn't affect the garbled path. >>>>> >>>>> I tried to use explicit GetStringChars from the JNI code but with same >>>>> result. >>>>> >>>>> Any pointers? I'm running short of ideas.. >>>> >>>> >>>> >>>> Looks like we need to use UTF-8, as in ITS#7992. I've just merged that >>>> patch >>>> into mdb.master, see if that works better. You should use >>>> GetStringUTFChars. >>>> >>>>> >>>>> On Thu, Nov 19, 2015 at 11:55 PM, Kristoffer Sjögren <[email protected]> >>>>> wrote: >>>>>> >>>>>> >>>>>> Oh, but the build file [2] have a CharacterSet element set to Unicode. >>>>>> Let me see if I can change this. >>>>>> >>>>>> [2] >>>>>> >>>>>> https://github.com/deephacks/lmdbjni/blob/master/lmdbjni/src/main/native-package/vs2010.vcxproj >>>>>> >>>>>> On Thu, Nov 19, 2015 at 11:45 PM, Kristoffer Sjögren <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> I can see the problem now from a local test. >>>>>>> >>>>>>> Hmm i'm a bit confused. The JNI code uses GetStringUTFChars calls [1] >>>>>>> for all char * arguments going from Java to C, which is an array of >>>>>>> bytes representing the string in modified UTF-8 encoding. Is this OK? >>>>>>> The other option is to use GetStringChars which is unicode... >>>>>>> >>>>>>> [1] >>>>>>> >>>>>>> http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html >>>>>>> >>>>>>> On Thu, Nov 19, 2015 at 10:32 PM, Howard Chu <[email protected]> wrote: >>>>>>>> >>>>>>>> >>>>>>>> Kristoffer Sjögren wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> The py-lmdb project have a comment [1] about reusing Python.h >>>>>>>>> headers >>>>>>>>> which seems to work for them. >>>>>>>>> >>>>>>>>> # Microsoft Visual Studio 9 ships with neither inttypes.h, stdint.h, >>>>>>>>> or a >>>>>>>>> sane >>>>>>>>> # definition for ssize_t, so here we add lib/win32 to the search >>>>>>>>> path, >>>>>>>>> which >>>>>>>>> # contains emulation header files provided by a third party. We >>>>>>>>> force-include >>>>>>>>> # Python.h everywhere since it has a portable definition of ssize_t, >>>>>>>>> which >>>>>>>>> # inttypes.h and stdint.h lack, and to avoid having to modify the >>>>>>>>> LMDB >>>>>>>>> source >>>>>>>>> # code. Advapi32 is needed for LMDB's use of Windows security APIs. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Note in ITS#8067 we added an ssize_t typedef for MSVC. Also in #8069 >>>>>>>> we >>>>>>>> changed to explicitly use the ANSI APIs, so if you're using Unicode >>>>>>>> file >>>>>>>> paths that will certainly be garbled now. >>>>>>>> >>>>>>>>> >>>>>>>>> [1] https://github.com/dw/py-lmdb/blob/master/setup.py >>>>>>>>> >>>>>>>>> On Thu, Nov 19, 2015 at 10:13 PM, Kristoffer Sjögren >>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The actual build command: >>>>>>>>>> >>>>>>>>>> $ cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release" >>>>>>>>>> >>>>>>>>>> On Thu, Nov 19, 2015 at 10:11 PM, Kristoffer Sjögren >>>>>>>>>> <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> That's the thing, the build doesn't complain about anything >>>>>>>>>>> missing. >>>>>>>>>>> But the binary seems broken because tests fail randomly. >>>>>>>>>>> >>>>>>>>>>> A user tried the generated binary on his machine and it worked but >>>>>>>>>>> also >>>>>>>>>>> said... >>>>>>>>>>> >>>>>>>>>>> "Hm, almost, the database file is no longer put to the requested >>>>>>>>>>> directory, and its name is totally garbled. It finds it >>>>>>>>>>> consistently >>>>>>>>>>> though, and the contents are correct." >>>>>>>>>>> >>>>>>>>>>> https://github.com/deephacks/lmdbjni/issues/48 >>>>>>>>>>> >>>>>>>>>>> On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Kristoffer Sjögren wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> I'm trying to build LMDB with Java/JNI bindings with Visual C++ >>>>>>>>>>>>> Project Builder 9.00.30729 (vcbuild). >>>>>>>>>>>>> >>>>>>>>>>>>> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, or >>>>>>>>>>>>> a >>>>>>>>>>>>> sane >>>>>>>>>>>>> ssize_t. So I searched around and found a few candidates [1] of >>>>>>>>>>>>> inttypes.h and stdint.h that seems to be working for py-lmdb. >>>>>>>>>>>>> >>>>>>>>>>>>> However, unistd.h seems broken for windows - and the python guys >>>>>>>>>>>>> use >>>>>>>>>>>>> python.h instead. >>>>>>>>>>>>> >>>>>>>>>>>>> Any ideas where I might find a good unistd.h? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I see you have one already - what is wrong with it, what >>>>>>>>>>>> definitions >>>>>>>>>>>> are >>>>>>>>>>>> missing? >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers, >>>>>>>>>>>>> -Kristoffer >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers >>>>>>>>>>>>> >> >> >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/
