LMDB and text encoding

2015-01-27 Thread Timur Kristóf
is? Best regards, Timur Kristóf

Re: LMDB and text encoding

2015-01-29 Thread Timur Kristóf
> mdb_dbi_open treats its name parameter as a C string. This means UTF-8 on > unixes and ANSI on Windows, which is problematic for cross-platform > applications. [...] Here is a patch that addresses this concern. If you like it, I'll move on to the other issue. From 8c32675cbc4d32fe151b76ef28268af

Re: LMDB and text encoding

2015-01-29 Thread Timur Kristóf
I've had a brief chat with Hallvard on IRC. We came up with several possible solutions, although each of them has its drawbacks. Writing cross-platform code that supports unicode is always a messy business. I vote for option 4, but would like to hear everyone's opinions before starting to work on a

Re: LMDB and text encoding

2015-01-29 Thread Timur Kristóf
Here is a fixed version of the patch. On Thu, Jan 29, 2015 at 10:29 AM, Timur Kristóf wrote: >> mdb_dbi_open treats its name parameter as a C string. This means UTF-8 on >> unixes and ANSI on Windows, which is problematic for cross-platform >> applications. [...] > &

Re: LMDB and text encoding

2015-02-01 Thread Timur Kristóf
Hi, I forgot to add an ENOMEM check. I added it now. I think this patch is ready for Howard and Hallvard to review. :) Timur On Thu, Jan 29, 2015 at 2:42 PM, Timur Kristóf wrote: > Here is a fixed version of the patch. > > On Thu, Jan 29, 2015 at 10:29 AM, Timur Kristóf

Fwd: LMDB and text encoding

2015-02-02 Thread Timur Kristóf
On Mon, Feb 2, 2015 at 3:37 AM, Howard Chu wrote: > Hallvard Breien Furuseth wrote: >> >> On 02/02/15 00:40, Howard Chu wrote: >>> >>> It looks OK to me. No one raises any concerns I'll commit it in a few >>> hours. >> >> >> Some sudden last thoughts: >> >> mdb_dump.c also has a check (memchr(key.

Fwd: LMDB and text encoding

2015-02-02 Thread Timur Kristóf
> I just had a look at how BDB handled this. As you can see they used a > TO_TSTRING macro to convert incoming pathnames from UTF8 to UTF16. > > https://gitorious.org/berkeleydb/berkeleydb/source/347d239a1e44ed4f773ae9274c2a32cf2b8999c0:src/os_windows/os_open.c > > https://gitorious.org/berkeleydb/

Re: Fwd: LMDB and text encoding

2015-02-02 Thread Timur Kristóf
> DB names are purely internal to LMDB, so they bear no relation to OS > filenames and none of this discussion matters to them. If we let the users treat db names as an MDB_val (essentially, an arbitrary byte array), then all bets are off: we can't even make the assumption that a db name is meanin

Re: Fwd: LMDB and text encoding

2015-02-02 Thread Timur Kristóf
>> DB names are purely internal to LMDB, so they bear no relation to OS >> filenames and none of this discussion matters to them. > > They're exposed to the programmer and the program's users. Either may > want them on command-line arguments, in config files, etc. It will be > inconvenient if LMD

Re: Fwd: LMDB and text encoding

2015-02-02 Thread Timur Kristóf
>> A path is always a Unicode string, while a DB name can be an arbitrary >> binary blob. So I don't think that we can treat them the same way. > > > Not the point. A program which uses LDMB can choose to treat its > own DB names in its own LMDB environments as the same kind of > strings as filena

Re: Fwd: LMDB and text encoding

2015-02-02 Thread Timur Kristóf
> I suggest we wait to deal with DB names until we also have a way to > deal with filenames. And this time test that it works is practice. > > And then I also suggest to try to make this mess simple to deal > with for programmers and or users. I guess I should have separated > that from the rest

Re: Fwd: LMDB and text encoding

2015-02-15 Thread Timur Kristóf
> > A path is always a Unicode string, while a DB name can be an arbitrary > > binary blob. > > On many POSIX platforms, a path is a blob which does not contain > '\000'. These systems do not enforce Unicode encoding at all. My mistake. I was unaware. On those platforms, how do you type a path na

Re: Fwd: LMDB and text encoding

2015-02-15 Thread Timur Kristóf
>>> > A path is always a Unicode string, while a DB name can be an arbitrary >>> > binary blob. >>> >>> On many POSIX platforms, a path is a blob which does not contain >>> '\000'. These systems do not enforce Unicode encoding at all. >> >> My mistake. I was unaware. >> On those platforms, how do

LMDB and gitorious

2015-04-22 Thread Timur Kristóf
Hi, As you might've heard, Gitorious is shutting down on 15th May. Maybe we start should look for a new home for the projects at https://gitorious.org/mdb - do you guys have a plan for this? Cheers, Timur

Re: lmdb: is read operation permitted inside the write transaction?

2015-05-19 Thread Timur Kristóf
On Mon, 2015-05-18 at 23:03 +0200, Hallvard Breien Furuseth wrote: > On 12/05/15 19:36, Howard Chu wrote: > > Dmytro Milinevskyy wrote: > > > Basically I need to have an opportunity to call mdb_get while in > > > write > > > transaction. > > > The sequence is smth like: > > > - txn = mdb_txn_beg

Re: lmdb: is read operation permitted inside the write transaction?

2015-05-19 Thread Timur Kristóf
On Tue, 2015-05-19 at 17:05 +0100, Howard Chu wrote: > Timur Kristóf wrote: > > On Mon, 2015-05-18 at 23:03 +0200, Hallvard Breien Furuseth wrote: > > > On 12/05/15 19:36, Howard Chu wrote: > > > > Dmytro Milinevskyy wrote: > > > > > Basically I need

Re: Compiling LMDB with Visual C++

2015-12-23 Thread Timur Kristóf
> > Made some code changes to compile in VC++. Does anyone want? > > We don't really support Microsoft tools. You're welcome to submit > code patches  > against the latest code in mdb.master. MSVC Project files and other > such stuff  > will not be accepted, we have no desire to maintain ancillar

Re: Fwd: LMDB and text encoding

2017-06-07 Thread Timur Kristóf
Hi Everyone, I've just came accross this old thread and am wondering, is this still an issue? Does LMDB have a way to use non-ASCII path names with mdb_env_open in a cross-platform way? If not, would you guys accept patches to LMDB with this regard? Thanks, Timur

Re: LMDB encryption support

2017-08-10 Thread Timur Kristóf
Hi, > I've recently added support for page-level encryption to LMDB 1.x > using user-supplied callbacks That does sound cool. :) > One question is whether we should actually make this pluggable like > this, or > we should just hardcode support for a specific algorithm and leave it > at that.