--0000000000000dacee05826a34a9 Content-Type: text/plain; charset="UTF-8"
Thank you, I appreciate it! Do you know how often or when this will get into the github mirror https://github.com/LMDB/lmdb/commits/mdb.RE/0.9 (or if I am misunderstanding how the branches are synced)? On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <[email protected]> wrote: > [email protected] wrote: > > Full_Name: Kristopher William Zyp > > Version: LMDB 0.9.23 > > OS: Windows Server 2012 R2, Windows 10 > > URL: > https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd32ab4e8e1f2.patch > > Submission from: (NULL) (71.199.6.148) > > > > Thanks for the report and patch, added to mdb.RE/0.9 > > > Calling mdb_env_set_mapsize to increase the map size, when a DB is also > in use > > by other processes, when MDB_WRITEMAP is enabled (and the db file has > been > > opened with PAGE_READWRITE access), on Windows, will occasionally (seems > like > > about 1/100 attempts fails) produce an error "The requested operation > cannot be > > performed on a file with a user-mapped section open", or segfaults. The > error > > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map that > is > > performed to increase the allocated file size to the map size, prior to > > CreateFileMapping. > > > > As it turns out this is pretty easy to solve, because manually expanding > the > > file size is not necessary when calling CreateFileMapping with > PAGE_READWRITE > > access, as Windows will automatically expand the file size for us, when > opened > > with the page write access enabled. Of course, this means all processes > must be > > consistent in use of MDB_WRITEMAP, but the documentation already makes > this > > explicit and clear. > > > > I believe this can be fixed by simply adding a check for MDB_WRITEMAP in > the if > > statement that calls SetFilePointer: > > > > if (!(flags & MDB_WRITEMAP) && (SetFilePointer(env->me_fd, > sizelo, &sizehi, 0) > > != (DWORD)sizelo > > || !SetEndOfFile(env->me_fd) > > || SetFilePointer(env->me_fd, 0, NULL, 0) != 0)) > > return ErrCode(); > > > > The attached URL has the change as a patch/diff as applied to our node > package. > > > > I am certainly happy to just keep this change on our own branches. There > may be > > nuances of this that I might not be aware of, but it seems to be working > great > > for us and I have tested this with MDB_WRITEMAP enabled and disabled. So > I > > thought I would offer/suggest this change, as it seems like it is > > straightforward change to improve stability. Thank you! > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > --0000000000000dacee05826a34a9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr">Thank you, I appreciate it! Do you know h= ow often or when this will get into the github mirror <a href=3D"https://gi= thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB/lmdb/commits= /mdb.RE/0.9</a> (or if I am misunderstanding how the branches are synced)?<= /div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_a= ttr">On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <<a href=3D"mailto:hyc@s= ymas.com">[email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_= quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,= 204);padding-left:1ex"><a href=3D"mailto:[email protected]"; target=3D"_blan= k">[email protected]</a> wrote:<br> > Full_Name: Kristopher William Zyp<br> > Version: LMDB 0.9.23<br> > OS: Windows Server 2012 R2, Windows 10<br> > URL: <a href=3D"https://github.com/kriszyp/node-lmdb/commit/6df903907f= 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D"noreferrer" target=3D"_blank">= https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd32= ab4e8e1f2.patch</a><br> > Submission from: (NULL) (71.199.6.148)<br> > <br> <br> Thanks for the report and patch, added to mdb.RE/0.9<br> <br> > Calling mdb_env_set_mapsize to increase the map size, when a DB is als= o in use<br> > by other processes, when MDB_WRITEMAP is enabled (and the db file has = been<br> > opened with PAGE_READWRITE access), on Windows, will occasionally (see= ms like<br> > about 1/100 attempts fails) produce an error "The requested opera= tion cannot be<br> > performed on a file with a user-mapped section open", or segfault= s. The error<br> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map tha= t is<br> > performed to increase the allocated file size to the map size, prior t= o<br> > CreateFileMapping.<br> > <br> > As it turns out this is pretty easy to solve, because manually expandi= ng the<br> > file size is not necessary when calling CreateFileMapping with PAGE_RE= ADWRITE<br> > access, as Windows will automatically expand the file size for us, whe= n opened<br> > with the page write access enabled. Of course, this means all processe= s must be<br> > consistent in use of MDB_WRITEMAP, but the documentation already makes= this<br> > explicit and clear.<br> > <br> > I believe this can be fixed by simply adding a check for MDB_WRITEMAP = in the if<br> > statement that calls SetFilePointer:<br> > <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!(flags &= ; MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo, &size= hi, 0)<br> > !=3D (DWORD)sizelo<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0|| !SetEndOfFile(env->me_fd)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0|| SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0))<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0return ErrCode();<br> > <br> > The attached URL has the change as a patch/diff as applied to our node= package.<br> > <br> > I am certainly happy to just keep this change on our own branches. The= re may be<br> > nuances of this that I might not be aware of, but it seems to be worki= ng great<br> > for us and I have tested this with MDB_WRITEMAP enabled and disabled. = So I<br> > thought I would offer/suggest this change, as it seems like it is<br> > straightforward change to improve stability. Thank you!<br> > <br> > <br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com"; rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div> --0000000000000dacee05826a34a9--
