d like so:
>
Fixed in git master
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
if (flags & MDB_APPEND) { < false flags = 0
>> MDB_val k2;
>> rc = mdb_cursor_last(mc, , );
>> if (rc == 0) {
>> rc = mc->mc_dbx->md_cmp(key, );
>> if (rc > 0) {
>> rc = MDB_NOTFOUND;
>> mc->mc_ki[mc->mc_top]++;
>> } else {
>> /* new key is <= last key */
>> rc = MDB_KEYEXIST;
>> }
>> }
>> } else {< executed
>> rc = mdb_cursor_set(mc, key, , MDB_SET, );
>> < returns -30798 which is MDB_NOTFOUND
>
> This is an important step. mdb_cursor_set only returns MDB_NOTFOUND when
> mc->mc_pg[mc->mc_top] != NULL.
> There are no code paths that can return this error code without setting the
> rootpage pointer mc->mc_pg[0] non-NULL.
> The only path that can allow this is if mc->mc_db->md_root == P_INVALID (the
> tree is empty) and we already know
> that's not true in this case. Nothing in LMDB will change these values
> between the time mdb_cursor_set
> returned and the time that mdb_cursor_put tries to reference mc->mc_pg[]. As
> such this sounds like a
> memory overwrite corruption from some other thread, unrelated to LMDB.
>
You can verify this by adding an assert after the mdb_cursor_set() call:
mdb_cassert(mc, rc != MDB_NOTFOUND || mc->mc_pg != NULL);
At this point I would check to see if your threads' stack sizes are large
enough. Since this
cursor came from mdb_put, it lives on the stack, and a stack overrun is the
most likely culprit.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
rc = MDB_NOTFOUND;
> mc->mc_ki[mc->mc_top]++;
> } else {
> /* new key is <= last key */
> rc = MDB_KEYEXIST;
> }
> }
> } else {< executed
> rc = mdb_cursor_set(mc, key, , MDB_SET, );
> < returns -30798 which is MDB_NOTFOUND
This is an important step. mdb_cursor_set only returns MDB_NOTFOUND when
mc->mc_pg[mc->mc_top] != NULL.
There are no code paths that can return this error code without setting the
rootpage pointer mc->mc_pg[0] non-NULL.
The only path that can allow this is if mc->mc_db->md_root == P_INVALID (the
tree is empty) and we already know
that's not true in this case. Nothing in LMDB will change these values between
the time mdb_cursor_set
returned and the time that mdb_cursor_put tries to reference mc->mc_pg[]. As
such this sounds like a
memory overwrite corruption from some other thread, unrelated to LMDB.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
(with 4KB pagesize), and the lockfile, which is always smaller
> than 2GB.
Ignore that, I was looking at the 0.9 branch when I wrote that.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP ht
ithub.com=
/kriszyp/node-lmdb/commit/726a9156662c703bf3d453aab75ee222072b990f
> >
>=20
>=20
> --=20
> =C2=A0 -- Howard Chu
> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0htt=
p://www.symas.com
> =C2=A0 Director, Highland Sun=C2=A0 =C
f putting this in the email thread (other code
>> discussion in GitHub), here is the latest squashed commit of the proposed
>> patch (with
>> the on-demand, retained overlapped array to reduce re-malloc and
>> opening event handles):
>>
>> http
072b990f
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
iggering a warning that may be of interest:
Thanks for the report. Appears to be a dup of ITS#8890, which was fixed in
master in 2018.
Looks like this fell through the cracks for the 2.4 release.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun
alid.
Which simply proves the point that just because something is not defined
in the C spec does not make it invalid. The compiler guys are idiots for
taking this position.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
|
> CompAlloc /* component matching has massively bitrotted */
> |
> * f
> )(...);
> ... when != E = E1
>when != E == NULL
>when != E != NULL
> (
> E->foo
> |
> E[...]
> |
> *(E)
> )
> --- malloc_check.cocci ---
>
> Patches coming s
}
> rc = mdb_txn_commit(txn);
>
> mdb_dbi_close(env, dbi);
> mdb_env_close(env);
> return 0;
> }
>
>>From my point of view the leak looks benign (=does not lead to
> database corruption or such), but I didn't dive deep enough in LMDB
> internals t
cn=monitor already have attributes for MDB page
> counts.
> The number of entries could be added there as attribute 'olmMDBEntries' or
> similar.
Added in git master
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/
ong, I replaced it with
>
> case MDB_DBNOSYNC:
>if ( c->value_int )
> mdb->mi_dbenv_flags |= MDB_NOSYNC;
> else
> mdb->mi_dbenv_flags &= ~MDB_NOSYNC;
>
> And it solves the issue on my system.
>
> Reproduced on master, and 2.4.48, on Debian
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
cause it only affects servers who are in
> already degraded condition (broken replication, invalid certificate on the
> provider) but I thought it still was worth reporting.
>
> I was able to reproduce this issue on the git master branch, on released
> 2.4.48,
> on Centos7 and on Debian Buster.
> OpenSSL version on the debian system: 1.1.1d-0+deb10u2
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
libc?), the
> sessionlog lands in the wrong chunk of uninitilized memory
>
> Additional info:
>
> # rpm -q glibc
> glibc-2.26-32.amzn2.0.2.x86_64
>
> # uname -a
> Linux 4.14.152-127.182.amzn2.x86_64 #1 SMP Thu Nov 14 17:32:43 UTC 2019 x86_64
> x86_64 x86_64 GNU/Linux
>
> # /usr/local/openldap/libexec/slapd -V
> @(#) $OpenLDAP: slapd 2.4.48 (Jan 8 2020 11:39:38) $
>
> (LTB build)
>
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
n http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
art with slappasswd -T=
, and I checked the newline.
I've run your steps. Your test is still invalid. Learn how to use the ech=
o command.
[root@507ab0515ed6 /]# od -xc /run/secrets/rootpw
0000a64
d \n
002
[root@507ab0515ed6 /]# echo -n d > /run/secrets/rootpw
[root@507ab0515ed6 /]# python /checkPassword.py $(slappasswd -T /run/secr=
ets/rootpw) d
Warning: Password file /run/secrets/rootpw is publicly readable/writeable
True
[root@507ab0515ed6 /]#
Closing this ITS.
>=20
>>> I saw the same issue in another openldap mail:
>>> https://www.openldap.org/lists/openldap-software/200805/msg00060.html=
but no
>>> answer.
>>>
>>>
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
in the Makefile, so none of this code is used.
Closing this ITS.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
HA, that's the point of its salt is
to
be random and different every time.
When using a password in a file you must ensure the trailing '\n' is omitted.
This
is already documented in the manpage.
>
> I saw the same issue in another openldap mail:
> https://www.openldap.org/lists/openlda
is to
> create a new mdb and populate it from the ground up. This has maintainance
> costs.
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
(argc=, argv=)
> at ../../../../servers/slapd/slapcat.c:53
> #10 0x55570867 in main (argc=1, argv=0x7fffe568) at
> ../../../../servers/slapd/main.c:410
> =
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
; + /* can't mdb_entry_return() because [Entry *x] init is incomplete */
> + op->o_tmpfree( x, op->o_tmpmemctx );
> + return rc;
> }
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
>
> This is a regression, because latest release 2.4.48 compiles without TLS
> succeessfully.
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
ture, you should tick the Major Security Issue button for potential CVEs so
they
can be handled privately before release.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
now calculated with signed years, but the result is still returned
as an unsigned value (with negative years properly sorting as less than positive
years).
Note that the slapd schema validator still rejects negative years at the moment.
Will need to look further at that later.
--
-- Howard Chu
CTO,
3D=3Dby 0x26B88B: slap_sasl_bind (sasl.c:1666)
>>> =3D=3D11019=3D=3Dby 0x21E130: fe_op_bind (bind.c:279)
>>> =3D=3D11019=3D=3Dby 0x21DCE1: do_bind (bind.c:205)
>>> =3D=3D11019=3D=3Dby 0x1F35BA: connection_operation (connection.c:=
1185)
>>> =3D=3D11019=3D=3Dby 0x1F3CE7: connection_read_thread (connection.=
c:1342)
>>> =3D=3D11019=3D=3Dby 0x35DFF9: ldap_int_thread_pool_wrapper (tpool=
.c:1048)
>>> =3D=3D11019=3D=3Dby 0x4DBE668: start_thread (pthread_create.c:479=
)
>>>
>>>
>>>
>>>
>>
>=20
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
on.c:1342)
> ==11019==by 0x35DFF9: ldap_int_thread_pool_wrapper (tpool.c:1048)
> ==11019==by 0x4DBE668: start_thread (pthread_create.c:479)
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
c3RlcGhhbkBzcmxhYnMuZGUgd3JvdGU6DQo+IEZ1bGxfTmFtZTogU3RlcGhhbiBaZWlzYmVy
Zw0KPiBWZXJzaW9uOiAyLjQuNDgNCj4gT1M6IEZlZG9yYSAzMSAoa2VybmVsIDUuMy4xMS0z
MDAuZmMzMS54ODZfNjQpDQo+IFVSTDogDQo+IFN1Ym1pc3Npb24gZnJvbTogKE5VTEwpICg3
OC41NC42NS4xMzkpDQo+IA0KPiANCj4gRGVhciBvcGVubGRhcCB0ZWFtIOKAlA0KPiANCj4g
regenerating any indices. Looks like
something
we'll rewrite for 2.5 but leave 2.4 alone.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
We could reproduce this with 2.4.42, 2.4.44 and 2.4.48. We hope you can
> reproduce this as well to see what is happening here.
Yes, can reproduce this. The function that converts a component-wise time
into a timet is referencing time since 1970. The date in 1956 would yield a
negative timet valu
d: 326777
Unable to reproduce. Please test the patch and report back if the problem still
shows up.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
ful
> to support MAP_NOSYNC as documented at
> http://nixdoc.net/man-pages/FreeBSD/mmap.2.html
>
>
Added in mdb.master
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
93519db205=
966f7988c03c920f5
>>> Doc (see method description):=C2=A0http://www.lmdb.tech/doc/group__md=
b.html#gaa2506ec8dab3d969b0e609cd82e619e5
>>
>> Yes, this is known. The point is to force you to explicitly set a sane=
value, therefore this
>> will never be changed.
>>
>>
e va=
lue, therefore this
will never be changed.
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
h...@openldap.org wrote:
> Full_Name: Howard Chu
> Version: 0.9.24
> OS: Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (84.203.24.208)
> Submitted by: hyc
>
>
> There are some cases when renewing a cursor in a read-only txn that may return
&
Full_Name: Howard Chu
Version: 2.4
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (84.203.24.208)
Submitted by: hyc
During connection setup, there are a couple of places where we perform
operations on a socket and log an error if it fails, but otherwise keep going
der - are th=
ey just too stupid
and incompetent to implement their own spec, or have they broken things i=
ntentionally, to
prevent their clients from working with non-Microsoft servers...
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
rld-readable files.
Closing this ITS and all the other copies of it.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
imgur.com/2RbFkIY.png
>
> Regards,
> Philip
>
>
> [1] https://tools.ietf.org/html/rfc4511#section-4.1.11
> [2]
> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/3c5e87db-4728-4f29-b164-01dd7d7391ea
> [3]
> https://github.com/openldap/openldap/blob/master/servers/slapd/controls.c#L1717
> [4]
> https://github.com/openldap/openldap/blob/master/clients/tools/ldapsearch.c#L1154
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
thing to do is commit more often so they don't grow so large.
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
is why it just internally loops over one item at a time.
> What do you think? I'm not sure if OpenLdap would profit here, but I'm quite
> certain that LMDB as a general database would. I'm also open to contribute if
> you are willing to give some guidance.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
not be used again after
calling commit or abort.
There is nothing to clean up, the txn is always gone when commit or abort
returns.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
s is also env->me_txn0 !)
> -> calls mdb_txn_abort
...
> - abort the transaction (again) with mdb_abort()
This is a bug in your code, you can't call txn_abort twice. This is
already documented. Closing this ITS.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director,
@example.com
>> l: Sunnyvale
>> ou: Human Resources
>> ou: People
>> ===
>>
>> When running the following commands:
>>
>> ===
>> rm -f /tmp/*.mdb
>> slapadd -f slapd.conf puppet_init.ldif
>> slapadd -f slapd.conf -c backup.ldif
>> ===
slapadd -f slapd.conf -l puppet_init.ldif
slapadd -f slapd.conf -l backup.ldif -j 5
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
t snapshot (3be82f40d5cd4ca050e10859ecb961f28c807c41) with no
> particular config options
> * Using cn=config rather than slapd.conf
> * On a real production system, rather than the simplified version presented
> here.
> * Using another "special" attribute such as pwdAccountLockedTime instead of
> "contextCSN"
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
MDB and whether they used its
API correctly.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Howard Chu
Version: 2.4
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (84.203.26.174)
Submitted by: hyc
In back-mdb the entry reindexer operates in batches of 500 entries at a time.
(I.e., it commits one LMDB txn for every 500 entries.) But a final txn_commit
to synchronize native
> threads, or outright not being able to use both interfaces in a single
> process?
>
Passing a native structure to the posixthreads interface (and vice versa) will
crash the program (or
worse - silently corrupt memory without crashing the program)
--
-- Howard Chu
ted for any purpose with or
> without attribution and/or other notice.
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Howard Chu
Version: 0.9.24
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (80.233.39.186)
Submitted by: hyc
When reloading a printable dump (from mdb_dump -p), actual backslashes in the
input
(represented as '\\') will be corrupted. This has been broken since
buted for any purpose with or without
> attribution
> and/or other notice.
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
that the structure is corrupted. Working as designed,
in other words.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
uot; cargo afl build -v
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
oward, do any of those look familiar and potentially related?
>
> Victor
>
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
_SUCCESS) {
> if (val == MDB_MAP_FULL) {
> val = resize(env);
> continue;
> }
> printf("mdb_txn_commit %d\n", val);
> goto out;
> }
> }
> out:
> if (env) {
> mdb_dbi_close(env, dbi);
> mdb_env_close(env);
> }
> printf("OK\n");
> }
>
> After compiling I run the following command a couple of times:
> for i in {1..20}; do echo $i; (lmdb_test &); done
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
elpful.
Sorry, still not seeing this over here. What else do you know about the
systems where this is occurring? RAM size, storage on HDD / SSD / USB ?
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
n looking for a way to support the hashing callbacks. Without them,
this won't be accepted.
Easiest at this point is simply to define a character (maybe space, or tab)
as a delimiter between seed and password.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, H
19 17:07, Howard Chu wrote:
>> Fyi, we recently revisited this issue and are seeing improved results with
>> commit e90e8c7d3c12d897bb0584ba04dc519d4f23acf9 in master.
>>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun ht
ity checks
on known attributes. You still have to at least define the existence of these
attributes
on all participating servers.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
hlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
--AED2FFEE73F73B1613190B64
Content-Type: text/x-csrc;
name="mkalias.c"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="mkalias.c"
#include
#include
#incl
outputs the same LDIF,
with aliases interspersed, on stdout.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
--0F1DE22CD9273A65E1AE5118
Content-T
the IDL processing itself is the main bottleneck now. We
would
need to switch to bitmaps or trees to avoid this bottleneck, but that's also a
much larger change than we can consider for this release.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun
ntries and 3.7M aliases, a
search with -a always , starting from the
DB suffix, took 4 minutes without this patch, and 1235 minutes with this patch.
Needless to say, that's not looking good. Still checking other test cases.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
r, so if it is provided it is expected to be used.
>=20
> Sure, on idassert connections only, though. When does back-ldap use one=
?
> I want to edit the linked script to do exercise that so we have a decen=
t
> test for this now.
idassert is used when you want back-ldap to propagate the incoming client=
's
identity to the remote server. It affects every operation that a client i=
ssues.
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
change with ITS#8427, not sure what
> the documentation implies should happen in these cases, whether the new
> behaviour is inconsistent with it or you've been relying on incorrect
> behaviour that has since been corrected.
>
> Regards,
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
cDatabase=3D{1}m=
db.ldif'
>=20
>=20
> I.e., I was rather expecting:
>=20
> olcBackend=3D{1}mdb.ldif
>=20
> or similar.
No, because order is irrelevant for these.
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
is
executing, and so you get
your corruption. You have to copy the key if you want this particular code to
work. But even so,
you're not using the API optimally. Since you have a cursor already, you should
be using
mdb_cursor_put instead.
> }
> mdb_cursor_close(cursor);
> if (MDB_NOTFOUND != rc)
> return rc;
>
> X(mdb_txn_commit(txn));
> return 0;
> }
>
> int main(void)
> {
> X(setup());
> X(dump());
> X(change_data());
> X(dump());
> return 0;
> }
Closing this ITS.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
s actually
> returning a failure code when it doesn't find the DBI's root node).
>
> -myk
>
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
27 = 0x1b, so DB_STALE is true. mdb_page_search ought to have been invoked to
set the root of the cursor.
> mdb_page_search(mc, NULL, MDB_PS_ROOTONLY);
> }
> }
>
> From this we know that when mdb_cursor_put() is called, the following values
> rema
; Anybody has seen similar issue before?
Doesn't sound familiar. But 0.9.18 is quite old. If you can reproduce this
issue in 0.9.23 then we'll take a look. Test code to reproduce the problem
would also be needed.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director,
rest of the code still builds
on Windows XP.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
in the tarball linked below.
The code works as designed. No changes to mapsize handling were made between
0.9.22 and 0.9.23.
Closing this ITS.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLD
ymas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
>
>>>>>> Then the tests are broken, because these are not bugs.
>>>>>
>>>>> They are absolutely OpenLDAP bugs. The unaligned accesses are
>>>>> Undefined Behavior.
>>>>
>>>> Simply because the C standard doesn't specify what the behavior is doesn't
>>>> make it a bug.
>>>
>>> The C standard does specify the behavior. It falls clearly in
>>> Undefined Behavior.
>>
>> The fact a behavior is undefined does not make it illegal.
>
> A program with undefined behavior is, by definition, and illegal program.
This is pure nonsense. You realize, don't you, that the C spec has no definition
of threads? All multithreaded programs are using undefined behavior. And yet
here we are, using threads for over 2 decades.
The gcc/clang folks have their heads up their asses. They've deliberately
misinterpreted the spec claiming undefined behaviors are forbidden, supposedly
to enable essential optimizations. Most of which only apply to obscure corner
cases in compiler benchmark suites, that nobody in the real world ever benefits
from.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
e unaligned accesses are
>>> Undefined Behavior.
>>
>> Simply because the C standard doesn't specify what the behavior is doesn't
>> make it a bug.
>
> The C standard does specify the behavior. It falls clearly in
> Undefined Behavior.
The fact a behavior is undefined does not make it illegal.
> And Appendix J.2, Undefined Behavior, says:
>
> Conversion between two pointer types produces a result
> that is incorrectly aligned (6.3.2.3).
>
> You are violating the C standard in macros like COPY_PGNO. Effectively
> you are casting a uint8_t array to an uint16_t*. You cannot do that
> because uint16_t* increases the alignment requirements.
All internal fields of LMDB are (at least) 2-byte aligned, so these copies
always
meet alignment requirements.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
havior.
>>
>> Simply because the C standard doesn't specify what the behavior is doesn't
>> make it a bug.
>> It would be unreasonable to expect the standards body to exhaustively list
>> all of the
>> possible behaviors on all of the possible hardware architectures that the
>> language is
>> implemented for. Leaving it undefined in the spec only means they chose not
>> to specify
>> a behavior. That means the actual behavior depends on the underlying
>> machine. On the
>> x86 family the behavior is well defined.
>
> Also note: Behaviors are left undefined when they are non-portable. But being
> non-portable
> isn't illegal in C. The entire purpose of conditional compilation is to
> support non-portable
> code.
>
Worth reading
https://www.yodaiken.com/2018/12/31/undefined-behavior-and-the-purpose-of-c/
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
;>
>> They are absolutely OpenLDAP bugs. The unaligned accesses are
>> Undefined Behavior.
>
> Simply because the C standard doesn't specify what the behavior is doesn't
> make it a bug.
> It would be unreasonable to expect the standards body to exhaustively list
> all of the
> possible behaviors on all of the possible hardware architectures that the
> language is
> implemented for. Leaving it undefined in the spec only means they chose not
> to specify
> a behavior. That means the actual behavior depends on the underlying machine.
> On the
> x86 family the behavior is well defined.
Also note: Behaviors are left undefined when they are non-portable. But being
non-portable
isn't illegal in C. The entire purpose of conditional compilation is to support
non-portable
code.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
oblem for testing a QA folks.
>>
>> Then the tests are broken, because these are not bugs.
>
> They are absolutely OpenLDAP bugs. The unaligned accesses are
> Undefined Behavior.
Simply because the C standard doesn't specify what the behavior is doesn't make
it a bug.
It would be unreasonable to expect the standards body to exhaustively list all
of the
possible behaviors on all of the possible hardware architectures that the
language is
implemented for. Leaving it undefined in the spec only means they chose not to
specify
a behavior. That means the actual behavior depends on the underlying machine.
On the
x86 family the behavior is well defined.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
programs and libraries which use
> OpenLDAP. The OpenLDAP bugs have cross-pollinated into other programs
> and libraries. It is not simply contained or limited to OpenLDAP.
>
> That's a big problem for testing a QA folks.
Then the tests are broken, because these are not bugs.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
achine-defined. Since our code is
already
properly ifdef'd for the machines which do or don't support it, the fact that
this is
"non-portable" is not an issue.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
ed behavior is not a bug, nor is it prohibited by the C spec. It is a
necessary
part of the language for its intended use as a system programming language,
writing
machine-specific programs. Anyone who says it is prohibited by the spec is
wrong.
> Jeff
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
can also
use mdb_env_set_mapsize() to change the size of an existing env.
This is not a bug in LMDB, this ITS will be closed.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
e loss or gain coming from?
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
ion... else TLS start failing.=
> OpenLDAP has worked this way since I first started using it in 2002=
> . This behavior is nothing new. And this is the correct beh=
> avior.This ITS will be closed.--Quanah--> Quanah Gibson-MountProduct ArchitectSymas CorporationPackaged,=
> certified, and supported LDAP solutions powered by OpenLDAP: f=3D"http://www.symas.com; rel=3D"nofollow" target=3D"_blank">http://www.sy=
> mas.com
>
>
> --=_Part_545863_1662769086.1557520342175--
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
benefits are even larger,
> particularly in writemap mode, where there can be a 50-100x reduction in the
> system CPU usage by using this patch. This brings windows performance with
> sync'ed transactions in LMDB back into the range of "lightning" performance
> :).
What is t
the certificate files, it merely t=
> ells the TLS
> library where they are. You must likely have a broken TLS library.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
hK6noyLc%3Dreserved=3D=
0>=A0with
> container logs.
>=20
> I see no error here.
>=20
> Using your cert/key files:
> There is no OpenLDAP bug here. Your server environment is broken.
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun h
Siddharth Jain wrote:
> we have documented complete steps to repro the bug=A0here <https://gith=
ub.com/siddjain/openldap-bug>=A0with container logs.
I see no error here.
Using your cert/key files:
> ls -l /tmp/jnj
total 12
-rw-r--r-- 1 hyc hyc 1592 Apr 24 17:34 jnj-ca-chain.pem
ovider. It also generated ~60GB of logs at loglevel stats in 1 day.
The consumer should not be reconnecting more frequently than its retry config.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
and/or other notice.
Thanks, fix applied in git master.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
alls)
> is
> the legitimate intention of the code.
>
> The attached URL contains the fix as a patch (against the node-lmdb project).
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
d for production use.
Use a tagged release or the release engineering branch.
Closing this ITS.
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
st the actual fix against.
>
> Best regards
> Henrik
>
>
> --
> Henrik Bohnenkamp
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
sactions only affect write ops.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
1 - 100 of 1873 matches
Mail list logo
