from:"Ludwig Krispenz"

[389-devel] Re: failed to build rpms

2021-02-24 Thread Ludwig Krispenz


Hi Mark,

thanks for the answer, it took me a step further, but I still didn't succed.


Hi William,

yes please update the contributing docs.


Now to the remaining/new problems.

1/3] I think Mark's suggestion does help the build with rust, but the 
case where I didn't configure --enable-rust and failed in rust land 
shouldn't happen - either rust is an option and everything should work 
if disabled or it has to be on and not optional.



2/3] Before receiving Marks hint about updating dependencies I had tried 
to build optimized rpms, (to overcome the debug and release mismatch) 
but got this strange error.


 /usr/bin/mkdir -p 
'/home/elkris/DEV/gh/two/389-ds-base/rpmbuild/BUILDROOT/389-ds-base-2.0.3-20210224gitbda2c53d0.fc32.x86_64/usr/lib64/dirsrv'
 /bin/sh ./libtool   --mode=install /usr/bin/install -c libslapd.la 
libldaputil.la libns-dshttpd.la librewriters.la 
'/home/elkris/DEV/gh/two/389-ds-base/rpmbuild/BUILDROOT/389-ds-base-2.0.3-20210224gitbda2c53d0.fc32.x86_64/usr/lib64/dirsrv'
libtool:   error: error: cannot install 'libslapd.la' to a directory not 
ending in /opt/dirsrv/lib/dirsrv

make[3]: *** [Makefile:4163: install-serverLTLIBRARIES] Error 1


3/3] When using the make command Mark provided everything seemed fin, 
until it finally failed again:


 /usr/bin/mkdir -p 
'/home/elkris/DEV/gh/two/389-ds-base/rpmbuild/BUILDROOT/389-ds-base-2.0.3-20210224gitbda2c53d0.fc32.x86_64/usr/lib64/dirsrv'
 /bin/sh ./libtool   --mode=install /usr/bin/install -c libslapd.la 
libldaputil.la libns-dshttpd.la librewriters.la 
'/home/elkris/DEV/gh/two/389-ds-base/rpmbuild/BUILDROOT/389-ds-base-2.0.3-20210224gitbda2c53d0.fc32.x86_64/usr/lib64/dirsrv'
libtool:   error: error: cannot install 'libslapd.la' to a directory not 
ending in /opt/dirsrv/lib/dirsrv

make[3]: *** [Makefile:4163: install-serverLTLIBRARIES] Error 1



.. contents:: :depth: 2

FAIL: test_slapd


==2968306==ASan runtime does not come first in initial library list; you 
should either link runtime to your application or manually preload it 
with LD_PRELOAD.

FAIL test_slapd (exit status: 1)

+ false
error: Bad exit status from /var/tmp/rpm-tmp.BQqMLc (%check)


Any ideas are welcome,

Best regards,

Ludwig

On 24.02.21 00:35, William Brown wrote:

Should we update the contributing/build docs to reflect these steps?


On 24 Feb 2021, at 07:10, Mark Reynolds  wrote:

You have to skip the npm audit check "SKIP_AUDIT_CI=1" for now, and the rust 
stuff is a pain and it is hardcoded to be enabled. You always have to update and download 
the latest Rust dependencies:

SKIP_AUDIT_CI=1 make -f rpm.mk update-cargo-dependencies 
download-cargo-dependencies rpms

HTH,

Mark

On 2/23/21 1:40 PM, Ludwig Krispenz wrote:

Hi,

since a long time I was trying to build rpms and failed, here are the issues I 
run into:

1] problem with npm/audit

I followed the suggestions here: 
https://www.port389.org/docs/389ds/contributing.html (pushd/npm fix/popd), but 
this didn't help, only commenting out audit-ci in 
src/cockpit/389-console/node_modules.mk got me over this

2] rust

2.1]

rpm build failed with:

error: failed to get `concread` as a dependency of package `librslapd v0.1.0 
(/home/elkris/DEV/gh/two/389-ds-base/rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/src/librslapd)`

Caused by:
   failed to load source for dependency `concread`

Caused by:
   Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
   failed to update replaced source registry 
`https://github.com/rust-lang/crates.io-index`

Caused by:
   failed to read root of directory source: 
/home/elkris/DEV/gh/two/389-ds-base/rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/vendor

Caused by:
   No such file or directory (os error 2)
make[1]: *** [Makefile:12715: 
.../rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/rs/rslapd/release/librslapd.a]
 Error 101


,

It was right that there wasno rs/rslapd/release/librslapd.a file, not even the 
directory rs existed. After configuring --enable-rust the directory was created 
and populated.

Q1: why does it try to pack rust stuff if it is not enabled ?

2.2] Now the directory was there, but I still did get the same error. A closer look showed that it 
was looking for 
.../rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/rs/rslapd/release/librslapd.a ,but what 
existed was .../rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/rs/rslapd/debug/librslapd.a. 
Note "debug" -- "release". configure was run with --enable-debug

Q2: Is there somewhere a hardcoded/default assumpion of "release" ? in the 
cargo spec?

Thanks for any suggestions


Regards,

Ludwig
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Gu

[389-devel] failed to build rpms

2021-02-23 Thread Ludwig Krispenz


Hi,

since a long time I was trying to build rpms and failed, here are the 
issues I run into:


1] problem with npm/audit

I followed the suggestions here: 
https://www.port389.org/docs/389ds/contributing.html (pushd/npm 
fix/popd), but this didn't help, only commenting out audit-ci in 
src/cockpit/389-console/node_modules.mk got me over this


2] rust

2.1]

rpm build failed with:

error: failed to get `concread` as a dependency of package `librslapd 
v0.1.0 
(/home/elkris/DEV/gh/two/389-ds-base/rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/src/librslapd)`


Caused by:
  failed to load source for dependency `concread`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to update replaced source registry 
`https://github.com/rust-lang/crates.io-index`


Caused by:
  failed to read root of directory source: 
/home/elkris/DEV/gh/two/389-ds-base/rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/vendor


Caused by:
  No such file or directory (os error 2)
make[1]: *** [Makefile:12715: 
.../rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/rs/rslapd/release/librslapd.a] 
Error 101



,

It was right that there wasno rs/rslapd/release/librslapd.a file, not 
even the directory rs existed. After configuring --enable-rust the 
directory was created and populated.


Q1: why does it try to pack rust stuff if it is not enabled ?

2.2] Now the directory was there, but I still did get the same error. A 
closer look showed that it was looking for 
.../rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/rs/rslapd/release/librslapd.a 
,but what existed was 
.../rpmbuild/BUILD/389-ds-base-2.0.3.20210223gitbda2c53d0/rs/rslapd/debug/librslapd.a. 
Note "debug" -- "release". configure was run with --enable-debug


Q2: Is there somewhere a hardcoded/default assumpion of "release" ? in 
the cargo spec?


Thanks for any suggestions


Regards,

Ludwig
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

[389-devel] Re: Mapping tree rework

2020-10-18 Thread Ludwig Krispenz

On 19.10.20 01:26, William Brown wrote:

On 16 Oct 2020, at 17:48, Pierre Rogier wrote:

Hi William,
I agree with your architecture points and that is why I said my proposal is a
less appealing trade off.

My real concern is your last point:
we just do not know and IMHO we are unable to predict what (or if) config
will cause problems, and I am afraid we will only discover it when people start
to complain.
So I still think that the benefit/risk ratio is bad)

I think this wasn't my point. The thing is *any* change will have that "unknown" risk.
Our job is to qualify and identify as many of those risks as we can, to remove them as unknowns.
Think about the work recently to merge the changelog to the main db, or BDB to LMDB work, even
changing from perl to python for installation. These are all significantly larger changes, which
would be "much riskier" but all of them have been managed effectively by the team
communicating, coordinating, analysing, designing and testing changes.

So I really don't accept this "unknown" risk argument. I have laid out a design
that explores the configuration, how it works today and how the values are currently
trusted, and a process to manage and understand this change in a way to minimise the
risk. There are associated tests, and it passes with address sanitiser, and other test
cases for mapping trees, replication and others.

If we just say "unknown risk" at every change we make we'd never progress. We
may as well packup and go home, the project is completed.

if you put it that way any change is justified because it is a change.
Changes are necessary to achieve something, eg features performance (and
I would distinguish changes from fixes).

This started, as you said yourself, because:

>>>

This has come up because there is a set of customer cases where they have
configured it incorrectly, due to bugs in lib389. The issues in lib389 arise
from a lack of validation/constraint in the checking of the
nsslapd-parent-suffix value in the server, allowing the client to create
invalid configurations.

So today, our own tools can easily, and trivially cause this situation.

<<<

So we have situation where the design has flaws, but in effect was
"working" and the we messed up ourselves by providing tools which can
easily break things. And here I would say it is justified to discuss the
balance of fixing the tools and eventually adding some checks to the
server vs reimplementing it with the risk that the design,
implementation and new tooling will als have challenges.

Ludwig

So I still stand by my design and the PR I have submitted in this case, and if
there are concerns about esoteric configurations, then we should identify and
understand them too beyond the testing I have already provided.

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs, Australia
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Mapping tree rework

2020-10-15 Thread Ludwig Krispenz

Hi Pierre,

you expressed my concerns much clearer. I agree with you,

Thanks,

Ludwig

On 15.10.20 13:00, Pierre Rogier wrote:

Hi,

For my part, I have seen mapping tree misconfiguration popping from 
time to time but it is quite rare  (maybe 7 or 8 times in 15 years)
So although in pure term or architecture your proposal is IMHO the 
cleanest,  in term of risks it is not a good solution:
 there will likely be regressions (it always happens when a component 
get redesigned)
  and that means that the number of people that will be annoyed by the 
change will be far greater than the few people that will be helped by 
the change.

So my feeling is that we should rather go for a trade off.
 Since the goal is to prevent that user misconfigures the mapping tree 
without warning,
  we should do that without changing the way the mapping tree is 
handled internally.
  simply by adding a consistency check when starting the server or 
changing the mapping tree configuration.

If we want to limit the risk we could phase things:
  in first phase we only log warning if inconsistency are found
  and latter on when we get more confident about the code we could 
reject the configuration change in case of inconsistency

I know that my proposal is less appealing in term of architecture but 
such a solution is safer because it does not change the way mapping 
tree are handled and that drastically limits the regression risks

 Regards,
     Pierre

On Wed, Oct 14, 2020 at 11:23 PM William Brown <mailto:wbr...@suse.de>> wrote:

This has come up because there is a set of customer cases where
they have configured it incorrectly, due to bugs in lib389. The
issues in lib389 arise from a lack of validation/constraint in the
checking of the nsslapd-parent-suffix value in the server,
allowing the client to create invalid configurations.

So today, our own tools can easily, and trivially cause this
situation.

One thought is to either document this issue or to fix lib389 -
but neither of the actions really fix the underlying problem,
which is that our server accepts an invalid configuration silently.

So the best thing for us to do is to make it impossible for the
server to get it wrong, which means we fix lib389 *and* any other
admin tooling/scripts in a single pass.

Which is what led to the interest in changing something that has
been "working" for a long time :)

> On 14 Oct 2020, at 19:47, Ludwig Krispenz mailto:krisp...@t-online.de>> wrote:
>
> Hi,
>
> you are right that it is possible to configure suffix
hierarchies which are broken, but in my experience this wasn't an
issue. people using sub suffixes did get it right.
>
> So is there really a need to change something that is working
for a long time ?
>
>
> Regards,
>
> Ludwig
>
>
> On 14.10.20 08:12, William Brown wrote:
>> https://github.com/mreynolds389/389wiki/pull/48
>>
>> This is a draft design, and probably of interest to thierry
whom I discussed this with last night :)
>>
>> Thanks!
>>
>> —
>> Sincerely,
>>
>> William Brown
>>
>> Senior Software Engineer, 389 Directory Server
>> SUSE Labs, Australia
>> ___
>> 389-devel mailing list -- 389-devel@lists.fedoraproject.org
<mailto:389-devel@lists.fedoraproject.org>
>> To unsubscribe send an email to
389-devel-le...@lists.fedoraproject.org
<mailto:389-devel-le...@lists.fedoraproject.org>
>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:

https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org
> ___
> 389-devel mailing list -- 389-devel@lists.fedoraproject.org
<mailto:389-devel@lists.fedoraproject.org>
> To unsubscribe send an email to
389-devel-le...@lists.fedoraproject.org
<mailto:389-devel-le...@lists.fedoraproject.org>
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:

https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs, Australia
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
<mailto:389-devel@lists.fedoraproject.org>
To unsubsc

[389-devel] Re: Mapping tree rework

2020-10-14 Thread Ludwig Krispenz


Hi,

you are right that it is possible to configure suffix hierarchies which 
are broken, but in my experience this wasn't an issue. people using sub 
suffixes did get it right.


So is there really a need to change something that is working for a long 
time ?



Regards,

Ludwig


On 14.10.20 08:12, William Brown wrote:

https://github.com/mreynolds389/389wiki/pull/48

This is a draft design, and probably of interest to thierry whom I discussed 
this with last night :)

Thanks!

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs, Australia
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Odd behaviour in import

2020-08-31 Thread Ludwig Krispenz



On 31.08.20 02:33, William Brown wrote:



On 28 Aug 2020, at 19:23, Ludwig Krispenz  wrote:


On 27.08.20 04:01, William Brown wrote:

Hey there,

I'm seeing some odd behaviour in an import test. I'm seeing that a large number 
of entries won't import unless the directory is restarted before the import 
task is performed.

The error appears to be:

[25/Aug/2020:14:14:58.973490600 +1000] - WARN - import_foreman - import userRoot: Skipping entry 
"cn=group0,ou=Groups,dc=example,dc=com" which has no parent, ending at line 154 of file 
"/opt/dirsrv/var/lib/dirsrv/slapd-standalone1/ldif/4f8afb8d-ec97-4246-94a2-ec343c0eacb4.ldif"
...
[25/Aug/2020:14:14:59.307477400 +1000] - INFO - bdb_import_main - import 
userRoot: Import complete.  Processed 14 entries (10 were skipped) in 1 
seconds. (14.00 entries/sec)


This is where a newly created backend *with* example entries, then has it's 
entire content overwriten during an import. Anything that is underneath the 
ou=* entries is not imported, but the ou= and dc=are fine.

I'm wondering if this is something related to the fact we are replacing the ou= 
entries with different ids/nsunique ids. IE

id 3
rdn: ou=groups
objectClass: top
objectClass: organizationalunit
ou: groups
aci: (targetattr="cn || member || gidNumber || nsUniqueId || 
description || ob
 jectClass")(targetfilter="(objectClass=groupOfNames)")(version 3.0; acl 
"Enab
 le anyone group read"; allow (read, search, 
compare)(userdn="ldap:///anyone;)
 ;)
aci: 
(targetattr="member")(targetfilter="(objectClass=groupOfNames)")(version
 3.0; acl "Enable group_modify to alter members"; allow 
(write)(groupdn="ldap:
 ///cn=group_modify,ou=permissions,dc=example,dc=com");)
aci: (targetattr="cn || member || gidNumber || description || 
objectClass")(ta
 rgetfilter="(objectClass=groupOfNames)")(version 3.0; acl "Enable 
group_admin
  to manage groups"; allow (write, add, 
delete)(groupdn="ldap:///cn=group_admi
 n,ou=permissions,dc=example,dc=com");)
creatorsName: cn=directory manager
modifiersName: cn=directory manager
createTimestamp: 20200827015033Z
modifyTimestamp: 20200827015033Z
nsUniqueId: b0fce42b-e80711ea-8141c872-2df18128
parentid: 1
entryid: 3
numSubordinates: 1

Becomes:

id 4
rdn: ou=Groups
createTimestamp: 20200224023755Z
creatorsName: cn=Manager,dc=example,dc=com
entryUUID: 67cc2212-eafa-1039-8830-152569770969
modifiersName: cn=Manager,dc=example,dc=com
modifyTimestamp: 20200224023755Z
objectClass: organizationalUnit
objectClass: top
ou: Groups
nsUniqueId: 87b64988-e68911ea-a943c898-6d74ab17
parentid: 1
entryid: 4


Given that these id's are changing I'm wondering if this is somehow breaking 
our import ordering? Any ideas on where I should start to investigate this?

shouldn't the nsuniqueid be preserved ? Otherwise you could not use import to 
initilaize a replica.

The use case that's happening is that after a backend is setup with sample 
entries, then you try to restore from a backup or ldif of different origin. So 
the nsunique and entryid's both could and probably will change in this case.
yes, but what Imean is that if the entry in the ldif contains a 
nsuniqueid it should be used




Thanks!

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fe

[389-devel] Re: Odd behaviour in import

2020-08-28 Thread Ludwig Krispenz



On 27.08.20 04:01, William Brown wrote:

Hey there,

I'm seeing some odd behaviour in an import test. I'm seeing that a large number 
of entries won't import unless the directory is restarted before the import 
task is performed.

The error appears to be:

[25/Aug/2020:14:14:58.973490600 +1000] - WARN - import_foreman - import userRoot: Skipping entry 
"cn=group0,ou=Groups,dc=example,dc=com" which has no parent, ending at line 154 of file 
"/opt/dirsrv/var/lib/dirsrv/slapd-standalone1/ldif/4f8afb8d-ec97-4246-94a2-ec343c0eacb4.ldif"
...
[25/Aug/2020:14:14:59.307477400 +1000] - INFO - bdb_import_main - import 
userRoot: Import complete.  Processed 14 entries (10 were skipped) in 1 
seconds. (14.00 entries/sec)


This is where a newly created backend *with* example entries, then has it's 
entire content overwriten during an import. Anything that is underneath the 
ou=* entries is not imported, but the ou= and dc=are fine.

I'm wondering if this is something related to the fact we are replacing the ou= 
entries with different ids/nsunique ids. IE

id 3
rdn: ou=groups
objectClass: top
objectClass: organizationalunit
ou: groups
aci: (targetattr="cn || member || gidNumber || nsUniqueId || 
description || ob
 jectClass")(targetfilter="(objectClass=groupOfNames)")(version 3.0; acl 
"Enab
 le anyone group read"; allow (read, search, 
compare)(userdn="ldap:///anyone;)
 ;)
aci: 
(targetattr="member")(targetfilter="(objectClass=groupOfNames)")(version
 3.0; acl "Enable group_modify to alter members"; allow 
(write)(groupdn="ldap:
 ///cn=group_modify,ou=permissions,dc=example,dc=com");)
aci: (targetattr="cn || member || gidNumber || description || 
objectClass")(ta
 rgetfilter="(objectClass=groupOfNames)")(version 3.0; acl "Enable 
group_admin
  to manage groups"; allow (write, add, 
delete)(groupdn="ldap:///cn=group_admi
 n,ou=permissions,dc=example,dc=com");)
creatorsName: cn=directory manager
modifiersName: cn=directory manager
createTimestamp: 20200827015033Z
modifyTimestamp: 20200827015033Z
nsUniqueId: b0fce42b-e80711ea-8141c872-2df18128
parentid: 1
entryid: 3
numSubordinates: 1

Becomes:

id 4
rdn: ou=Groups
createTimestamp: 20200224023755Z
creatorsName: cn=Manager,dc=example,dc=com
entryUUID: 67cc2212-eafa-1039-8830-152569770969
modifiersName: cn=Manager,dc=example,dc=com
modifyTimestamp: 20200224023755Z
objectClass: organizationalUnit
objectClass: top
ou: Groups
nsUniqueId: 87b64988-e68911ea-a943c898-6d74ab17
parentid: 1
entryid: 4


Given that these id's are changing I'm wondering if this is somehow breaking 
our import ordering? Any ideas on where I should start to investigate this?
shouldn't the nsuniqueid be preserved ? Otherwise you could not use 
import to initilaize a replica.


Thanks!

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Please have a look at rewriters design

2020-04-01 Thread Ludwig Krispenz

Ok, so Thierry's solution is useful to make using rewriters simpler, but
I really want to have its use and interface documented somewhere
outside the code, PR, or design doc on the 389ds wiki - it needs to go
to the official doc eg plugin guide.

Regards,
Ludwig

On 04/01/2020 01:02 AM, William Brown wrote:

On 1 Apr 2020, at 01:04, Ludwig Krispenz wrote:

Hi,

I was away and am late in the discussion, maybe too late.

Not too late, it's not released in production yet ;). There are two PR's that
have been discussed here:

https://pagure.io/389-ds-base/pull-request/50988

https://pagure.io/389-ds-base/pull-request/50981

In my understanding what you mean by "generic" is that for a new rewriter you do not need a
plugin, but to provide some rewrite functions and specify them in a rewriters config entry. But there is
still the need to write rewriter functions, compile and deploy them, and instead of plugins you now
have a new interface of "filterRewriter" and "returendAttrRewriter functions - so far not
documented anywhere.

Under generic rewriter I would understand an approach where you do not need to
provide own functions, but have a rewriter plugin, which does rewriting based
on rules in rewrite config entries, eg in which subtree, for which entries
(filter to select), how to map a saerch filter, how to rename attrs on
return,

I had the same feeling too, to have these statically in libslapd, and much
simpler than resolving symbols and dlopen. However, it's looking more like it
will be a plugin style, but without using the current slapi plugin architecture
- just a symload at start up. The reason for this that thierry explained is
that freeipa plans to link to samba or sssd as part of one of the rewriter
classes, and we don't want that to become a dependency of 389-ds.

I have argued in the past for a "lib-ipa" that has the needed shared logic
between various pieces of the project, but honestly, I forgot if that ever happened. I
think these days sssd is libipa in a lot of ways ...

Anyway, that's why Thierry want's to have a symload in this case :)

Best regards,
Ludwig

On 03/19/2020 01:09 AM, William Brown wrote:

On 19 Mar 2020, at 04:08, thierry bordaz wrote:

On 3/18/20 1:51 AM, William Brown wrote:

On 18 Mar 2020, at 04:08, thierry bordaz wrote:

Hi William,

I updated the design according to our offline exchange

Thanks Thierry, I appreciate the conversation and the updates to the document:
it made clear there were extra details up in your brain but not in words yet :)
it's always hard to remember all the details as we write things, so thanks for
the discussion. Like you said, it's always good to have a team who is really
invested and cares about the work we do!

Your design for the core server version looks much better! Thank you. I still
think there are some missing points. The reason to have a libpath rather than
inbuild is to avoid a potential linking to sssd/samba. I think also that the
problem space of the global catalog here needs to be looked at too. This
feature is not in isolation, it's really a part of that.

Okay, I will work on a new PR making core server able to retrieve/registers
rewriters.

I think the "need" to improve the usability of rewriters is not specific to
global catalog. Global Catalog is just an opportunity to implement it. I think parts of
slapi-nis, integration of vsphere, GC (and likely others) are also use case for
rewriters. They were implemented in different ways because rewriters were not easy to use
or simply not known.

Yes, that's perfectly reasonable, and shouldn't stop your idea from being
created - what's concerning me is that without a full picture you don't know
how far to take these rewriters or what direction, or what might be needed.

This means we have a whole set of deployment cases to look at.

So the deployment will look like:

IPA DS --> IPA GC

So an ipaAccount from the IPA DS instance will be "copied and transformed" into
the IPA GC. This process is as yet undefined (it sounds like it may be offline or
something else ...). We are simply not dealing with one instance now, but an out-of-band
replication and transformation process. It's unclear whether the data transform is during
this loading process, or in the IPA GC somehow.

From what I understand, it sounds like a method to take an ipaAccount and
transform it to an AD GC account stub. Then inside of that IPA GC there are
some virtual attributes you wish to add like objectSid binary vs string
representations, objectCategory, maybe others.

So from our discussion, we have currently focused on "how do we transform entries
within a single directory server". But that's not the problem here. We are saying:

"We take an entry from IPA DS, transform it to an IPA GC stub entry, and then apply a set of
further "in memory" transformations"

One of the

[389-devel] Re: Please have a look at rewriters design

2020-03-31 Thread Ludwig Krispenz

Hi,

I was away and am late in the discussion, maybe too late.

In my understanding what you mean by "generic" is that for a new
rewriter you do not need a plugin, but to provide some rewrite functions
and specify them in a rewriters config entry. But there is still the
need to write rewriter functions, compile and deploy them, and instead
of plugins you now have a new interface of "filterRewriter" and
"returendAttrRewriter functions - so far not documented anywhere.

Under generic rewriter I would understand an approach where you do not
need to provide own functions, but have a rewriter plugin, which does
rewriting based on rules in rewrite config entries, eg in which subtree,
for which entries (filter to select), how to map a saerch filter, how to
rename attrs on return,

Best regards,
Ludwig

On 03/19/2020 01:09 AM, William Brown wrote:

On 19 Mar 2020, at 04:08, thierry bordaz wrote:

On 3/18/20 1:51 AM, William Brown wrote:

On 18 Mar 2020, at 04:08, thierry bordaz wrote:

Hi William,

I updated the design according to our offline exchange

Okay, I will work on a new PR making core server able to retrieve/registers
rewriters.

This means we have a whole set of deployment cases to look at.

So the deployment will look like:

IPA DS --> IPA GC

So from our discussion, we have currently focused on "how do we transform entries
within a single directory server". But that's not the problem here. We are saying:

"We take an entry from IPA DS, transform it to an IPA GC stub entry, and then apply a set of
further "in memory" transformations"

One of the biggest issue with GC is schema. IPA DS and IPA GC have not
compatible schema. They can not be in the same replication topology.
So provisioning of IPA GC requires transformations rules to present an other
"view" of IPA DS data. Those transformations will be on the write path (i.e.
stored in DB/indexed). This transformation work is almost done and is completely
independent of 389-ds.
All of this is "write" path: provisioning (online or offline) and
transformation.

The problem for IPA GC is now on the "read" path. AD clients are use to smart
shortcuts/control that are supported by IPA GC.
This is the IPA GC instance that will register the rewriters to act as GC does.

Yep, I'm aware :)

If that's the process, why not do all the transforms as required in the DS -> GC
load process? You raised a critically key point - we have a concern about the write
path as the transform point due to IO or time to do the transform, but it sounds like
you have to do this anyway as an element of the DS -> GC process.

Some of the transformation rules, on the write path, are quite complex. Looking
at slapi-nis config entries gives an idea what is needed. In addition to those
transformations, DS to GC online provisioning is not simple at all. Relying on
sync-repl, you then need to transform a received entry into an update. At the
moment it is an offline provisioning via transformation and

[389-devel] Re: Thoughts on swapping to rfc2307bis.ldif by default

2020-03-03 Thread Ludwig Krispenz


Hi,

I have no expertise in this area, but would like to get also Alexander's 
opinion and view from IPA


Regards,
Ludwig

On 03/03/2020 10:17 AM, thierry bordaz wrote:



On 3/3/20 4:12 AM, William Brown wrote:



On 3 Mar 2020, at 11:18, William Brown  wrote:




On 3 Mar 2020, at 04:32, thierry bordaz  wrote:



On 3/2/20 7:24 AM, William Brown wrote:

Hi all,

As you may know, I'm currently working on a migration utility to 
help move from other ldap servers to 389-ds. Something that I have 
noticed in this process is that other servers default to 
rfc2307bis.ldif [0] by default. As part of the migration I would 
like to handle this situation a bit better. It's likely not viable 
for me to simply plaster rfc2307bis into 99user.ldif as part of 
the migration process, so I want to approach this better.


rfc2307 and rfc2307bis are incompatible schemas that redefine the 
same OIDs with new/different meanings. Some key examples:


* posixGroup in rfc2307 only requires gidNumber, rfc2307bis 
requires cn and gidNumber.

Is not it the opposite ?

I was reading the schema as I was reading this.
I need to apologise for being so short in this answer! Thierry was 
correct in this case.


Here is the full set of differences between the two:

uidNumber: +EQUALITY integerMatch
gidNumber: +EQUALITY integerMatch
gecos: +EQUALITY caseIgnoreIA5Match SUBSTR 
caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 
-SYNTAX 1.3.6.1.4.1.1466.115.121.1.15

homeDirectory: +EQUALITY caseExactIA5Match
loginShell: +EQUALITY caseExactIA5Match
shadowLastChange: +EQUALITY integerMatch
shadowMin: +EQUALITY integerMatch
shadowMax: +EQUALITY integerMatch
shadowWarning: +EQUALITY integerMatch
shadowInactive: +EQUALITY integerMatch
shadowExpire: +EQUALITY integerMatch
shadowFlag: +EQUALITY integerMatch
memberUid: +EQUALITY caseExactIA5Match
memberNisNetgroup: +EQUALITY caseExactIA5Match SUBSTR 
caseExactIA5SubstringsMatch

nisNetgroupTriple: +EQUALITY caseIgnoreIA5Match
ipServicePort: +EQUALITY integerMatch
ipServiceProtocol: +SUP name -SYNTAX 1.3.6.1.4.1.1466.115.121.1.15
ipProtocolNumber: +EQUALITY integerMatch
oncRpcNumber: +EQUALITY integerMatch
ipHostNumber: +SUP name -SYNTAX 1.3.6.1.4.1.1466.115.121.1.15
ipNetworkNumber: +SUP name -SYNTAX 1.3.6.1.4.1.1466.115.121.1.15
ipNetmaskNumber: +EQUALITY caseIgnoreIA5Match SYNTAX 
1.3.6.1.4.1.1466.115.121.1.26 -SYNTAX 1.3.6.1.4.1.1466.115.121.1.15
macAddress: +EQUALITY caseIgnoreIA5Match SYNTAX 
1.3.6.1.4.1.1466.115.121.1.26 -SYNTAX 1.3.6.1.4.1.1466.115.121.1.15

bootParameter: +EQUALITY caseExactIA5Match
nisMapName: +SUP name -SYNTAX 1.3.6.1.4.1.1466.115.121.1.15
nisMapEntry: +EQUALITY caseExactIA5Match SUBSTR 
caseExactIA5SubstringsMatch


+ attributeTypes: ( 1.3.6.1.1.1.1.28 NAME 'nisPublicKey' DESC 'NIS 
public key' EQUALITY octetStringMatch SYNTAX 
1.3.6.1.4.1.1466.115.121.1.40 SINGLE-VALUE )
+ attributeTypes: ( 1.3.6.1.1.1.1.29 NAME 'nisSecretKey' DESC 'NIS 
secret key' EQUALITY octetStringMatch SYNTAX 
1.3.6.1.4.1.1466.115.121.1.40 SINGLE-VALUE )
+ attributeTypes: ( 1.3.6.1.1.1.1.30 NAME 'nisDomain' DESC 'NIS 
domain' EQUALITY caseIgnoreIA5Match SYNTAX 
1.3.6.1.4.1.1466.115.121.1.26 )
+ attributeTypes: ( 1.3.6.1.1.1.1.31 NAME 'automountMapName' DESC 
'automount Map Name' EQUALITY caseExactIA5Match SUBSTR 
caseExactIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 
SINGLE-VALUE )
+ attributeTypes: ( 1.3.6.1.1.1.1.32 NAME 'automountKey' DESC 
'Automount Key value' EQUALITY caseExactIA5Match SUBSTR 
caseExactIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 
SINGLE-VALUE )
+ attributeTypes: ( 1.3.6.1.1.1.1.33 NAME 'automountInformation' DESC 
'Automount information' EQUALITY caseExactIA5Match SUBSTR 
caseExactIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 
SINGLE-VALUE )


posixAccount:
shadowAccount:
posixGroup: +AUXILLARY -MUST cn STRUCTURAL
ipService:
ipProtocol:
oncRpc:
ipHost: - MAY o $ ou $ owner $ seeAlso $ serialNumber +MAY userPassword
ipNetwork: -MUST cn +MAY cn
nisNetgroup:
nisMap:
nisObject:
ieee802Device: -MUST cn MAY description $ l $ o $ ou $ owner $ 
seeAlso $ serialNumber
bootableDevice: -MUST cn MAY description $ l $ o $ ou $ owner $ 
seeAlso $ serialNumber

nisMap: +OID 1.3.6.1.1.1.2.9 -OID 1.3.6.1.1.1.2.13

+ objectClasses: ( 1.3.6.1.1.1.2.14 NAME 'nisKeyObject' SUP top 
AUXILIARY DESC 'An object with a public and secret key' MUST ( cn $ 
nisPublicKey $ nisSecretKey ) MAY ( uidNumber $ description ) )
+ objectClasses: ( 1.3.6.1.1.1.2.15 NAME 'nisDomainObject' SUP top 
AUXILIARY DESC 'Associates a NIS domain with a naming context' MUST 
nisDomain )
+ objectClasses: ( 1.3.6.1.1.1.2.16 NAME 'automountMap' SUP top 
STRUCTURAL MUST ( automountMapName ) MAY description )
+ objectClasses: ( 1.3.6.1.1.1.2.17 NAME 'automount' SUP top 
STRUCTURAL DESC 'Automount information' MUST ( automountKey $ 
automountInformation ) MAY description ) ## namedObject is needed for 
groups without members
+ objectClasses: ( 1.3.6.1.4.1.5322.13.1.1 NAME

[389-devel] Re: [PATCH] prevent slapd from hanging under unlikely circumstances

2020-02-06 Thread Ludwig Krispenz



On 02/06/2020 12:57 AM, William Brown wrote:



On 5 Feb 2020, at 20:08, Ludwig Krispenz  wrote:


On 02/05/2020 10:53 AM, thierry bordaz wrote:


On 2/5/20 2:30 AM, William Brown wrote:

On 5 Feb 2020, at 03:10, Ludwig Krispenz  wrote:

I think I can agree with 1-8, 9 is one solution to fix the problem you 
reported, but not yet validate that there are no other side effects, there are 
potential postop plugins which should NOT be called for tombstone delete, eg 
retro cl, we need to investigate side effects of the patch and evaluate other 
solutions, I suggested one in anotehr reply.

for 10, the claim that not crying hurrah and merging a patch without further 
investigation and testing is "doctrinal objection" is very strong and I have to 
reject this.

Regards,
Ludwig

On 02/04/2020 05:00 PM, Jay Fenlason wrote:

Ok, let's take this from the top:

1: Defects that cause a server to become unresponsive are bad and must
be repaired as soon as possible.

I'm not objecting to this.


2: Some #1 class defects are exploitable and require a CVE.  As far as
I can tell, this one does not, but you should keep an eye out for the
possibility.

3: The #1 class defect I have found can be triggered in at least two
ways.  One requires ipa-replica-install and is hard to reproduce in a
test environment.  The other requires ldapdelete and is easy to
reproduce in an isolated VM.

Ipa replica install and ldapdelete of a tombstone/conflict both require 
cn=directory manager, which would automatically cause any reported CVE to be 
void - cn=directory manager is game over.


4: The defect is a mismatch between the plugin ABI as implemented by
389-ds, and the behavior the NIS plugin expects.

5: I have found no specification or documentation on said ABI, so I
can only guess what the correct behavior is here.

6: The ABI includes two functions in the plugin: pre_delete and
post_delete.

7: The NIS plugin expects that every call to pre_delete will be
followed by a call to post_delete.  It takes a lock in pre_delete and
releases it in post_delete.

But you didn't report an issue in NIS? You reported an issue with ldapdelete on 
tombstones and conflicts ... the slapi-nis plugin is maintained by freeipa and 
if an issue in it exists, please report it to them with reproduction steps.

Hi Jay,

Thanks for your great investigations/patch/summary.

I made the change in NIS plugin to acquire/release map lock in pre/post ops. At 
that time I was assuming the pre/post callback were balanced.
I was wrong and already hit similar issue in #1694263. I updated NIS plugin to 
workaround that bug.

The new bug you are reporting is a new example that pre/post are sometime not 
balanced :(
The proposed fixes (prevent preop call on tombstone, force postop call even on 
tombstone) looks valid but will have wider impact and need additional 
investigation.
An other approach would be to make NIS plugin ignore operations on tombstone. 
It also require additional investigation.

I think that for tombstone deletions we should also prevent the call of the 
preop plugins, no plugin should deal with these operations

That sounds reasonable. Can we create an issue for this?
In fact discussing this yesterday we did get a step further. The call 
for preop plugins is already prevented for tombstone deletions, BUT


the detection if it is a delete of a tombstone flag comes from two 
sources, an operation flag, passed by the internal tombstone purging 
thread and by checking the entry flag of the entry to be deleted, which 
would be used if an external operation deletes a tombstone. 
Unfortunately this check is after the call for preops, so for external 
deletes the preops are called and the postops are not called.  Thierry 
and I agreed that the solution should be to do the check of the entry 
flag earlier and use the already existing bypass for pre and post 
plugins when a tombstone is deleted.


And yes, we need an issue for this.



best regards
thierry



8: Under some circumstances 389-ds can call pre_delete but fail to
call post_delete.  Because of #5, there is no way to tell if this is
expected behavior, but the NIS plugin clearly does not expect it.

9: My patch ensures that every call to pre_delete is followed by a
corresponding call to post_delete.  V1 of the patch also ensures that
every call to post_delete is after a corresponding pre_delete call.
V2 relaxes the second requirement, allowing post_delete calls without
a corresponding pre_delete call because someone expressed worry about
potential regressions.

10: You are refusing to merge my patch because of a doctrinal
objection to the use of ldapdelete in the simple reproducer.

It's not really a doctrinal objection. Replication is a really complex topic, 
and difficult to get correct. Ludwig knows a lot in this area, and has really 
good comments to make on the topic. I understand that you have an issue you 
have found in your setup, and you want to resolve it, but we have t

[389-devel] Re: [PATCH] prevent slapd from hanging under unlikely circumstances

2020-02-05 Thread Ludwig Krispenz



On 02/05/2020 10:53 AM, thierry bordaz wrote:



On 2/5/20 2:30 AM, William Brown wrote:



On 5 Feb 2020, at 03:10, Ludwig Krispenz  wrote:

I think I can agree with 1-8, 9 is one solution to fix the problem 
you reported, but not yet validate that there are no other side 
effects, there are potential postop plugins which should NOT be 
called for tombstone delete, eg retro cl, we need to investigate 
side effects of the patch and evaluate other solutions, I suggested 
one in anotehr reply.


for 10, the claim that not crying hurrah and merging a patch without 
further investigation and testing is "doctrinal objection" is very 
strong and I have to reject this.


Regards,
Ludwig

On 02/04/2020 05:00 PM, Jay Fenlason wrote:

Ok, let's take this from the top:

1: Defects that cause a server to become unresponsive are bad and must
be repaired as soon as possible.

I'm not objecting to this.


2: Some #1 class defects are exploitable and require a CVE.  As far as
I can tell, this one does not, but you should keep an eye out for the
possibility.

3: The #1 class defect I have found can be triggered in at least two
ways.  One requires ipa-replica-install and is hard to reproduce in a
test environment.  The other requires ldapdelete and is easy to
reproduce in an isolated VM.
Ipa replica install and ldapdelete of a tombstone/conflict both 
require cn=directory manager, which would automatically cause any 
reported CVE to be void - cn=directory manager is game over.



4: The defect is a mismatch between the plugin ABI as implemented by
389-ds, and the behavior the NIS plugin expects.

5: I have found no specification or documentation on said ABI, so I
can only guess what the correct behavior is here.

6: The ABI includes two functions in the plugin: pre_delete and
post_delete.

7: The NIS plugin expects that every call to pre_delete will be
followed by a call to post_delete.  It takes a lock in pre_delete and
releases it in post_delete.
But you didn't report an issue in NIS? You reported an issue with 
ldapdelete on tombstones and conflicts ... the slapi-nis plugin is 
maintained by freeipa and if an issue in it exists, please report it 
to them with reproduction steps.


Hi Jay,

Thanks for your great investigations/patch/summary.

I made the change in NIS plugin to acquire/release map lock in 
pre/post ops. At that time I was assuming the pre/post callback were 
balanced.
I was wrong and already hit similar issue in #1694263. I updated NIS 
plugin to workaround that bug.


The new bug you are reporting is a new example that pre/post are 
sometime not balanced :(
The proposed fixes (prevent preop call on tombstone, force postop call 
even on tombstone) looks valid but will have wider impact and need 
additional investigation.
An other approach would be to make NIS plugin ignore operations on 
tombstone. It also require additional investigation.
I think that for tombstone deletions we should also prevent the call of 
the preop plugins, no plugin should deal with these operations


best regards
thierry



8: Under some circumstances 389-ds can call pre_delete but fail to
call post_delete.  Because of #5, there is no way to tell if this is
expected behavior, but the NIS plugin clearly does not expect it.

9: My patch ensures that every call to pre_delete is followed by a
corresponding call to post_delete.  V1 of the patch also ensures that
every call to post_delete is after a corresponding pre_delete call.
V2 relaxes the second requirement, allowing post_delete calls without
a corresponding pre_delete call because someone expressed worry about
potential regressions.

10: You are refusing to merge my patch because of a doctrinal
objection to the use of ldapdelete in the simple reproducer.
It's not really a doctrinal objection. Replication is a really 
complex topic, and difficult to get correct. Ludwig knows a lot in 
this area, and has really good comments to make on the topic. I 
understand that you have an issue you have found in your setup, and 
you want to resolve it, but we have to consider the deployments of 
many thousands of other users and their replication experience too. 
For us it's important to be able to reproduce, and see the issue, to 
understand the consequences, and to make these changes carefully. 
Accepting the patch without clear understanding of it's consequences 
is not something we do.


At this time we still don't have a clear reproducer from you on "how" 
you constructed the invalid directory state. You reported the following:



I found the bug by doing a series of "ipa-client-install" (with lots
of arguments, followed by
echo ca_host = {a not-firewalled IPA CA} >> /etc/ipa/default.conf
echo [global] > /etc/ipa/installer.conf
echo ca_host = {ditto} >> /etc/ipa/installer.conf
echo {password} | kinit admin
ipa hostgroup-add-member ipaservers --hosts $(hostname -f)
ipa-relica-install --setup-ca --setup-dns --forwarder={ip addr}

followed

[389-devel] Re: [PATCH] prevent slapd from hanging under unlikely circumstances

2020-02-04 Thread Ludwig Krispenz

I think I can agree with 1-8, 9 is one solution to fix the problem you 
reported, but not yet validate that there are no other side effects, 
there are potential postop plugins which should NOT be called for 
tombstone delete, eg retro cl, we need to investigate side effects of 
the patch and evaluate other solutions, I suggested one in anotehr reply.


for 10, the claim that not crying hurrah and merging a patch without 
further investigation and testing is "doctrinal objection" is very 
strong and I have to reject this.


Regards,
Ludwig

On 02/04/2020 05:00 PM, Jay Fenlason wrote:

Ok, let's take this from the top:

1: Defects that cause a server to become unresponsive are bad and must
be repaired as soon as possible.

2: Some #1 class defects are exploitable and require a CVE.  As far as
I can tell, this one does not, but you should keep an eye out for the
possibility.

3: The #1 class defect I have found can be triggered in at least two
ways.  One requires ipa-replica-install and is hard to reproduce in a
test environment.  The other requires ldapdelete and is easy to
reproduce in an isolated VM.

4: The defect is a mismatch between the plugin ABI as implemented by
389-ds, and the behavior the NIS plugin expects.

5: I have found no specification or documentation on said ABI, so I
can only guess what the correct behavior is here.

6: The ABI includes two functions in the plugin: pre_delete and
post_delete.

7: The NIS plugin expects that every call to pre_delete will be
followed by a call to post_delete.  It takes a lock in pre_delete and
releases it in post_delete.

8: Under some circumstances 389-ds can call pre_delete but fail to
call post_delete.  Because of #5, there is no way to tell if this is
expected behavior, but the NIS plugin clearly does not expect it.

9: My patch ensures that every call to pre_delete is followed by a
corresponding call to post_delete.  V1 of the patch also ensures that
every call to post_delete is after a corresponding pre_delete call.
V2 relaxes the second requirement, allowing post_delete calls without
a corresponding pre_delete call because someone expressed worry about
potential regressions.

10: You are refusing to merge my patch because of a doctrinal
objection to the use of ldapdelete in the simple reproducer.
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Sitz: Grasbrunn,
Handelsregister: Amtsgericht München, HRB 153243,
Geschäftsführer: Charles Cachera, Laurie Krebs, Michael O'Neill, Thomas Savage
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: [PATCH] prevent slapd from hanging under unlikely circumstances

2020-02-04 Thread Ludwig Krispenz


Hi,

I agree with you that calls to pre and post should be balance, but not 
sure if your approach is the correct one. There is a condition for post 
"!delete_tombstone_entries" which prevented the call for postop plugins 
in case of the deletion of a tombstone entry. Your patch now ensures 
that it will be called in this cases. I think the correct way would be 
to also prevent the calls to preop in case of delete_tombstone_entry, 
this is an operation to remove replication artifacts and should not be 
known by plugins


Regards,
Ludwig

On 02/03/2020 04:57 PM, Jay Fenlason wrote:

Here's a backtrace of the hung server:
All of the other threads are in pthread_cond_wait(), select() or poll()

#0  0x7fce1454f35e in pthread_rwlock_wrlock ()
 at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_rwlock_wrlock.S:85
#1  0x7fce16e1acca in slapi_rwlock_wrlock (rwlock=)
 at ldap/servers/slapd/slapi2nspr.c:237
#2  0x7fce06a74748 in wrap_rwlock_wrlock (rwlock=)
 at wrap.c:328
#3  0x7fce06a7344c in plugin_wrlock () at map.c:1242
#4  0x7fce06a5d639 in backend_be_pre_write_cb (pb=)
 at back-sch.c:2381
#5  0x7fce16df8028 in plugin_call_func (list=0x55ddade18840, 
operation=operation@entry=453, pb=pb@entry=0x55ddaf08afc0, 
call_one=call_one@entry=0)
 at ldap/servers/slapd/plugin.c:2028
#6  0x7fce16df82e3 in plugin_call_list (pb=0x55ddaf08afc0, operation=453, 
list=) at ldap/servers/slapd/plugin.c:1972
#7  0x7fce16df82e3 in plugin_call_plugins (pb=pb@entry=0x55ddaf08afc0, 
whichfunction=whichfunction@entry=453) at ldap/servers/slapd/plugin.c:442
#8  0x7fce08a31c88 in ldbm_back_delete (pb=0x55ddaf08afc0)
 at ldap/servers/slapd/back-ldbm/ldbm_delete.c:373
#9  0x7fce16da8feb in op_shared_delete (pb=pb@entry=0x55ddaf08afc0)
 at ldap/servers/slapd/delete.c:324
#10 0x7fce16da9383 in do_delete (pb=pb@entry=0x55ddaf08afc0)
 at ldap/servers/slapd/delete.c:97
#11 0x55ddac64e62c in connection_dispatch_operation (pb=0x55ddaf08afc0, 
op=0x55ddadd17340, conn=0x55ddaf12ed00) at ldap/servers/slapd/connection.c:615
#12 0x55ddac64e62c in connection_threadmain ()
 at ldap/servers/slapd/connection.c:1790
#13 0x7fce14babc5b in _pt_root (arg=0x55ddaf0186c0)
 at ../../../nspr/pr/src/pthreads/ptthread.c:201
#14 0x7fce1454be65 in start_thread (arg=0x7fcdf405d700)
 at pthread_create.c:307
#15 0x7fce13bf788d in clone ()
 at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111



My thought when writing the patch was that calls to the pre delete callback
and the post delete callback should be balanced: either both of them should be
called or neither.  But you know the plugin abi better than I do, and if
you think it's ok to have the post callback called without a corresponding pre
who am I to argue?  I've just confirmed that this patch also prevents
the hangs I'm seeing:

--- a/ldap/servers/slapd/back-ldbm/ldbm_delete.c.orig   2020-01-31 
07:28:04.085861174 -0500
+++ b/ldap/servers/slapd/back-ldbm/ldbm_delete.c2020-01-31 
07:30:33.932947489 -0500
@@ -81,6 +81,7 @@
  Connection *pb_conn;
  int32_t parent_op = 0;
  struct timespec parent_time;
+int pre_delete_called = 0;
  
  if (slapi_pblock_get(pb, SLAPI_CONN_ID, _id) < 0) {

  conn_id = 0; /* connection is NULL */
@@ -371,6 +372,7 @@
  }
  if (retval == 0) {
  retval = plugin_call_plugins(pb, 
SLAPI_PLUGIN_BE_PRE_DELETE_FN);
+   pre_delete_called = 1;
  }
  if (retval)
  {
@@ -1491,7 +1493,7 @@
   * The bepostop is called even if the operation fails,
   * but not if the operation is purging tombstones.
   */
-if (!delete_tombstone_entry) {
+if (!delete_tombstone_entry || pre_delete_called) {
  plugin_call_plugins(pb, SLAPI_PLUGIN_BE_POST_DELETE_FN);
  }
  /* Need to return to cache after post op plugins are called */


Signed-off-by: Jay Fenlason 
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Sitz: Grasbrunn,
Handelsregister: Amtsgericht München, HRB 153243,
Geschäftsführer: Charles Cachera, Laurie Krebs, Michael O'Neill, Thomas Savage
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:

[389-devel] Re: Example for a password storage scheme plug-in (SLAPI_PLUGIN_PWD_STORAGE_SCHEME) - bcrypt?

2020-01-15 Thread Ludwig Krispenz


Hi William,

I think Harald was asking how to extend an existing deployment with a 
plugin, not to build a new plugin into the core. Just use it with a 
standard build.


Unfortunately our plugin guide is a bit old and the example plugins are 
no longer installed with the server. The best you could get is checking 
out the code and then look into the folder test-plugins, there are 
examples and Makefiles - but not up to date.


Regards,
Ludwig

On 01/15/2020 07:20 AM, William Brown wrote:

Hi Harald,

The most recently developed scheme was pbkdf2_pwd.c, so it likely has the 
"best" example of how to make a module. I've attached the original PBKDF2 diff 
so you can see the other files that may need to be altered, but the list is:

+++ Makefile.am
+++ dirsrvtests/tests/tickets/ticket397_test.py
+++ ldap/ldif/template-dse.ldif.in
+++ ldap/servers/plugins/pwdstorage/pbkdf2_pwd.c
+++ ldap/servers/plugins/pwdstorage/pwd_init.c
+++ ldap/servers/plugins/pwdstorage/pwdstorage.h
+++ ldap/servers/slapd/pw.c
+++ ldap/servers/slapd/pw.h

There have also been a number of changes to the pbkdf2 module since, so it's best to look 
at the "latest" version of pbkdf2_pwd.c of course.

I'd like to ask what scheme you were planning to add, as if it's relevant we 
could consider upstreaming it into the server. It's also good as we can provide 
code review and advice to help as well.

Hope this helps!

PS: I'm at a conference so I may be slow to respond this week





On 15 Jan 2020, at 03:52, Harald Strack  wrote:

Hi,

we need to implement a  password storage scheme plug-in for the 389 directory 
server. Especially we need to implement bcrypt support.  We checked the source 
code and documentation and found out that we need to write a 
SLAPI_PLUGIN_PWD_STORAGE_SCHEME plugin.

Some plugins of this type are in the source of 389-base are in 
389-ds-base/ldap/servers/plugins/pwdstorage, a good starting point seems to be 
one of these

clear_pwd.c
crypt_pwd.c
md5c.c
md5.h
md5_pwd.c
ns-mta-md5_pwd.bu
ns-mta-md5_pwd.c
pbkdf2_pwd.c
pwd_init.c
pwdstorage.h
pwd_util.c
sha_pwd.c
smd5_pwd.c
ssha_pwd.c

But these are core plugins. How do we implement a plugin as extension? An 
example project with autoconf / makefile(s) etc. would be great. Any help would 
be greatly appreciated!

br

Harald






___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs



___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Sitz: Grasbrunn,
Handelsregister: Amtsgericht München, HRB 153243,
Geschäftsführer: Charles Cachera, Laurie Krebs, Michael O'Neill, Thomas Savage

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Performance discussion

2019-11-14 Thread Ludwig Krispenz



On 11/14/2019 12:17 PM, William Brown wrote:



On 14 Nov 2019, at 19:06, Ludwig Krispenz  wrote:


On 11/14/2019 09:29 AM, William Brown wrote:

On 14 Nov 2019, at 18:22, Ludwig Krispenz  wrote:

Hi William,

before further thinking about this, I need some clarification, or maybe I just 
missed this. When you talk about 1..16 threads do you mean worker threads ?

Server worker threads. ldclt is set to only use 10 client threads - which is 
surprising that with 10 client threads we see a decline when workers > 10 (one 
would assume it should stabilise).


Or concurrent client connection threads in ldclt/rsearch/ - how many 
concurrent connections do you have and how does varying this number change 
results ?

I will add more tests to this to allow varying the ldclt numbers.

ok, and I assume that you are using a version with nunc-stans removed, could 
you please also verify the effect of tubo-mode on/off ?

Correct, I'm using git master. Yes I'll check that also. I plan to add 
permutations like this to the test harness so it's easier for us to repeat in 
the future when we make changes.

I also need to find a way to wire in perf/stap so we can generate flamegraphs 
from each test run too for later analysis.

Thanks for the great ideas :)

Thanks, and one more idea ;-)
Can you separate the client and the server on two different machines, 
I've seen ldclt or other clients impacting cpu usage a lot, there will 
be some network overhead, but this should be ok (and more realistic)



Regards,
Ludwig

On 11/14/2019 03:34 AM, William Brown wrote:

Hi all,

After our catch up, we were discussing performance matters. I decided to start 
on this while waiting for some of my tickets to be reviewed and to see what's 
going on.

These tests were carried out on a virtual machine configured in search 6 to 
have access to 6 CPU's, and search 12 with 12 CPU. Both machines had access to 
8GB of ram.

The hardware is an i7 2.2GHz with 6 cores (12 threads) and 32GB of ram, with 
NVME storage provided.

The rows are the VM CPU's available, and the columns are the number of threads 
in nsslapd-threadnumber. No other variables were changed. The database has 6000 
users and 4000 groups. The instance was restarted before each test. The search 
was a randomised uid equality test with a single result. I provided the thread 
6 and 12 columns to try to match the VM and host specs rather than just the 
traditional base 2 sequence we see.

I've attached a screen shot of the results, but I have some initial thoughts to 
provide on this. What's interesting is our initial 1 thread performance and how 
steeply it ramps up towards 4 thread. This in mind it's not a linear increase. 
Per thread on s6 we go from ~3800 to ~2500 ops per second, and a similar ratio 
exists in s12. What is stark is that after t4 we immediately see a per thread 
*decline* despite the greater amount of available computer resources. This 
indicates that it is poor locking and thread coordination causing a rapid 
decline in performance. This was true on both s6 and s12. The decline 
intesifies rapidly once we exceed the CPU avail on the host (s6 between t6 to 
t12), but still declines even when we do have the hardware threads available in 
s12.

I will perform some testing between t1 and t6 versions to see if I can isolate 
which functions are having a growth in time consumption.

For now an early recommendation is that we alter our default CPU auto-tuning. 
Currently we use a curve which starts at 16 threads from 1 to 4 cores, and then 
tapering down to 512 cores to 512 threads - however in almost all of these 
autotuned threads we have threads greater than our core count. This from this 
graph would indicate that this decision only hurts our performance rather than 
improving it. I suggest we change our thread autotuning to be 1 to 1 ratio of 
threads to cores to prevent over contention on lock resources.

Thanks, more to come once I setup this profiling on a real machine so I can 
generate flamegraphs.



—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs



___
389-devel mailing list --
389-devel@lists.fedoraproject.org

To unsubscribe send an email to
389-devel-le...@lists.fedoraproject.org

Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines

List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

--
Red Hat GmbH,
http://www.de.redhat.com/
, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
ht

[389-devel] Re: Performance discussion

2019-11-14 Thread Ludwig Krispenz



On 11/14/2019 09:29 AM, William Brown wrote:



On 14 Nov 2019, at 18:22, Ludwig Krispenz  wrote:

Hi William,

before further thinking about this, I need some clarification, or maybe I just 
missed this. When you talk about 1..16 threads do you mean worker threads ?

Server worker threads. ldclt is set to only use 10 client threads - which is 
surprising that with 10 client threads we see a decline when workers > 10 (one 
would assume it should stabilise).


Or concurrent client connection threads in ldclt/rsearch/ - how many 
concurrent connections do you have and how does varying this number change 
results ?

I will add more tests to this to allow varying the ldclt numbers.
ok, and I assume that you are using a version with nunc-stans removed, 
could you please also verify the effect of tubo-mode on/off ?



Regards,
Ludwig

On 11/14/2019 03:34 AM, William Brown wrote:

Hi all,

After our catch up, we were discussing performance matters. I decided to start 
on this while waiting for some of my tickets to be reviewed and to see what's 
going on.

These tests were carried out on a virtual machine configured in search 6 to 
have access to 6 CPU's, and search 12 with 12 CPU. Both machines had access to 
8GB of ram.

The hardware is an i7 2.2GHz with 6 cores (12 threads) and 32GB of ram, with 
NVME storage provided.

The rows are the VM CPU's available, and the columns are the number of threads 
in nsslapd-threadnumber. No other variables were changed. The database has 6000 
users and 4000 groups. The instance was restarted before each test. The search 
was a randomised uid equality test with a single result. I provided the thread 
6 and 12 columns to try to match the VM and host specs rather than just the 
traditional base 2 sequence we see.

I've attached a screen shot of the results, but I have some initial thoughts to 
provide on this. What's interesting is our initial 1 thread performance and how 
steeply it ramps up towards 4 thread. This in mind it's not a linear increase. 
Per thread on s6 we go from ~3800 to ~2500 ops per second, and a similar ratio 
exists in s12. What is stark is that after t4 we immediately see a per thread 
*decline* despite the greater amount of available computer resources. This 
indicates that it is poor locking and thread coordination causing a rapid 
decline in performance. This was true on both s6 and s12. The decline 
intesifies rapidly once we exceed the CPU avail on the host (s6 between t6 to 
t12), but still declines even when we do have the hardware threads available in 
s12.

I will perform some testing between t1 and t6 versions to see if I can isolate 
which functions are having a growth in time consumption.

For now an early recommendation is that we alter our default CPU auto-tuning. 
Currently we use a curve which starts at 16 threads from 1 to 4 cores, and then 
tapering down to 512 cores to 512 threads - however in almost all of these 
autotuned threads we have threads greater than our core count. This from this 
graph would indicate that this decision only hurts our performance rather than 
improving it. I suggest we change our thread autotuning to be 1 to 1 ratio of 
threads to cores to prevent over contention on lock resources.

Thanks, more to come once I setup this profiling on a real machine so I can 
generate flamegraphs.



—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs



___
389-devel mailing list --
389-devel@lists.fedoraproject.org

To unsubscribe send an email to
389-devel-le...@lists.fedoraproject.org

Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines

List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

--
Red Hat GmbH,
http://www.de.redhat.com/
, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.

[389-devel] Re: Performance discussion

2019-11-14 Thread Ludwig Krispenz


Hi William,

before further thinking about this, I need some clarification, or maybe 
I just missed this. When you talk about 1..16 threads do you mean worker 
threads ? Or concurrent client connection threads in ldclt/rsearch/ 
- how many concurrent connections do you have and how does varying this 
number change results ?


Regards,
Ludwig

On 11/14/2019 03:34 AM, William Brown wrote:

Hi all,

After our catch up, we were discussing performance matters. I decided 
to start on this while waiting for some of my tickets to be reviewed 
and to see what's going on.


These tests were carried out on a virtual machine configured in search 
6 to have access to 6 CPU's, and search 12 with 12 CPU. Both machines 
had access to 8GB of ram.


The hardware is an i7 2.2GHz with 6 cores (12 threads) and 32GB of 
ram, with NVME storage provided.


The rows are the VM CPU's available, and the columns are the number of 
threads in nsslapd-threadnumber. No other variables were changed. The 
database has 6000 users and 4000 groups. The instance was restarted 
before each test. The search was a randomised uid equality test with a 
single result. I provided the thread 6 and 12 columns to try to match 
the VM and host specs rather than just the traditional base 2 sequence 
we see.


I've attached a screen shot of the results, but I have some initial 
thoughts to provide on this. What's interesting is our initial 1 
thread performance and how steeply it ramps up towards 4 thread. This 
in mind it's not a linear increase. Per thread on s6 we go from ~3800 
to ~2500 ops per second, and a similar ratio exists in s12. What is 
stark is that after t4 we immediately see a per thread *decline* 
despite the greater amount of available computer resources. This 
indicates that it is poor locking and thread coordination causing a 
rapid decline in performance. This was true on both s6 and s12. The 
decline intesifies rapidly once we exceed the CPU avail on the host 
(s6 between t6 to t12), but still declines even when we do have the 
hardware threads available in s12.


I will perform some testing between t1 and t6 versions to see if I can 
isolate which functions are having a growth in time consumption.


For now an early recommendation is that we alter our default CPU 
auto-tuning. Currently we use a curve which starts at 16 threads from 
1 to 4 cores, and then tapering down to 512 cores to 512 threads - 
however in almost all of these autotuned threads we have threads 
greater than our core count. This from this graph would indicate that 
this decision only hurts our performance rather than improving it. I 
suggest we change our thread autotuning to be 1 to 1 ratio of threads 
to cores to prevent over contention on lock resources.


Thanks, more to come once I setup this profiling on a real machine so 
I can generate flamegraphs.



—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs



___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Future of nunc-stans

2019-10-09 Thread Ludwig Krispenz


Hi William,

I like your radical approach :-)

In my opinion our connection code is getting to complicated by 
maintaining two different implementations in parallel -  not separated, 
but intermangled (and even more complicated by turbo mode). So I agree 
we should have only one, but which one ? In my opinion nunc stans is the 
theoretically better approach, but nobody is confident enough to rely on 
nunc stans alone. The conntable mode has its problems (especially if 
handling many concurrent connections, and worse if they are established 
almost at the same time)(otherwise we would not have experimented with 
nunc stans), but is stable and for most of the use cases efficient enough.


So reducing the complexity by removing nunc stans (and maybe also turbo 
mode) and then do cleanup and try to improve the bottlenecks would be an 
acceptable approach to me.


In my opinion the core of the problem of the "old" connection code is 
that the main thread is handling new connections and already established 
connections and so does iterate over the connection table. Using an 
event model looks like the best way to handle this, but if it doesn't 
work we need to look for other improvements without breaking things.
Your suggestion to make the conn table data structure more lean and 
flexible is one option. In sun ds, when I didn't know about event queues 
I did split the main thread, one handling new connections and multiple 
to handle established connections (parts of teh conn table) - reusing 
the existing mechanisms, just splitting the load. Maybe we can also 
think in this direction.


Regards,
Ludwig

On 10/09/2019 01:32 AM, William Brown wrote:



On 9 Oct 2019, at 09:18, Rich Megginson  wrote:

On 10/8/19 4:55 PM, William Brown wrote:

Hi everyone,
In our previous catch up (about 4/5 weeks ago when I was visiting Matus/Simon), 
we talked about nunc-stans and getting it at least cleaned up and into the code 
base.
I've been looking at it again, and really thinking about it and reflecting on 
it and I have a lot of questions and ideas now.
The main question is *why* do we want it merged?
Is it performance? Recently I provided a patch that yielded an approximate ~30% 
speed up in the entire server through put just by changing our existing 
connection code.
Is it features? What features are we wanting from this? We have no complaints 
about our current threading model and thread allocations.
Is it maximum number of connections? We can always change the conntable to a 
better datastructure that would help scale this number higher (which would also 
yield a performance gain).

It is mostly about the c10k problem, trying to figure out a way to use epoll, 
via an event framework like libevent, libev, or libtevent, but in a 
multi-threaded way (at the time none of those were really thread safe, or 
suitable for use in the way we do multi-threading in 389).

It wasn't about performance, although I hoped that using lock-free data structures might 
solve some of the performance issues around thread contention, and perhaps using a 
"proper" event framework might give us some performance boost e.g. the idle 
thread processing using libevent timeouts.  I think that using poll() is never going to 
scale as well as epoll() in some cases e.g. lots of concurrent connections, no matter 
what sort of datastructure you use for the conntable.

The conntable was bottlenecking because when you had:

| conn | conn | freeslot | 

it would attempt to lock each conn to see if it was free or not. This meant if 
a conn was in an io, we would block waiting for it to finish before we could 
move to the next conn to check if it was free. After changing to pthread, we 
can now do trylock, where if trylock fails it can be implied the conn slot must 
be in use, so skip it. This is how we got the 30% speed up recently (my laptop 
went from about 4200 conn/sec to over 6000).

However the conntable exists to allow the conn struct to be re-used over and 
over. There are multiple ways we could solve this. On conn free, we could 
append to a freelist to prevent iterating over the list to find a slot. We 
could use a btreemap and just alloc/dealloc conns as needed to prevent the need 
to walk and find conns (this would help sanitizers with memory too). We could 
bring in libevent directly to the main server and have it replace poll too.

And even from then, I think we should be using flamegraphs to work out where 
our time limits are too.


As far as features goes - it would be nice to give plugins the ability to 
inject event requests, get timeout events, using the same framework as the main 
server engine.


The more I have looked at the code, I guess with time and experience, the more 
hesitant I am to actually commit to merging it. It was designed by people who 
did not understand low-level concurrency issues and memory architectures of 
systems,

I resemble that remark.  I suppose you could "turn off" the lock-free code and 
use mutexes.

[389-devel] please review: PR #50523: Ticket 50490 objects and memory leaks

2019-08-08 Thread Ludwig Krispenz


https://pagure.io/389-ds-base/pull-request/50523

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Replication agreement status messages: JSON or text?

2019-06-12 Thread Ludwig Krispenz


Hi Mark,

On 06/11/2019 08:15 PM, Mark Reynolds wrote:
I am currently working on a revision of replication agreement status 
messages.  Previously we logged the status like so:


Error (%d) - message (sub-message) ...

just to get it clear what you suggest, I was a bit confused about first.

Do you talk about logging (as in the error log) or about the value of 
the replicaLastUpdateStatus attribute ?


For logging into the error log I prefer to keep the current, "readable" 
format - until we do a real rework of logging.
For the storage of a state in the agreement I think switching to the 
json object is ok


If Error was set to 0 it meant success, but this caused confusion 
because of the word "Error".  So I am working on changing this.


There are two options here: change the static "Error" text to be 
dynamic: "Info", "Warning", or "Error" depending on the state. Or, 
move away from a human friendly text string to a machine friendly 
simple JSON object.  There are pro's and con's to both. I think moving 
towards a JSON object is the correct way - easier to maintain, and 
easier to be consumed by other applications.  The cons are that it is 
a disruptive change to the previous behavior, and it could be 
confusing to an Admin who might not understand JSON.


This is the basic JSON object I was thinking of

{"status": "Good|Warning|Bad", "status code": NUMBER(aka error 
code), "date": "2019117485748745Z", "message": "Message text"}


or maybe multiple messages (list):

{"status": "Good|Warning|Bad", "status code": NUMBER(aka error 
code), "date": "2019117485748745Z", "message": ["the replication 
status is...", "Connection error 91", "Server Down"]}



The JSON object can easily be extended without breaking clients, but 
it's not easy to read for a human.


Thoughts?

Thanks,

Mark
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Logging future direction and ideas.

2019-05-22 Thread Ludwig Krispenz



On 05/17/2019 12:44 AM, William Brown wrote:

I think this would be a "final goal", so to formalise the stages:

* Add build tooling and a simple (dummy) log thread as a "getting started". 
Supplement with documentation on wiki.
* Fill-in the log thread to support an "operation log", and add basic operation 
log hooks in the server.
* Fill in more operation log points in the server to build detail
* change slapi_log_err to log to the new rust thread, continue to generate 
error file
* change slapi_log_audit to log to the new rust thread, continue to generate 
audit file
* change slapi_log_access to log to the new rust thread, continue to generate 
access file.
* remove former logging code


I wonder if we really could have one api  eg slapi_log_* and different 
implementations, keep the current, implement a new one and allow to chose

I don't think I understand this comment, could you explain a bit more what you 
have in mind?
what I wanted to say is that right now for logging we have a split in 
"what" and "how", a function wants to log  something calls slapi_log_* 
and provides the loglevel, type (_err, _access), and information 
(formatstring and params). It does not know or care about log buffering, 
log rotation, they could even all go to the same file, if it is a 
dedicated thread or 


If we want to change logging I would like to keep this separation, and 
if we want to concentrate on performance we should first address the 
"how" part, doin all together might be too big a task and too much work. 
And we would not have to throw away the current impelmentation, it could 
be configured "how" slapi_log _* perform theit task.

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Logging future direction and ideas.

2019-05-16 Thread Ludwig Krispenz

On 05/16/2019 02:56 AM, William Brown wrote:

Replying to both Simon and Ludwig in the one mail ...

To just comment to one point from Ludwig:

This may sound too negative, I agree that logging deserves attention and
improvement, but we need to agree on what we want to achieve.

I don't think this is negative, at all and I think it's exactly the kind of
feedback I wanted. :) So thank you!

Hi William,
I am more than interested!

I'd like to learn it one day anyway.
So if there will be no objections and we'll go forward now,
I think, it is important to agree on some points of decreasing a bus factor:

- As you said, it should have very detailed comments on all of the
language features and technics you'll use.
- I think it will be nice to have an additional documentation which will
describe the basic setup for the development you use. All the toold you
need to develop, compile, test and debug.
- Also, some nice links for the basic tutorials on Rust types, concepts, etc.
- I think, we should have detailed unit tests. It will help to
understand the code better and we will have less bugs (hopefully).

So I'm writing a document for the wiki to cover this *and* rust supports tests
as part of the system - that will be all be included. Is that acceptable?

And the final and big point I wanted to mention:

- We should be prepared for a slow review process because we have quite limited
recources in the team and a lot of work should be done (WebUI still has to be
refactored to React,
and it is only a small piece of the workload).
Also, I think, it makes sense to have the smallest Rust PRs that can be put
together as an independent unit.

That's totally reasonable. I'll try to keep it to small parts and will build up
as we go. See below for ideas on how I'll work toward this.

But everything is just my opinion and I don't know what others think and
if everyone is prepared to join it. I am feeling excited though :)

Thanks,
Simon

P.S. check the contribution guide please. Espesially a part about
'rebase-force-push'. I think it is nice not to force push during
the review process (and rebase-squash only after you got an ACK).

Yep, I saw this change. I'll keep it in mind :)

On 15 May 2019, at 19:37, Ludwig wrote:

Hi William,

this is a good starting point to discuss and decide how to move forward with
logging (although it looks like you are already some steps ahead).

I have no strong ties to existing logging format and if we do it in rust or not
I don't care, but I do not want to use existing functionality for the sake af a
new unified approach.

First we need to define what is the purpose and usage of our logs, what do we
want to keep or extend. I see these main area:

- statistics

- auditing (authentications, modifications)

. debugging (for my usage the most important aspect)

Next: what are the real problems

- performance: yes, it would be nice to have less logging impact on
performance, but I haven't seen real complaints and doing it differently does
not guarantee better performance, we have debug levels (like tracing) which are
not usable because they slow down everything, I do not think this will be
resolved by just replacing the library

- information: we continue to add information to the logs, and there are still
open requests

So: If I look at your suggestions I do like some and have concerns with others,
and I am not sure if the priorities are right.

You list a couple of tickets related to logging, but forgot others, eg:

https://pagure.io/389-ds-base/issue/47822 - improve logging of ACL decisions

https://pagure.io/389-ds-base/issue/48680 - enable logging for 3rd party libs

https://pagure.io/389-ds-base/issue/50002 - improve password policy logging

https://pagure.io/389-ds-base/issue/50187 - improve replication logging

These are all things that you you are correct, that I missed.

So I want to focus on your point about what we want from logs, IE stats,
auditing, debugging. I think this is a great summary of what we want, but to
make it more detailed:

* Stats about operations from start to finish
* Auditing of client operations
* Debugging of client operations
* Debugging of internal server machinery and state
* Reporting of errors outside of the normal operation system.

Today we current have:

access - auditing of operations at a highlevel
audit - auditing of modifications and writes
error - debugging of internal server machinery (we tend not to enable the
levels of debugging operations ...)

So I think there is room to improve.

Now I think that performance so far is only a barrier in terms of preventing us from
adding more detail, because it slows down operations. This is why it's pretty high on the
list of "to fix".

these requests for improvements of logging exist for quite some time, they were accepted as useful
and good to have, but we didn't have the capacity to work on them. The work is tedious, going thru
ACL or replication code and

[389-devel] Re: Groups are not accessible by filter

2019-05-07 Thread Ludwig Krispenz



On 05/07/2019 02:08 PM, William Brown wrote:



On 7 May 2019, at 22:03, Viktor Ashirov  wrote:

On Mon, Apr 29, 2019 at 6:48 AM William Brown  wrote:




On 29 Apr 2019, at 12:33, Anuj Borah  wrote:

@William Brown

Thanks for the tip!

(Pdb) len(topo.standalone.search_s(DEFAULT_SUFFIX, 
ldap.SCOPE_SUBTREE,"testUserAccountControl:1.2.840.113556.1.4.803:=8388608", 
['attrlist=cn:sn:uid:testUserAccountControl']))
6
(Pdb) len(Accounts(topo.standalone, 
DEFAULT_SUFFIX).filter("(testUserAccountControl:1.2.840.113556.1.4.803:=8388608)"))
6

We cant not mix up ['attrlist=cn:sn:uid:testUserAccountControl'] with filter , 
like we do with search_s .

(Pdb) len(Accounts(topo.standalone, 
DEFAULT_SUFFIX).filter("(testUserAccountControl:1.2.840.113556.1.4.803:=8388608)",
 ['attrlist=cn:sn:uid:testUserAccountControl']))
*** TypeError: filter() takes 2 positional arguments but 3 were given
(Pdb) len(Accounts(topo.standalone, 
DEFAULT_SUFFIX).filter("(testUserAccountControl:1.2.840.113556.1.4.803:=8388608), 
['attrlist=cn:sn:uid:testUserAccountControl']"))
*** ldap.FILTER_ERROR: {'desc': 'Bad search filter', 'errno': 2, 'info': 'No 
such file or directory'}

Again i have to use "re" module for the same .



What are you trying to achieve?

Test case is very simple: search for entries using different filters
and request specific attributes.

But those entries have types and classes - you know what you are expecting to 
get.


The problem that Anuj is facing is that filter() doesn't support
attrlist. Moreover, _unsafe_raw_entry() doesn't return *all*
attributes, it omits operational attributes (like nsRoleDN).
IMHO, search_s is good enough here.

If you want to avoid any of the "magic" use DSLdapObjects(instance).filter() 
then because that doesn't prescribe any classes. But it does take a lot of the safety out 
of the library, and I still think that there is something missing in the approach here.
but we are testing ldap clients and need to be able to do anything the 
protocol allows, not only what you declare safe





Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org



--
Viktor
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Reminder: please review: PR 50307 - Revise CleanAllRUV restart process

2019-04-03 Thread Ludwig Krispenz



On 04/03/2019 03:24 PM, Mark Reynolds wrote:


On 4/3/19 9:01 AM, Ludwig Krispenz wrote:

Hi,
added some small comments, but it is overall ok.

I'll review them shortly :-)


Just thinking about it, wouldn't it be an option that the task entry 
survives until it is completed and written to the dse.ldif at 
shutdown, so at startup tasks could be resumed from the task entry 
not from params in the repl entry ?


I like the idea, but I don't know if the server will.  I ran into 
"timing" issues just trying to add the task during plugin startup.  
I'm worried that when we process dse.ldif it might be too soon to 
start a task. 
but i think the event queue is only started after plugins did start, so 
registering an event should be ok
Either way, it's a good idea and it can remove my hacky code and allow 
us to cleanup some of the cleanAllRUV code too, so I'm going to try 
it!  I'll add a "nsTaskPersist" attribute to the task, etc...


Thanks,
Mark



Ludwig

On 04/03/2019 02:27 PM, Mark Reynolds wrote:

Friendly reminder :-)


https://pagure.io/389-ds-base/pull-request/50307
___
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org




--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Reminder: please review: PR 50307 - Revise CleanAllRUV restart process

2019-04-03 Thread Ludwig Krispenz


Hi,
added some small comments, but it is overall ok.

Just thinking about it, wouldn't it be an option that the task entry 
survives until it is completed and written to the dse.ldif at shutdown, 
so at startup tasks could be resumed from the task entry not from params 
in the repl entry ?


Ludwig

On 04/03/2019 02:27 PM, Mark Reynolds wrote:

Friendly reminder :-)


https://pagure.io/389-ds-base/pull-request/50307
___
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

[389-devel] Re: Filter does not work with Anonymous connection

2019-02-27 Thread Ludwig Krispenz



On 02/26/2019 04:42 PM, Matus Honek wrote:

This kinda leads me to thinking we should implement ACIs management
within the DSLdapObjects like this (probably specific to a particular
subclass, to a degree). One that would take care of this added
requirement for objectclass ACIs because of hidden .filter's behavour.
Because that is currently really hard to be understood at a first
glance, or second.
I think the problem starts with the hiding, if methods do hidden things 
like transforming a filter, it is difficult for an application to know 
what is going on. But if you start adding magic to handle eg the aci 
problem you only increase and will get new unexpected behaviour. I would 
strongly object on silently adding/changing acis.


In my opinion we are at a crossroad with lib389, it has to fulfill two 
purposes


1] provide methods to access and manage a server, the methods should be 
easy to use and as safe as possible


2] provide methods to write test cases, also for scenarios where the 
client does "bad" things, you want to have full control an what is sent 
to the server at a very detailed level. Silently doing additional 
searches or transforming filters is a nogo in that kind of application.


I have the feeling we are more and more drifting to support 1] and 
making it harder to achieve 2]. The announcement that _s will be 
deprecated without offering a replacement for rwa access to the server 
fits in. BTW when was this deprecation discussed and decided ? This is 
nothing just to do on the side way.


Ludwig

On Tue, Feb 26, 2019 at 4:02 AM William Brown  wrote:




On 26 Feb 2019, at 12:58, Anuj Borah  wrote:

@William Brown

ACI syntax in test is correct,  it meant to give access to (mail = * ) only not 
any thing else . In the same case as mansion bellow:

Ummm, that’s not what I’m saying? I’m saying that you may *only* be giving access 
to the mail attribute, so as a result when the .filter generates and expands to 
(&(objectClass=account)(mail=*)), the objectClass is denied on the searcch, 
causing the test to fail (to prevent disclosure).

That’s why I suggest changing the aci to allowing mail AND objectClass, and 
testing again. I think this is atn aci issue not a python, and I’d like to rule 
out that first.


Domain(topo.standalone, DEFAULT_SUFFIX).replace("aci", 
'(target="ldap:///{};)(targetattr="mail")(version 3.0; acl "Test";allow (read,search,compare) 
(userdn = "ldap:///anyone;); )'.format(DEFAULT_SUFFIX))

 conn = Anonymous(topo.standalone).bind()
 # filter does not works with Anonymous
 assert 0 == Accounts(conn, DEFAULT_SUFFIX).filter('(mail=*)')   -  It 
does not work
 assert 3 == len(conn.search_s(DEFAULT_SUFFIX, ldap.SCOPE_SUBTREE, 
"mail=*")) - it works


We can clearly see sarch_s works under conn while ACI access to (mail=*) , in 
the same condition filter does not work at all . It gives 0 result , while 
search_s gives 3 .



On Tue, Feb 26, 2019 at 5:06 AM William Brown  wrote:



On 26 Feb 2019, at 05:09, Anuj Borah  wrote:



Hi all,

We have recently implemented Filter and Anonymous to lib389  . But it seems 
like Filter does not work with Anonymous connection .
It actually does not work with any kind of connection whether ACI allow or not  
rather than root  .

My suspense is it is related to this issue which is not yet fixed: 
https://pagure.io/389-ds-base/issue/50137

Please check attached test case .

I suspect they are not related, more likely the access control in your test 
doesn’t allow anonymous to search objectClass under DEFAULT_SUFFIX. If you 
change it to:

 Domain(topo.standalone, DEFAULT_SUFFIX).replace("aci", '(target="ldap:///{};)(targetattr=“mail || 
objectClass")(version 3.0; acl "Test";allow (read,search,compare) (userdn = "ldap:///anyone;); 
)'.format(DEFAULT_SUFFIX))

(I hope I have the aci syntax correct)



Regards
Anuj Borah
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

—
Sincerely,

William Brown
Software Engineer, 389 Directory Server
SUSE Labs
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org

—
Sincerely,

William Brown
Software Engineer, 389 Directory Server
SUSE Labs
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe

[389-devel] Re: [discuss] Entry cache and backend txn plugin problems

2019-02-26 Thread Ludwig Krispenz

Hi, I need a bit of time to read the docs and clear my thoughts, but one 
comment below

On 02/25/2019 01:49 AM, William Brown wrote:



On 23 Feb 2019, at 02:46, Mark Reynolds  wrote:

I want to start a brief discussion about a major problem we have backend 
transaction plugins and the entry caches.  I'm finding that when we get into a 
nested state of be txn plugins and one of the later plugins that is called 
fails then while we don't commit the disk changes (they are aborted/rolled 
back) we DO keep the entry cache changes!

For example, a modrdn operation triggers the referential integrity plugin which 
renames the member attribute in some group and changes that group's entry cache 
entry, but then later on the memberOf plugin fails for some reason.  The 
database transaction is aborted, but the entry cache changes that RI plugin did 
are still present :-(  I have also found other entry cache issues with modrdn 
and BE TXN plugins, and we know of other currently non-reproducible entry cache 
crashes as well related to mishandling of cache entries after failed operations.

It's time to rework how we use the entry cache.  We basically need a 
transaction style caching mechanism - we should not commit any entry cache 
changes until the original operation is fully successful.  Unfortunately the 
way the entry cache is currently designed and used it will be a major change to 
try to change it.

William wrote up this doc: 
http://www.port389.org/docs/389ds/design/cache_redesign.html

But this also does not currently cover the nested plugin scenario either (not 
yet).  I do know how how difficult it would be to implement William's proposal, 
or how difficult it would be to incorporate the txn style caching into his 
design.  What kind of time frame could this even be implemented in?  William 
what are your thoughts?

I like coffee? How cool are planes? My thoughts are simple :)

I think there is a pretty simple mental simplification we can make here though. 
Nested transactions “don’t really exist”. We just have *recursive* operations 
inside of one transaction.

Once reframed like that, the entire situation becomes simpler. We have one 
thread in a write transaction that can have recursive/batched operations as 
required, which means that either “all operations succeed” or “none do”. 
Really, this is the behaviour we want anyway, and it’s the transaction model of 
LMDB and other kv stores that we could consider (wired tiger, sled in the 
future).
I think the recursive/nested transaction on the database level are not 
the problem, we do this correctly already, either all or no change 
becomes persistent.
What we do not manage is modifications we do in parallel on the in 
memory structure like the entry cache, changes to the EC are not managed 
by any txn and I do not see how any of the database txn models would 
help, they do not know about ec and can abort changes.
We would need to incorporate the EC into a generic txn model, or have a 
way to flag ec entries as garbage for if a txn is aborted



If William's design is too huge of a change that will take too long to safely implement 
then perhaps we need to look into revising the existing cache design where we use 
"cache_add_tentative" style functions and only apply them at the end of the op. 
 This is also not a trivial change.

It’s pretty massive as a change - if we want to do it right. I’d say we need:

* development and testing of a MVCC/COW cache implementation (proof that it 
really really works transactionally)
* allow “disable/disconnect” of the entry cache, but with the higher level 
txn’s so that we can prove the txn semantics are correct
* re-architect our transaction calls so that they are “higher” up. An example 
is that internal_modify shouldn’t start a txn, it should be given the current 
txn state as an arg. Combined with the above, we can prove we haven’t corrupted 
our server transaction guarantees.
* integrate the transactional cache.

I don’t know if I would still write a transactional cache the same way as I 
proposed in that design, but I think the ideas are on the right path.


And what impact would changing the entry cache have on Ludwig's plugable 
backend work?

Should be none, it’s seperate layers. If anything this change is going to make 
Ludwig’s work better because our current model won’t really take good advantage 
of the MVCC nature of modern kv stores.


Anyway we need to start thinking about redesigning the entry cache - no matter 
what approach we want to take.  If anyone has any ideas or comments please 
share them, but I think due to the severity of this flaw redesigning the entry 
cache should be one of our next major goals in DS (1.4.1?).

Thanks,

Mark
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines:

[389-devel] Re: creating a universal class to replace "Entry"

2019-02-12 Thread Ludwig Krispenz



On 02/11/2019 08:35 AM, William Brown wrote:



On 11 Feb 2019, at 17:01, Anuj Borah  wrote:

Hi,

As 'Entry' is not allowed to use now , In replace of Entry we are suggested to 
use UserAccounts and UserAccount which is very limited to some object classes . 
(['account','posixaccount','inetOrgPerson','organizationalPerson’]
)


There are many other types than this though?

src/lib389/lib389/plugins.py:23:class Plugin(DSLdapObject):
src/lib389/lib389/plugins.py:199:class MEPConfig(DSLdapObject):
src/lib389/lib389/plugins.py:233:class MEPTemplate(DSLdapObject):
src/lib389/lib389/plugins.py:444:class ReferentialIntegrityConfig(DSLdapObject):
src/lib389/lib389/plugins.py:758:class MemberOfSharedConfig(DSLdapObject):
src/lib389/lib389/plugins.py:887:class AutoMembershipRegexRule(DSLdapObject):
src/lib389/lib389/plugins.py:905:class AutoMembershipDefinition(DSLdapObject):
src/lib389/lib389/plugins.py:1015:class AutoMembershipRegexRule(DSLdapObject):
src/lib389/lib389/plugins.py:1120:class LinkedAttributesConfig(DSLdapObject):
src/lib389/lib389/plugins.py:1551:class AccountPolicyConfig(DSLdapObject):
src/lib389/lib389/plugins.py:1598:class DNAPluginConfig(DSLdapObject):
src/lib389/lib389/idm/organizationalrole.py:17:class 
OrganizationalRole(DSLdapObject):
src/lib389/lib389/idm/nscontainer.py:11:class nsContainer(DSLdapObject):
src/lib389/lib389/idm/nscontainer.py:49:class nsHiddenContainer(DSLdapObject):
src/lib389/lib389/idm/domain.py:11:class Domain(DSLdapObject):
src/lib389/lib389/idm/posixgroup.py:18:class PosixGroup(DSLdapObject):
src/lib389/lib389/idm/organization.py:17:class Organization(DSLdapObject):
src/lib389/lib389/idm/ipadomain.py:11:class IpaDomain(DSLdapObject):
src/lib389/lib389/idm/group.py:17:class Group(DSLdapObject):
src/lib389/lib389/idm/group.py:104:class UniqueGroup(DSLdapObject):
src/lib389/lib389/idm/organizationalunit.py:16:class 
OrganizationalUnit(DSLdapObject):
src/lib389/lib389/idm/account.py:15:class Account(DSLdapObject):
src/lib389/lib389/tombstone.py:18:class Tombstone(DSLdapObject):
src/lib389/lib389/backend.py:393:class Backend(DSLdapObject):
src/lib389/lib389/backend.py:831:class DatabaseConfig(DSLdapObject):
src/lib389/lib389/tasks.py:24:class Task(DSLdapObject):
src/lib389/lib389/pwpolicy.py:287:class PwPolicyEntry(DSLdapObject):
src/lib389/lib389/config.py:28:class Config(DSLdapObject):
src/lib389/lib389/config.py:205:class Encryption(DSLdapObject):
src/lib389/lib389/config.py:233:class RSA(DSLdapObject):
src/lib389/lib389/config.py:358:class LDBMConfig(DSLdapObject):
src/lib389/lib389/monitor.py:16:class Monitor(DSLdapObject):
src/lib389/lib389/monitor.py:109:class MonitorLDBM(DSLdapObject):
src/lib389/lib389/monitor.py:143:class MonitorBackend(DSLdapObject):
src/lib389/lib389/monitor.py:190:class MonitorChaining(DSLdapObject):
src/lib389/lib389/saslmap.py:12:class SaslMapping(DSLdapObject):
src/lib389/lib389/index.py:27:class Index(DSLdapObject):
src/lib389/lib389/index.py:64:class VLVSearch(DSLdapObject):
src/lib389/lib389/index.py:147:class VLVIndex(DSLdapObject):
src/lib389/lib389/encrypted_attributes.py:15:class EncryptedAttr(DSLdapObject):
src/lib389/lib389/_mapped_object.py:84:class DSLdapObject(DSLogging):
src/lib389/lib389/_mapped_object.py:441:if not issubclass(type(obj1), 
DSLdapObject) or not issubclass(type(obj2), DSLdapObject):
src/lib389/lib389/_mapped_object.py:442:raise ValueError("Invalid 
arguments: Expecting object types that inherits 'DSLdapObject' class")
src/lib389/lib389/agreement.py:23:class Agreement(DSLdapObject):
src/lib389/lib389/cos.py:18:class CosTemplate(DSLdapObject):
src/lib389/lib389/cos.py:65:class CosIndirectDefinition(DSLdapObject):
src/lib389/lib389/cos.py:111:class CosPointerDefinition(DSLdapObject):
src/lib389/lib389/replica.py:881:class Replica(DSLdapObject):
src/lib389/lib389/replica.py:1288:class 
BootstrapReplicationManager(DSLdapObject):
src/lib389/lib389/mappingTree.py:381:class MappingTree(DSLdapObject):
src/lib389/lib389/chaining.py:18:class ChainingConfig(DSLdapObject):
src/lib389/lib389/chaining.py:59:class ChainingDefault(DSLdapObject):
src/lib389/lib389/chaining.py:81:class ChainingLink(DSLdapObject):
src/lib389/lib389/changelog.py:20:class Changelog5(DSLdapObject):
src/lib389/lib389/rootdse.py:15:class RootDSE(DSLdapObject):
src/lib389/lib389/referral.py:14:class Referral(DSLdapObject):
src/lib389/lib389/schema.py:71:class Schema(DSLdapObject):

At worst, you can always use a raw:

DSLdapObject(instance, dn)

to get “minimal” access to *anything* regardless of type. The entire value 
proposition of lib389 is that we wrap *the most common and useful types*. Each 
one of these sub-classes of DSLdapObject provides wrappers, helps, guides, 
functions and more to interacting with nearly every aspect of the server.


This way we will be able to create any kind of entry whether it may be 
user/role/sub suffix  or anything .

I think this is already a solved problem, so I don’t think we really need

[389-devel] Re: [discuss] composable object types in lib389

2019-01-17 Thread Ludwig Krispenz

On 01/17/2019 09:57 AM, Anuj Borah wrote:

Hay William.

Here  i am not using nsUserAccount  in nsUserAccountRole as it 
requires 'uid' which is not allowed in  nsFilteredRoleDefinition and 
nsRoleDefinition . Below are usages:

user=nsUserAccountRole(topo.standalone,'cn=tuser1,ou=People,dc=example,dc=com')
user_props={'cn':'Anuj', 'nsRoleFilter':'cn=*'}
user.create(properties=user_props, basedn=SUFFIX)

nsUserAccountRoles(topo.standalone, DEFAULT_SUFFIX).list()[0].dn
>> 'cn=tuser1,ou=People,dc=example,dc=com'

nsUserAccountRoles(topo.standalone, 
DEFAULT_SUFFIX).create(properties=user_props)

>>> 
nsUserAccountRoles(topo.standalone, DEFAULT_SUFFIX).list()[1].dn
>>> 'cn=Anuj,ou=People,dc=example,dc=com'

Let me know , what you think .

Maybe I do not understand how it works because of some lib389 magic, but 
I think this is not how roles work.

You are  creating cn=tuser1 and cn=Anju and they will have the role 
objectclasses, but the benefit of roles is that you do NOT have to touch 
the useres to assign roles to them. There is a class of users and a 
class of role definitions and ONLY the change in the role definition 
will determine if a user has a role or not.

Regards
Anuj Borah

On Tue, Jan 15, 2019 at 6:30 AM William Brown > wrote:

> On 14 Jan 2019, at 19:28, Anuj Borah mailto:abo...@redhat.com>> wrote:
>
> Hi William ,
>
> Just find out a way to do it .

This isn’t quite what I had in mind.

Remember, we should be able to compose nsRole types to various
other objects if required (despite my dislike of nsRoles …).

We have "nsUserAccount(Account)”, and we need to be able to extend
it with nsRole types.

One way to achieve this is:

class nsRoleDefinition(object):
def __init__(self, instance, dn=None):
if ‘_create_objectclasses’ not in self:
raise Exception ….
# So we must have been combined with a type to add roles
self._create_objectclasses.append(’nsFilteredRoleDefinition’)

class nsUserAccountRole(nsUserAccount, nsRoleDefinition):
def __init__(self, instance, dn=None):
super(nsUserAccount, self).__init__(instance, dn)
super(nsRoleDefinition, self).__init__(instance, dn)

Then you would use the nsUserAccountRole like normal. (I think we
may need a similar “nsUserAccountRoles" for the muliple search parts)

A benefit to this, is you could have role-specific functions like
setting/changing the filter, but you never loose the “account”
features like bind. Provided a method is uniquely sourced, I think
python takes the implementation that is unique, or it takes the
“first”. So this should all just work.

The main benefit here is it’s really clean, we can compose this to
other types. It also avoids any duplication of class definitions
and logic etc.

I think this is how I would like to see it created. It may be
worth making a “ticket” just for the nsRole parts and splitting
your test for nsRoles out of the mega-patch you have.

>
> class UserAccountnsRole(Account):
>
> def __init__(self, instance, dn=None):
> super(UserAccountnsRole, self).__init__(instance, dn)
> self._rdn_attribute = RDN
> self._create_objectclasses = [
> 'top',
> 'LDAPsubentry',
> 'nsRoleDefinition',
>  'nsComplexRoleDefinition',
>  'nsFilteredRoleDefinition'
> ]
> user_compare_exclude = [
> 'nsUniqueId',
> 'modifyTimestamp',
> 'createTimestamp',
> 'entrydn'
> ]
> self._compare_exclude = self._compare_exclude +
user_compare_exclude
> self._protected = False
>
> def _validate(self, rdn, properties, basedn):
> if 'ntUserDomainId' in properties and 'ntUser' not in
self._create_objectclasses:
>  self._create_objectclasses.append('ntUser')
>
> return super(UserAccountnsRole, self)._validate(rdn,
properties, basedn)
>
>
> def create_test_user_nsrole(instance, cn, nsRoleFilter,
description, suffix=None):
> global test_user_id
> if cn is None:
> cn = "testuser_" + str(test_user_id)
> test_user_id += 1
> if suffix is None:
> suffix =  DEFAULT_SUFFIX
> dn = "cn=" + cn + "," + suffix
> properties = {
> 'cn': cn,
> "nsRoleFilter": nsRoleFilter,
> "description": description,
> }
> user = UserAccountnsRole(instance, dn)
>  user.create(properties=properties)
> return user
> Now just using create_test_user_nsrole we will be able to create
entries with nsRoles.
>
>
> Regards
> Anuj Borah
>
>
>
>
>
>
> On Mon, Jan 7, 2019 at 2:30 PM

[389-devel] Please review: ticket #49551 Child entry cenotaphs should not prevent the deletion of the parent

2018-02-07 Thread Ludwig Krispenz


https://pagure.io/389-ds-base/issue/49551

https://pagure.io/389-ds-base/issue/raw/files/e81577a307fc6e719e96776f355181223a7e58470157de7fb12326e705081ed9-0001-Ticket-49551-v2-correct-handling-of-numsubordinates-.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Do aci's work in nested groups?

2018-01-10 Thread Ludwig Krispenz



On 01/10/2018 07:03 AM, William Brown wrote:

If I have:

(targetattr=x)(version 3.0;  allow(read, search)(groupdn=cn=x);)

If cn=x has member cn=y, and cn=y member uid=z

Does uid=z have permission to the targetattr here? IE do our aci's work
through nested groups?

yes, they should, but there is configurable max nesting level





--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Ticket 49493 - heap-use-after-free in csn_as_string

2017-12-13 Thread Ludwig Krispenz


https://pagure.io/389-ds-base/issue/49493
https://pagure.io/389-ds-base/issue/raw/files/d368eb763847fbabddf8bc9b474259e6443e9184db7254eb5c2a27584cae40d5-0001-Ticket-49493-heap-use-after-free-in-csn_as_string.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Ticket 49443: scope one searches in 1.3.7 give incorrect results

2017-11-09 Thread Ludwig Krispenz


https://pagure.io/389-ds-base/issue/49443
https://pagure.io/389-ds-base/issue/raw/files/cdc2c0bf853fbc119c61befc5496efe73845353f1b86365eb70f605d5fc214e5-0001-Ticket-49443-scope-one-searches-in-1.3.7-give-incorr.patch

Note: the testcase passes in master, but I think it is masked by another 
filter optimisation and this fix should be applied to master as well.


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: #49431 replicated MODRDN fails breaking replication

2017-10-30 Thread Ludwig Krispenz

This is a patch for 1.3.6, 1.3.7 has the new repl conflict code and the 
issue doesn't exist there.


https://pagure.io/389-ds-base/issue/49431
https://pagure.io/389-ds-base/issue/raw/files/20a14477469be8ef77d92d1023456c6d8409d1c8e966837759293e11a47643de-0001-Ticket-49431-replicated-MODRDN-fails-breaking-replic.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: please review: Ticket 49038 - Remove legacy replication

2017-10-04 Thread Ludwig Krispenz


Hi,

this patch contains a script to remove the config entry on upgrade, but 
Alexander says on f26 it isn't called. any idea ?


Ludwig

On 07/12/2017 09:48 PM, Mark Reynolds wrote:

https://pagure.io/389-ds-base/issue/49038

https://pagure.io/389-ds-base/issue/raw/fd4fec903b65d4a7cb32861012f24f5be76d1b1804823c972bc02963de9a3d20-0001-Ticket-49038-Remove-legacy-replication.patch 




___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Build failed in Jenkins: NIGHTLY #71

2017-09-06 Thread Ludwig Krispenz


I think this error
*
**Could not open the LDIF template file 
'\''/usr/share/dirsrv/data/template-pampta.ldif'\''.  Error: No such file or 
directory*

points to the changes in https://pagure.io/389-ds-base/issue/49371

On 09/06/2017 06:37 AM, marey...@redhat.com wrote:

See 


--
[...truncated 4714 lines...]

tickets/ticket47462_test.py:155:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../lib389/lib389/__init__.py:2588: in upgrade
 DirSrvTools.runUpgrade(self.prefix, online)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

prefix = '\''/usr'\'', online = True

 @staticmethod
 def runUpgrade(prefix, online=True):
 '\'''\'''\''
 Run "setup-ds.pl --update"  We simply pass in one DirSrv isntance, 
and
 this will update all the instances that are in this prefix.  For 
the
 update to work we must fix/adjust the permissions of the scripts 
in:
 
 /prefix/lib[64]/dirsrv/slapd-INSTANCE/

 '\'''\'''\''
 
 libdir = os.path.join(_ds_paths.lib_dir, '\''dirsrv'\'')
 
 # Gather all the instances so we can adjust the permissions, otherwise

 servers = []
 path = os.path.join(_ds_paths.sysconf_dir, '\''dirsrv'\'')
 for files in os.listdir(path):
 if files.startswith('\''slapd-'\'') and not 
files.endswith('\''.removed'\''):
 servers.append(os.path.join(libdir, files))
 
 if len(servers) == 0:

 # This should not happen
 log.fatal('\''runUpgrade: no servers found!'\'')
 assert False
 
 '\'''\'''\''

 The setup script calls things like 
/lib/dirsrv/slapd-instance/db2bak,
 etc, and when we run the setup perl script it gets permission 
denied
 as the default permissions are 750.  Adjust the permissions to 755.
 '\'''\'''\''
 for instance in servers:
 for files in os.listdir(instance):
 os.chmod(os.path.join(instance, files), 755)
 
 # Run the "upgrade"

 try:
 prog = os.path.join(_ds_paths.sbin_dir, PATH_SETUP_DS)
 process = subprocess.Popen([prog, '\''--update'\''], shell=False,
stdin=subprocess.PIPE)
 # Answer the interactive questions, as "--update" currently does
 # not work with INF files
 process.stdin.write('\''yes\n'\'')
 if(online):
 process.stdin.write('\''online\n'\'')
 for x in servers:
 process.stdin.write(DN_DM + '\''\n'\'')
 process.stdin.write(PW_DM + '\''\n'\'')
 else:
 process.stdin.write('\''offline\n'\'')
 process.stdin.close()
 process.wait()
 if process.returncode != 0:
 log.fatal('\''runUpgrade failed!  Error: %s '\'' % 
process.returncode)

   assert(False)

E   assert False

../../../lib389/lib389/tools.py:952: AssertionError
 Captured stdout setup -
OK group dirsrv exists
OK user dirsrv exists
OK group dirsrv exists
OK user dirsrv exists
('\''Update succeeded: status '\'', '\''0 Total update succeeded'\'')
 Captured stderr setup -
INFO:lib389.topologies:Instance with parameters {'\''ldap-port'\'': 39001, 
'\''suffix'\'': '\''dc=example,dc=com'\'', '\''krb5_realm'\'': None, 
'\''deployed-dir'\'': '\''/usr'\'', '\''inst-backupdir'\'': '\''/tmp'\'', 
'\''hostname'\'': '\''localhost'\'', '\''server-id'\'': '\''master1'\'', 
'\''root-pw'\'': '\''password'\'', '\''root-dn'\'': '\''cn=Directory 
Manager'\'', '\''group-id'\'': None, '\''InstScriptsEnabled'\'': None, 
'\''user-id'\'': None, '\''ldap-secureport'\'': None} was created.
INFO:lib389:Found entry dn: cn=replrepl,cn=config
cn: bind dn pseudo user
cn: replrepl
objectClass: top
objectClass: person
sn: bind dn pseudo user
userPassword: 
{SSHA512}f6aZQxcJAMoeRJTIEGkMpmi0vRWkeZvZiLcgMJSG5eEbcB7gyp5C0Fcc5ACw72vjMAJauFL3uf3tIVsPP67LSkTPEXM3yYJg


INFO:lib389.topologies:Instance with parameters {'\''ldap-port'\'': 39002, 
'\''suffix'\'': '\''dc=example,dc=com'\'', '\''krb5_realm'\'': None, 
'\''deployed-dir'\'': '\''/usr'\'', '\''inst-backupdir'\'': '\''/tmp'\'', 
'\''hostname'\'': '\''localhost'\'', '\''server-id'\'': '\''master2'\'', 
'\''root-pw'\'': '\''password'\'', '\''root-dn'\'': '\''cn=Directory 
Manager'\'', '\''group-id'\'': None, '\''InstScriptsEnabled'\'': None, 
'\''user-id'\'': None, '\''ldap-secureport'\'': None} was created.
INFO:lib389:Found entry dn: cn=replrepl,cn=config
cn: bind dn pseudo user
cn: replrepl
objectClass: top

[389-devel] Re: Strategy proposal for making DB dump in LDIF format from dbscan

2017-08-22 Thread Ludwig Krispenz



On 08/22/2017 01:31 AM, William Brown wrote:

I have a question / concern though. I thought that we want dbscan 2
ldif for emergency recovery scenarios when all else has gone bad and
assuming that id2entry is still readable. In the approach you
described we make the assumption that the parentid index is readable
as well. So we depend on two files instead of one for exporting the
database. Does this matter or we don't care at all?

There are two scenarios here in my opinion.  Backup, and emergency
backup :-)  As I've previously stated: performance is important.  It
should not take forever to process a 100 million entry database.  I
think the tool should use multiple index files (id2entry + friends) if
we can generate the LDIF faster.  But, if some of those indexes are
corrupted, then we need an alternate algorithm to generate it just from
id2entry.  Also, if we are dealing with a corrupted db, then performance
is not important, recovery is.  So if we can do it fast, do it,
otherwise grind it out.

All that being said there is something we need to consider, which I
don't have an answer for, and that is when databases do get corrupted
which files typically get corrupted?  Is it indexes, or is it id2entry?
To be honest database corruption doesn't happen very often, but the tool
should be smart enough to realize that the data could be inaccurate.
Perhaps a parent could be missing, etc.  So the tool should be robust
enough to use multiple techniques to complete an entry, and if it can't
it should log something, or better yet create a rejects file that an
Admin can take and repair manually.

I know this is getting more complicated, but we need to keep these
things in mind.

Regards,
Mark

With the current design of id2entry and friends, we can't automatically
detect this so easily. I think we should really just have a flag on
dbscan that says "ignore everything BUT id2entry" and recover all you
can. We should leave this to a human to make that call.

If our database had proper checksumming of content and pages, we could
detect this, but today that's not the case :(
well, BDB has db_verify to ensure that a db file is consistent in itself 
and can be processed, this should be good enough to decide if it is usable.
But, as Mark mentioned backup, if the backup is an online backup we can 
not be sure that the id2entry alone is sane backup/restore relies on 
backup of txn logs and recovery on restore. That said, after a crash we 
can also have the situation that pages are not flushed from dbcache.
Generating an ldif from id2entry can be a best effort only and might 
fail in situations wher it is most needed.


About the general strategy, maybe we can use another approach (yes, 
always new ideas).
In generating the ldif we have to solve the problem that when cursoring 
thru id2entry child entries can come before parent entries. And we do it 
in different places: total replication init, db2ldif and now in a utility.
Wouldn't it be better to make ldif import smarter, and stack entries 
without parents until the parent is read ? This would simplify the 
export,init and be solved in one place.




___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Strategy proposal for making DB dump in LDIF format from dbscan

2017-08-17 Thread Ludwig Krispenz


Hi,
Ilias' proposal follows the db2ldif approach and I think it will work, 
even if it might need some tweaks to handle multiple out of order oparents.


An other option would be to follow the total update approach using the 
parentid index and direct get from id2entry.

You start with the suffix entry and get the entryid. id_suf
Then lookup the parentid index for id_suf and get the direct children 
ids and access them in the id2entry. For each entry sent you check if it 
has children and ...
The advantage would be to touch each entry once, the entries will be in 
parent first order and you always have the dn of the parent when writing 
the children, so the dn of the children can easily be constructed


Ludwig

On 08/17/2017 02:55 AM, William Brown wrote:

On Tue, 2017-08-15 at 22:03 +0300, Ilias Stamatis wrote:

2017-08-15 9:56 GMT+03:00 William Brown :


On Fri, 2017-08-11 at 17:49 +0300, Ilias Stamatis wrote:

Hi everybody,

Following Ludwig's and Mark's suggestions on how to perform a database

dump

in LDIF format from dbscan, I have come up with a strategy. I'm talking
about ticket #47567: https://pagure.io/389-ds-base/issue/47567

I'd like to discuss this strategy and get some feedback.

The general idea is:

- We are cursing though id2entry.db printing entries in order.
- Parents should be printed before children.
- Hence if for some entry parenitd > entryid, we have to print its parent
first (out of order) and track that we did so.
- In order to do the above, we don't need to move the db cursor. We can
just go fetch something from a random place in the db and the cursor will
remain in its place, so we can continue from where we left off after

we're

done with printing a parent.
- We also need to construct and print the DN of each entry using its RDN
and the DN of its father.
- Let's use something like a hash map to pair entryids with dns (only for
entries that has children), e.g. map[1] = "dc=example,dc=com", map[4] =
"ou=People,dc=example,dc=com", etc.

I'll present the algorithm that I came up with in python-like

pseudo-code.

First, the following function constructs the entry's DN and updates the
hash map if needed. We can know whether an entry is a parent or not, by

the

presence of the numSubordinates attribute.

# assumes that parentid already exists in the map
function display_entry(e, map):
 if not e.parentid:
 e.dn = e.rdn
 else:
 e.dn = e.rdn + map[e.parentid]
 if isparent(e):
 map[e.entryid] = e.dn
 print_to_ldif_format(e)

Then, the main loop:

map = new(hashmap)
printed_in_advance = []

for e in entries:
 if e.entryid in printed_in_advance:
 continue # skip this, we have already displayed it

 if e.parentid < e.entryid:
 display_entry(e, map)

 else:
 # we need to display parents before children

 list_of_parents = []

 p = e
 while p.parentid > p.entryid:
 p = get_from_db(key = p.parentid)
 list_of_parents.append(p) # see note below (*)

 for p in reversed(list_of_parents):
 display_entry(p, map)
 printed_in_advance.append(p.entryid)


* here we store the whole entry in the list (aka its data) and not just
its id, in order to avoid fetching it from the database again

As a map, we can use a B+ tree implementation from libsds.

I would like to know how the above idea sounds to you before going ahead
and implementing it.


Looks like a pretty good plan to me.

What happens if you have this situation?

rdn: a
entryid: 1
parentid: 2

rdn: b
entryid: 2
parentid: 3

rdn: c
entryid: 3
parentid: 4

etc.

Imagine we have to continue to work up ids to get to the parent. Can
your algo handle this case?


Unless I'm mistaken, yes. Provided of course that there is a root entry
which has no parentid.

This is the part that handles this:

  p = e
  while p.parentid > p.entryid:
  p = get_from_db(key = p.parentid)
  list_of_parents.append(p)

We may jump at a few random points in the db for this and we will have to
keep just a few entries in memory. But I think this number of entries is
always going to be very small e.g. 1-2 or maybe even 7-8 in more "extreme"
cases, but never more than let's say 15. I might as well be completely
wrong about this, so, please correct me if that's the case.



It seems like the way you could approach this is to sort the id order
you need to display them in *before* you start parsing entries. We can
file allids in memory anyway, because it's just a set of 32bit ints, so
even at 50,000,000 entries, this only takes 190MB of ram to fit allids.
So I would approach this as an exercise for a comparison function to the
set.

universal_entry_set = [] # entryrdn / id2entry
# The list of entries to print
print_order = []

for e in universal_entry_set:
 printer_order_insert_sorted(e)

So we can imagine that this would invoke a sorting function. Something

[389-devel] Re: Trying to understand entryrdn.db

2017-08-04 Thread Ludwig Krispenz



On 08/04/2017 02:08 PM, Ilias Stamatis wrote:
Okay, now that I have read and understood dbscan's code, I have a few 
more questions.


2017-08-03 10:10 GMT+03:00 Ludwig Krispenz <lkris...@redhat.com 
<mailto:lkris...@redhat.com>>:


Hi, now that I know the context here are some more comments.
If the purpose is to create a useful ldif file, which could
eventually be used for import then formatting an entry correctly
is not enough. Order of entries matters: parents need to come
before children. We already handle this in db2ldif or replication
total update.
That said, whenever you write an entry you always have seen the
parent and could stack the dn with the parentid and createt the dn
without using the entryrdn index.
You even need not to keep track of all the entry rdsn/dns - only
the ones with children will be needed later, the presence of
"numsubordinates"
identifies a parent.


Is it guaranteed that parents are going to appear before children in 
id2entry.db?
no. that's what I said before, it is possible that parentid > entryid. 
It happens if an entry is moved by modrdn to aother subtree


If so, here's what could probably work:

- Start reading entries from id2entry sequentially.
- For each entry, if it has a numSubordinates attribute it means it is 
a parent for other entries. So we can store it's ID - DN pair in a 
hash map.
- For entries that they have a parentid and so we need to figure out 
their parent's DN, we just look for hashmap[parentid].


To make it even more efficient (if really needed though, because it 
will make things more complicated) we can store the value of 
numSubordinates with each parent as well somehow in the map. Every 
time a parentid is looked in the map we can decrease the value of 
numSubordinates by 1. When it becomes 0, it means there are no more 
children of this ID so we can safely remove it from the map.


However, I don't know if we would really need this last thing. In a 
100 million entry db how many parents would we expect to have 
approximately?


Also, do we have a hash map implemented somewhere?

If parents are not guaranteed to appear before children in 
id2entry.db, then we would have to alter the above strategy.


Thanks!



___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Trying to understand entryrdn.db

2017-08-03 Thread Ludwig Krispenz



On 08/02/2017 09:12 PM, Mark Reynolds wrote:



On 08/02/2017 02:19 PM, Ilias Stamatis wrote:

I see now, thank you both very much!

Follow-up:

[1]  Get entry from id2entry and use its ID
[2]  Look in entryrdn for the parent of the ID
[3]  Keep looking for parents, building the DN as you go along


Example:

[1]  Get entry from id2entry:  ID 6 --> "cn=Accounting Managers"
[2]  Check entryrdn for "P".  In this case it's "P6" which is
"ou=Groups" with ID 3
[3]  So find "P3", which is "dc=example,dc=com" with ID 1, and
look for "P1".  But there is no P1, so we stop the process/loop.


Why do we need to look at entryrdn for parent's id? Is it faster?
I have not looked closely into it - so it might not be necessary to 
use entryrdn.  I thought it might be more efficient to use it. If you 
just use id2entry, you have to keep scanning it over and over, and 
starting over every time you need to read the next entry.  Maybe not 
though, maybe you can just "search" it and not have to scan it 
sequentially when trying to find parents and entries.  I'll leave that 
up to you to find out ;-)

Hi, now that I know the context here are some more comments.
If the purpose is to create a useful ldif file, which could eventually 
be used for import then formatting an entry correctly is not enough. 
Order of entries matters: parents need to come before children. We 
already handle this in db2ldif or replication total update.
That said, whenever you write an entry you always have seen the parent 
and could stack the dn with the parentid and createt the dn without 
using the entryrdn index.
You even need not to keep track of all the entry rdsn/dns - only the 
ones with children will be needed later, the presence of "numsubordinates"

identifies a parent.

Last but not least, since I think dbscan is broken for entryrdn, 
investigating and fixing this would also be nice


I mean the same information can be found in id2entry (?). Or this is 
not the case and dbscan does the exact same process you just 
described in order to print "parentid: X" for each entry when you do 
"dbscan -f id2entry.db"?


Thanks again,





___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Trying to understand entryrdn.db

2017-08-02 Thread Ludwig Krispenz


Hi,

I think this is a problem of dbscan, which tries to  prettyprint the 
entryrdn index and seems to loop a bit.

If you do

 db_dump -d a entryrdn.db

you get the raw contents of the file , and you get much fewer records.

Ludwig

On 08/02/2017 05:49 PM, Ilias Stamatis wrote:

Hello,

I would like some help in order to understand entryrdn.db. When I do 
"dbscan -f entryrdn.db" I get something like:


3
  ID: 3; RDN: "ou=Groups"; NRDN: "ou=groups"

C3
ID: 6; RDN: "cn=Accounting Managers"; NRDN: "cn=accounting managers"

P6
ID: 3; RDN: "ou=Groups"; NRDN: "ou=groups"

I understand that 3 is this entry's ID, C3 means child of entry 3 and 
P6 means parent of entry 6.


What I don't understand however is why those entries are repeated 
again and again. For example " ID: 7; RDN: "cn=HR Managers"; NRDN: 
"cn=hr managers" is repeated about a dozen of times in my entryrdn. 
And I don't mean like a parent, child, or whatever. It is repeated 
lots of time as ID 7 for example (but also many times as C3, etc.).


I attach the complete output of what I get when I run "dbscan -f 
entryrdn.db", in order to demonstrate what I mean (my db contains 
almost default entries only).


So my question is; how is this database filled?

Thank you very much,
Ilias


___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: ticket 49334: backup fails if changelog is enabled

2017-07-28 Thread Ludwig Krispenz


https://pagure.io/389-ds-base/issue/49334
https://pagure.io/389-ds-base/issue/raw/files/ce2d76220df07c7142ec745496ebb763f28e174f5a698f4948f770b6417c1ae8-0001-Ticket-49334-fix-backup-restore-if-changelog-exists.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] please review: ticket 49091 - remove changelog semaphore

2017-07-18 Thread Ludwig Krispenz


https://pagure.io/389-ds-base/issue/49091

https://pagure.io/389-ds-base/issue/raw/5a346dfe553aebcc5d3a637200659ee42c0b7aff83c75ba5bdb39fbc22a8ac80-0001-remove-usage-of-changelog-semaphore.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review: ticket 49287 - csn pending lists don't work across multiple backends

2017-07-14 Thread Ludwig Krispenz

here is a revised patch, integrating Williams comments and a 
contribution by Thierry


https://pagure.io/389-ds-base/issue/raw/6d855f63e2e6968692eb06332c322aedbdc3e528232e63881344077864be75ec-0001-Ticket-49287-v3-extend-csnpl-handling-to-multiple-ba.patch

On 07/05/2017 01:02 PM, Ludwig Krispenz wrote:


ticket:
https://pagure.io/389-ds-base/issue/49287

design:
http://www.port389.org/docs/389ds/design/csn-pending-lists-and-ruv-update.html 



fix:
https://pagure.io/389-ds-base/issue/raw/7accb768b1e06a3623c2674daf8b0108daf594b72887014bf4c4831c06b59eeb-0001-Ticket-49287-extend-csnpl-handling-to-multiple-backe.patch 



ci-test:
https://pagure.io/389-ds-base/issue/raw/7077a3becff06e5936b34ea4a6bf0b83127d25134fd3a2a8ba06ba1b4c7cfe45-0001-test-for-pending-list-handling-across-multiple-backe.patch 





--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Implementing Referential Integrity by using a queue instead of a file

2017-07-07 Thread Ludwig Krispenz



On 07/07/2017 03:17 PM, Mark Reynolds wrote:



On 07/07/2017 04:44 AM, Ludwig Krispenz wrote:


On 07/07/2017 07:10 AM, William Brown wrote:

Any thoughts or objections on the above would be welcome.

The only problem with going to a queue is if the server goes down
unexpectedly.  In such a case those RI updates would be lost.

We already have this issue because there is a delay between the change
to the object and the log being sync() to disk. So we can already lose
changes here. TBH the only fix is ot remove the async model. I actually
question why we still need async/delay processing of the refint
plugin ...

Historically speaking, a long time ago, we used to see high CPU when the
RI plugin was engaged.  Setting the delay to 1 second, and allowing the
log thread to do the work, improved performance.  Of course this is now
obsolete with the betxn plugin model and other improvements, but I
wanted to share why the feature even existed.

I guess that would be related to internal op searches / lack of
indexing. These days it's not as big of an issue.

boldly said. How do you know, did you verify it ?
Sorry I didn't realize there was still an issue here.  Back in NDS 
3.12 (maybe early 4.0) this was a very common problem (I personally 
handled countless cases on it), but I have not seen it reported since 
then.  If it's still useful, then we should definitely keep it, and 
moving to a queue is fine with me if it really adds value.  I really 
thought the log/delay was obsolete and no one used it anymore.
well, the latest issue I remember was on RHEL6.x with 1.2.10/11, but if 
we didn't hear it could als very well mean that it is used by 
recommendation of support.
That is why I asked if it is confirmed that there is no more problem, as 
far as I know we did only change configuration, nothing in the plugin 
itself.
Why do we want to fix soemthing which is not broken ? The delay option 
is there forever, some deployments are using it, otehrs use the default. 
And I would als say any effort to optimize it by using a queue is not 
justified, nobody complained about peformance in delay mode.


I would also not move it away from betxn, if someone uses the delay 
option, the update is outside the txn, that is a deliberate decision. 
But for all the deployments using the default delay=0 it has to stay 
where it is.


It still breaks the be txn model, so moving it back to a plain postop 
plugin is the right move if we continue to use the delay.  In that 
case, should we make the delay the default? If it's a still a common 
problem as you said, then lets just apply the "fix/delay" out of the box.
we have seen many customer issues with refereint which were resolved 
by making it async, just removing this option without proof of a 
better solution is no good.
I also am not sure if we need to tie anything into the betxn. There 
are operations, which, in my opinion, can be delayed, even redone by 
fixup tasks, so it is not necessary to have it in one txn, and if the 
option is there to delay it if you want, we should not take it away

So, lets open a ticket to remove delayed processing mode then?

you can, but I will oppose to implement it :-)



___
389-devel mailing list --389-devel@lists.fedoraproject.org
To unsubscribe send an email to389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander


___
389-devel mailing list --389-devel@lists.fedoraproject.org
To unsubscribe send an email to389-devel-le...@lists.fedoraproject.org




--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Implementing Referential Integrity by using a queue instead of a file

2017-07-07 Thread Ludwig Krispenz



On 07/07/2017 10:44 AM, Ludwig Krispenz wrote:


On 07/07/2017 07:10 AM, William Brown wrote:

Any thoughts or objections on the above would be welcome.

The only problem with going to a queue is if the server goes down
unexpectedly.  In such a case those RI updates would be lost.

We already have this issue because there is a delay between the change
to the object and the log being sync() to disk. So we can already lose
changes here. TBH the only fix is ot remove the async model. I actually
question why we still need async/delay processing of the refint
plugin ...

Historically speaking, a long time ago, we used to see high CPU when the
RI plugin was engaged.  Setting the delay to 1 second, and allowing the
log thread to do the work, improved performance.  Of course this is now
obsolete with the betxn plugin model and other improvements, but I
wanted to share why the feature even existed.

I guess that would be related to internal op searches / lack of
indexing. These days it's not as big of an issue.

boldly said. How do you know, did you verify it ?
we have seen many customer issues with refereint which were resolved 
by making it async, just removing this option without proof of a 
better solution is no good.
I also am not sure if we need to tie anything into the betxn. There 
are operations, which, in my opinion, can be delayed, even redone by 
fixup tasks, so it is not necessary to have it in one txn, and if the 
option is there to delay it if you want, we should not take it away

So, lets open a ticket to remove delayed processing mode then?

you can, but I will oppose to implement it :-)
I would even go and suggest to implement the delay feature for memberof, 
like a continuous fixup task.



___
389-devel mailing list --389-devel@lists.fedoraproject.org
To unsubscribe send an email to389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander


___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Implementing Referential Integrity by using a queue instead of a file

2017-07-07 Thread Ludwig Krispenz



On 07/07/2017 07:10 AM, William Brown wrote:

Any thoughts or objections on the above would be welcome.

The only problem with going to a queue is if the server goes down
unexpectedly.  In such a case those RI updates would be lost.

We already have this issue because there is a delay between the change
to the object and the log being sync() to disk. So we can already lose
changes here. TBH the only fix is ot remove the async model. I actually
question why we still need async/delay processing of the refint
plugin ...

Historically speaking, a long time ago, we used to see high CPU when the
RI plugin was engaged.  Setting the delay to 1 second, and allowing the
log thread to do the work, improved performance.  Of course this is now
obsolete with the betxn plugin model and other improvements, but I
wanted to share why the feature even existed.

I guess that would be related to internal op searches / lack of
indexing. These days it's not as big of an issue.

boldly said. How do you know, did you verify it ?
we have seen many customer issues with refereint which were resolved by 
making it async, just removing this option without proof of a better 
solution is no good.
I also am not sure if we need to tie anything into the betxn. There are 
operations, which, in my opinion, can be delayed, even redone by fixup 
tasks, so it is not necessary to have it in one txn, and if the option 
is there to delay it if you want, we should not take it away


So, lets open a ticket to remove delayed processing mode then?

you can, but I will oppose to implement it :-)




___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: ticket 49287 - csn pending lists don't work across multiple backends

2017-07-05 Thread Ludwig Krispenz



ticket:
https://pagure.io/389-ds-base/issue/49287

design:
http://www.port389.org/docs/389ds/design/csn-pending-lists-and-ruv-update.html

fix:
https://pagure.io/389-ds-base/issue/raw/7accb768b1e06a3623c2674daf8b0108daf594b72887014bf4c4831c06b59eeb-0001-Ticket-49287-extend-csnpl-handling-to-multiple-backe.patch

ci-test:
https://pagure.io/389-ds-base/issue/raw/7077a3becff06e5936b34ea4a6bf0b83127d25134fd3a2a8ba06ba1b4c7cfe45-0001-test-for-pending-list-handling-across-multiple-backe.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Ticket 49043

2017-06-09 Thread Ludwig Krispenz


Hi everybody,

here is the result of my work on replication conflicts. I would like you 
to review and comment. I know that given the complexity of the problem 
and the volume of teh patches this is not an easy task - I'm sure there 
is need for further clarification and correction, but I think I reached 
a state where it handles the most important scenarios and more, and does 
not break anything.


First there is the design doc:

https://docs.google.com/a/redhat.com/document/d/1Nv8Y7Lc6E2inSKBTVWvI7eztAFluAv67fFhzDzb0bAU/edit?usp=sharing

unfortunately I failed to convert to markup so far, so it is not yet 
available on teh 389 wiki, but it will be soon. If anybody has a problem 
with the link please contact me and I'll send a copy.


Next there are two test suites and a script to verify consistency of the 
database after running the tests (this consistency check needs to go 
into rth etests themselve, started, but not completed):


https://pagure.io/389-ds-base/issue/raw/00571921ea43decbc7d643449783871aefe323fd3c314427161fb228dc22a463-ticket49043_1_test.py
https://pagure.io/389-ds-base/issue/raw/d67802efd3d3623d2ec5cda2de79b395c4905c85aab6ff4ff78b4d876d20ccfc-ticket49043_2_test.py
https://pagure.io/389-ds-base/issue/raw/885ee5b2aa7c5bb54dcbb831bf312e8274d1636c59d261c9fec62fa91d1d4ec7-check-db.sh

An then the patches:
A few patches were needed and tracked in other tickets:
https://pagure.io/389-ds-base/issue/raw/d3e4d156c3c40743957388be3e5eb8142d3fed7a82f5f10dce67f2e516480fac-0001-fix-for-ticket-49161.patch
https://pagure.io/389-ds-base/issue/raw/9940ede75ff7e8a560903e0d803ba0f0c84f252443ec37b2879e72c598949077-0001-Ticket-49050-make-objectclass-ldapsubentry-effective.patch

and then the conflict patches themselves, they are split into four 
patches, owing separate development steps, but final review maybe best 
after applying all of them:


https://pagure.io/389-ds-base/issue/raw/909764b4533245bfed8f7620ba4df67b761c3ec8f2ce36b4801b04eec582567f-0001-ticket-49043-part1-manage-replication-conflicts.patch
https://pagure.io/389-ds-base/issue/raw/d89d2dcb9d7e2bf1af832de22c0db43a0bc5551d5315119600a4aa76c8c6d47f-0002-ticket-49043-part2-factor-out-mmr-repl-plugin-from-g.patch
https://pagure.io/389-ds-base/issue/raw/158ac556e45e5f44a08d6f2d7e3a650cd3865b0bd02605a4bbaa416ea20cef3f-0003-ticket-49043-part3-hanle-complex-scenarios.patch
https://pagure.io/389-ds-base/issue/raw/77b134c6f3595ceff13f15100cfad7268eeaecdb41250bb2539595079bba8886-0004-ticket-49043-part4-fix-errors-and-leaks-reported-by-.patch

and then there is a memeory leak which was already there but exposed by 
my changes:

https://pagure.io/389-ds-base/issue/raw/5ddeab0c189edfc7571a281a8ae80053f5c9d188b4c0feda74ec6822622f8259-0001-Ticket-49285-memory-leak-when-resurrecting-tombstone.patch

Regards,
Ludwig

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] please review: ticket 49238 : AddressSanitizer: heap-use-after-free in libreplication

2017-05-03 Thread Ludwig Krispenz


https://pagure.io/389-ds-base/issue/49238

https://pagure.io/389-ds-base/issue/raw/files/bd73120782642da0733daa91ab812c1765912ccc52ff1f017aaa852b5480aca4-0001-ticket-49238-AddressSanitizer-heap-use-after-free-in.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Q. on lib389 Re: Please review: #49121 ns-slapd crashes in ldif_sput due to the output buf size is less than the real size.

2017-02-13 Thread Ludwig Krispenz


 Well, I'm not at the position to insist anything any more here :p,

your comments and suggestions and requests are still very welcome and 
needed, independent of your "position". For me it is such a pity to see 
you leaving the team and I hope you will still find some time to help us 
and give us your valuable comments - I am preparing  a doc about 
replication conflicts and would really appreciate if you could 
participate in a review.


Regards,
Ludwig

PS: how is your cat, did the therapy help ?

I'm not sure about the check for sbin dir though,
because it shouldn't matter what we use. Is there a reason you need to
check for root with just those dirs? Is it related to where the DS
instance is from (prefix build vs rpm)
I'm running the test against the server installed in $PREFIX as my 
account.  I hope it's kept supported.

Thanks,
--noriko

2) if they look fine, is it ok to push them to fedorahosted git repository?

I would ask Mark this, he knows the most about the situation.




___
389-devel mailing list --389-devel@lists.fedoraproject.org
To unsubscribe send an email to389-devel-le...@lists.fedoraproject.org





___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: ticket 49008 - aborted operation can leave RUV in incorrect state

2017-01-17 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/49008

https://fedorahosted.org/389/attachment/ticket/49008/ticket49008_test.py

https://fedorahosted.org/389/attachment/ticket/49008/0001-Ticket-49008-aborted-operation-can-leave-RUV-in-inco.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Extending the pblock

2016-11-21 Thread Ludwig Krispenz



On 11/21/2016 02:17 AM, William Brown wrote:

Hi,

I have to add some new getters to pblock.c, but I think we should talk
about how to add these.

Right now, we have a nearly ~2000 line case switch statement in
slapi_pblock_get(pb, TYPE, *void). No matter how we cut it, this is
pretty insane. We have huge amounts of defines in slapi-plugin.h for
this, and we can't expose those to other langs (python, rust), because
they are in a C header file.

As well, case switch statements that large, at some point, it's going to
be inefficient to access.

I think that we should try to break this up, it's just a bit much at
this point.

So for new additions I would like to use:

slapi_pblock_get_(pblock, out) {
 *out = pb->item
}

This is better, as the function is exported in the .so, it's accessible
to other languages, it's simpler to implement, we stop needing more
macro definitions.

It also has performance benefits, as these functions can be inlined by
GCC, the function lookup is likely quicker than the case switch, it's
also easier to set breakpoints on for debugging.

So for now, I want to make new additions to pblock.c in this form.

One day in the future we can tackle breaking pblock.c down from the
current case switch style, but it is not this day.

What do we think? Are we happy for me to use this style for new
additions?
I'm happy with this style of access, but I am not sure if I'm happy with 
new additions to the pblock :-)




___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Design Doc: Automatic server tuning by default

2016-11-04 Thread Ludwig Krispenz



On 11/04/2016 06:51 AM, William Brown wrote:

http://www.port389.org/docs/389ds/design/autotuning.html

I would like to hear discussion on this topic.

thread number:
 independent of number of cpus I would have a default minmum number of 
threads,
your test result for reduced thread number is with clients quickly 
handling responses and short operations.
 But if some threads are serving lazy clients or do database access and 
have to wait, you can quickly run out of threads handling new ops


entry cache:
you should not only take the available memory into account but also the 
size of the database, it doesn't make sense to blow up the cache and its 
associated data (eg hashtables) for a small database just because the 
memory is there




___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review design proposal for tickets 47986 and 48976

2016-10-25 Thread Ludwig Krispenz


Thanks,
I may have missed it, but your suggestion about configuring and 
behaviour sounds good.  I will update my doc.


Regards,
Ludwig

On 10/12/2016 05:19 PM, thierry bordaz wrote:

Hello,

I would think of two options

  * If admin decides to switch to backend, it should not be prevented
and the backend moves to 'backend'
  * periodic (hourly) checking (IMHO not configurable and always run),
checking being the same mechanism as 'auto'
  o in-sync->backend
  o not-in-sync it keeps referral-on-update

I think that the delay option is not necessary. If periodic checking 
fails to move to referral-on-update, it will log a msg saying which 
consumer knows higher csn and it will be the admin task to make sure 
to push those updates.


For internal operation, I do not think to any simple solution. The 
mechanism in your design is a real progress from what is now. Let's 
wait for CU cases to see if we need to also address internal ops.


regards
thierry



On 10/07/2016 05:58 PM, Ludwig Krispenz wrote:
there is a problem not yet covered in the proposal: setting the 
backend to "referral-on-update" until the topology is in sync 
prevents to ealry client updates, but what to do about internal 
updates, eg passwordpolicy attributes.


I have a wild idea, but maybe someone  has a suggestion on how to 
handle this


thanks,
Ludwig

On 10/05/2016 05:51 PM, Ludwig Krispenz wrote:


On 09/30/2016 02:15 AM, Noriko Hosoi wrote:

Hi Ludwig,

On 09/29/2016 05:43 AM, Ludwig Krispenz wrote:

This is the initial proposal, thanks for your feedback

http://www.port389.org/docs/389ds/design/delay-accepting-updates-after-init.html 




Please help me understanding the design...

I'm having a bit hard time to figure out the 
relationship/dependency among these 3 config parameters.


sorry if I was not clear enough, I will update the doc, but let me 
try to explain here


nsslapd-replica-accept-updates-state: on/off
nsslapd-replica-accept-updates-delay: -1/0/n
nsslapd-replica-accept-updates-auto: on/off

Are they independent or dependent?  Do they take any combinations 
-- 2 * 3 * 2 == 12.



no. the primary parameter is: nsslapd-replica-accept-updates-state
If it is off, the other determine when it should be set to on again 
(without an explicite change by an admin).

if it is on, the other two will not be used

independent of auto on/off the "delay" defines if(>=0) the state 
will be reset to on and when


the "auto" param determines if the server should in the defined 
"delay" it should try to detect if it is in sync and switch to "on" 
earlier.


There are 12 different behaviors?  (assuming n for -delay is one 
case :)


What is your recommendation to the customers?  I mean, what is the 
default setting?


that is a good question, there is the option to choose the default 
by what is "my" recommendation (auto: on, delay: n) or what is 
backward compatible (no change in default behaviour: auto off, delay: 0)


  For instance, if -auto is "on", when an online init is executed 
on the master, the scenario is automatically kicked in?


Thanks,
--noriko





||


___
389-devel mailing list --389-devel@lists.fedoraproject.org
To unsubscribe send an email to389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander


___
389-devel mailing list --389-devel@lists.fedoraproject.org
To unsubscribe send an email to389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander


___
389-devel mailing list --389-devel@lists.fedoraproject.org
To unsubscribe send an email to389-devel-le...@lists.fedoraproject.org




___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Ticket #48133 v2 - Non tombstone entry which dn starting with "nsuniqueid=...," cannot be deleted

2016-10-21 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/48133

https://fedorahosted.org/389/attachment/ticket/48133/0001-Ticket-48133-v2-Non-tombstone-entry-which-dn-startin.patch


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Ticket 49009: trace args debug logging must be more restrictive

2016-10-20 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/49009

https://fedorahosted.org/389/attachment/ticket/49009/0001-Ticket-49009-args-debug-logging-must-be-more-restric.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review design proposal for tickets 47986 and 48976

2016-10-07 Thread Ludwig Krispenz

there is a problem not yet covered in the proposal: setting the backend 
to "referral-on-update" until the topology is in sync prevents to ealry 
client updates, but what to do about internal updates, eg passwordpolicy 
attributes.


I have a wild idea, but maybe someone  has a suggestion on how to handle 
this


thanks,
Ludwig

On 10/05/2016 05:51 PM, Ludwig Krispenz wrote:


On 09/30/2016 02:15 AM, Noriko Hosoi wrote:

Hi Ludwig,

On 09/29/2016 05:43 AM, Ludwig Krispenz wrote:

This is the initial proposal, thanks for your feedback

http://www.port389.org/docs/389ds/design/delay-accepting-updates-after-init.html 




Please help me understanding the design...

I'm having a bit hard time to figure out the relationship/dependency 
among these 3 config parameters.


sorry if I was not clear enough, I will update the doc, but let me try 
to explain here


nsslapd-replica-accept-updates-state: on/off
nsslapd-replica-accept-updates-delay: -1/0/n
nsslapd-replica-accept-updates-auto: on/off

Are they independent or dependent?  Do they take any combinations -- 
2 * 3 * 2 == 12.



no. the primary parameter is: nsslapd-replica-accept-updates-state
If it is off, the other determine when it should be set to on again 
(without an explicite change by an admin).

if it is on, the other two will not be used

independent of auto on/off the "delay" defines if(>=0) the state will 
be reset to on and when


the "auto" param determines if the server should in the defined 
"delay" it should try to detect if it is in sync and switch to "on" 
earlier.



There are 12 different behaviors?  (assuming n for -delay is one case :)

What is your recommendation to the customers?  I mean, what is the 
default setting?


that is a good question, there is the option to choose the default by 
what is "my" recommendation (auto: on, delay: n) or what is backward 
compatible (no change in default behaviour: auto off, delay: 0)


  For instance, if -auto is "on", when an online init is executed on 
the master, the scenario is automatically kicked in?


Thanks,
--noriko





||


___
389-devel mailing list --389-devel@lists.fedoraproject.org
To unsubscribe send an email to389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander


___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review design proposal for tickets 47986 and 48976

2016-10-05 Thread Ludwig Krispenz



On 09/30/2016 02:15 AM, Noriko Hosoi wrote:

Hi Ludwig,

On 09/29/2016 05:43 AM, Ludwig Krispenz wrote:

This is the initial proposal, thanks for your feedback

http://www.port389.org/docs/389ds/design/delay-accepting-updates-after-init.html 




Please help me understanding the design...

I'm having a bit hard time to figure out the relationship/dependency 
among these 3 config parameters.


sorry if I was not clear enough, I will update the doc, but let me try 
to explain here


nsslapd-replica-accept-updates-state: on/off
nsslapd-replica-accept-updates-delay: -1/0/n
nsslapd-replica-accept-updates-auto: on/off

Are they independent or dependent?  Do they take any combinations -- 2 
* 3 * 2 == 12.



no. the primary parameter is: nsslapd-replica-accept-updates-state
If it is off, the other determine when it should be set to on again 
(without an explicite change by an admin).

if it is on, the other two will not be used

independent of auto on/off the "delay" defines if(>=0) the state will be 
reset to on and when


the "auto" param determines if the server should in the defined "delay" 
it should try to detect if it is in sync and switch to "on" earlier.



There are 12 different behaviors?  (assuming n for -delay is one case :)

What is your recommendation to the customers?  I mean, what is the 
default setting?


that is a good question, there is the option to choose the default by 
what is "my" recommendation (auto: on, delay: n) or what is backward 
compatible (no change in default behaviour: auto off, delay: 0)


  For instance, if -auto is "on", when an online init is executed on 
the master, the scenario is automatically kicked in?


Thanks,
--noriko





||


___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review design proposal for tickets 47986 and 48976

2016-09-29 Thread Ludwig Krispenz


This is the initial proposal, thanks for your feedback

http://www.port389.org/docs/389ds/design/delay-accepting-updates-after-init.html

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review 48992: Total init may fail if the pushed schema is rejected

2016-09-23 Thread Ludwig Krispenz


Hi Thierry,

the description in the commit is now fine, but given that the choice of 
LDAP_CONSTRAINT_VIOLATION is a bit arbitrary it would be good to have a 
comment where it is set, explaining why this error code was used.
About which error code to choose, if you have to pick one of the errors 
which will allow "keep_going" it is fine, although I think the original 
choice of unwilling to perform was a better match, operations_error or 
ldap_other would, in my opinion, also be good candidates - but they are 
in the wrong category.


Looking at ignore_error_and_keep_going, I am wondering if this partition 
in go|stop is really still correct ? maybe we should investigate this as 
well.


Ludwig

On 09/23/2016 10:08 AM, thierry bordaz wrote:
Thanks Noriko for your review. I updated the patch to give more 
explanation why  the fix is in modify_schema_dse.
I pick up LDAP_CONSTRAINT_VIOLATION in replacement of 
UNWILLING_TO_PERFORM but I have not strong opinion on appropriate 
value of that returned value. In the logic of that fix, it just needs 
to be not fatal regarding ignore_error_and_keep_going.


https://fedorahosted.org/389/attachment/ticket/48992/0002-Ticket-48992-Total-init-may-fail-if-the-pushed-schem.patch 



https://fedorahosted.org/389/ticket/48992
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review: ticket #48766 Replication changelog can incorrectly skip over updates

2016-09-09 Thread Ludwig Krispenz


Hi,

here is the latest correction to the changelog fix, in fact it is 
Thierry's version of teh fix which simplified the logic a bit

https://fedorahosted.org/389/attachment/ticket/48766/0001-PATCH-use-a-consumer-maxcsn-only-as-anchor-if-suppli.patch

Ludwig

On 05/23/2016 03:06 PM, Ludwig Krispenz wrote:

This is the latest version of the "changelog buffer processing" fixes.


https://fedorahosted.org/389/ticket/48766

https://fedorahosted.org/389/attachment/ticket/48766/0001-reworked-clcach-buffer-code-following-design-at-http.patch 



The background for the fix is here, I would like to get feedback on 
this as well to clarify what is unclear
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html 





--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review fix for regression by #48402 patch

2016-08-25 Thread Ludwig Krispenz


https://fedorahosted.org/389/attachment/ticket/48402/0001-provide-backend-dir-in-suffix-template.patch

On 08/24/2016 05:51 PM, Ludwig Krispenz wrote:
This is a heads up, Mark found that the commit of the patch for #48402 
breaks new installs with setup-ds.pl.


I don't understand why, looks like the instance dir and name are not 
set in ldif2db, so I need to investigate.


Either do not use latest master or try the attached temporary patch

Sorry,
Ludwig



--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] commit for #48402 breaks install of new instances

2016-08-24 Thread Ludwig Krispenz

This is a heads up, Mark found that the commit of the patch for #48402 
breaks new installs with setup-ds.pl.


I don't understand why, looks like the instance dir and name are not set 
in ldif2db, so I need to investigate.


Either do not use latest master or try the attached temporary patch

Sorry,
Ludwig

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

>From 30c7685d65ddf05c7956f292016eec628083f6f8 Mon Sep 17 00:00:00 2001
From: Ludwig Krispenz <lkris...@redhat.com>
Date: Wed, 24 Aug 2016 17:34:11 +0200
Subject: [PATCH] handle cases when import file cannot  be generated

---
 ldap/servers/slapd/back-ldbm/dblayer.c | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/ldap/servers/slapd/back-ldbm/dblayer.c b/ldap/servers/slapd/back-ldbm/dblayer.c
index c84196a..f192517 100644
--- a/ldap/servers/slapd/back-ldbm/dblayer.c
+++ b/ldap/servers/slapd/back-ldbm/dblayer.c
@@ -7062,9 +7062,13 @@ error_out:
 static char *
 dblayer_import_file_name(ldbm_instance *inst)
 {
-char *fname = slapi_ch_smprintf("%s/.import_%s",
+char *fname = NULL;
+
+if (inst->inst_parent_dir_name && inst->inst_dir_name) {
+	fname = slapi_ch_smprintf("%s/.import_%s",
 inst->inst_parent_dir_name,
 inst->inst_dir_name);
+}
 return fname;
 }
 
@@ -7097,6 +7101,8 @@ dblayer_import_file_init(ldbm_instance *inst)
 int rc = -1;
 PRFileDesc *prfd = NULL;
 char *fname = dblayer_import_file_name(inst);
+if (NULL == fname) return 0;
+
 rc = dblayer_file_open(fname, PR_RDWR | PR_CREATE_FILE | PR_TRUNCATE, inst->inst_li->li_mode, );
 if (prfd) {
 PR_Close(prfd);
@@ -7126,6 +7132,7 @@ dblayer_import_file_update(ldbm_instance *inst)
 {
 PRFileDesc *prfd;
 char *fname = dblayer_import_file_name(inst);
+if (NULL == fname) return;
 dblayer_file_open(fname, PR_RDWR, inst->inst_li->li_mode, );
 
 if (prfd) {
@@ -7177,6 +7184,8 @@ dblayer_import_file_check(ldbm_instance *inst)
 {
 int rc;
 char *fname = dblayer_import_file_name(inst);
+if (NULL == fname) return 0;
+
 rc = dblayer_file_check(fname, inst->inst_li->li_mode);
 slapi_ch_free_string();
 return rc;
-- 
2.4.3

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review: 48951 dsconf and dsadm foundations

2016-08-19 Thread Ludwig Krispenz


Hi William,

On 08/19/2016 02:22 AM, William Brown wrote:

On Wed, 2016-08-17 at 14:53 +1000, William Brown wrote:

https://fedorahosted.org/389/ticket/48951

https://fedorahosted.org/389/attachment/ticket/48951/0001-Ticket-48951-dsadm-and-dsconf-base-files.patch
https://fedorahosted.org/389/attachment/ticket/48951/0002-Ticket-48951-dsadm-and-dsconf-refactor-installer-cod.patch
https://fedorahosted.org/389/attachment/ticket/48951/0003-Ticket-48951-dsadm-and-dsconf-Installer-options-mana.patch
https://fedorahosted.org/389/attachment/ticket/48951/0004-Ticket-48951-dsadm-and-dsconf-Ability-to-unit-test-t.patch
https://fedorahosted.org/389/attachment/ticket/48951/0005-Ticket-48951-dsadm-and-dsconf-Backend-management-and.patch



As a follow up, here is a design / example document

http://www.port389.org/docs/389ds/design/dsadm-dsconf.html
thanks for this work, it is looking great and is something we were 
really missing.


But of course I have some comments  (and I know I am late).
- The naming dsadm and dsconf, and the split of tasks between them, is 
the same as in Sun/Oracle DSEE, and even if there is probably no legal 
restriction to use them; I'd prefer to have own names for our own tools.
- I'm not convinced of splitting the tasks into two utilities, you will 
have different actions and options for the different 
resources/subcommands anyway, so you could have one for all.
Also, I think, the goal should be to make all actions available local 
and remote, the start/stop/install should be possible remotely via rsh 
or another mechanism as long as the utilities are available on the 
target machine, so I propose one dsmanage or 389manage
- could this be made interactive ? run the command, providing some or 
none options and then have a shell like env

dsmanage
>>> help
.. connect
.. create-x
>>> connect -h 
... replica-enable 
?


More details to come.



--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Please review: Ticket 48954 - replication fails because anchorcsn cannot be found

2016-08-12 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/48954

https://fedorahosted.org/389/attachment/ticket/48954/0001-Ticket-48954-replication-fails-because-anchorcsn-can.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Replication after full online init

2016-07-06 Thread Ludwig Krispenz

Hi Noriko,

I have test scenario for not correctly handled modrdns during total init:

- have a database with n entries, n large enough tthat total init takes
long enough to be able to apply an update while it is running

- add two entries
-- n+1: cn=child,$SUFFIX
-- n+2: cn=parent,$SUFFIX
both have the same parentid and n+2 will be replayed after n+1
- start total update
- do a modrdn
cn=child,$SUFFIX
changetype: modrdn
newrdn: cn=child
newsuperior: cn=parent,$SUFFIX

now cn=child,cn=paren,$SUFFIX will be sent before its parent

I do not know how we can fix this, I think it is a corner case, but we
should keep it in mind

Ludwig

On 06/30/2016 11:53 PM, Noriko Hosoi wrote:

On 06/30/2016 12:45 AM, Ludwig Krispenz wrote:

Hi William,

The test case would be something like this?
1. run online init on the supplier.
2. do some operation like move entries against the supplier while the
online init is still running on the consumer.
3. do some operation which depends upon the previous operation done in
the step 2.

4. check the consumer is healthy or not.

Isn't it a timestamp issue from which operation should be replayed
after the total update? Regardless of the way how to fix 48755,
unless the step 2 operation(s) are replayed after the online init is
done, the consumer could get broken/inconsistent?

Thanks,
--noriko
Therfor the update resolution/ entry state resolution on the consumer
side has to handle this, ignore changes already applied and apply new
changes. And it handles it, if there are bugs they have to be fixed.
Now, I am no longer sure if the fix for 48755 handles correctly all
modrdns received after the id list was prepared, the parentid might
change while the total init is on progress.
This brings up my origimal suggestion to handle the modrdn problems
also on the consumer side.

Ludwig

On 06/30/2016 02:34 AM, William Brown wrote:

Hi,

Now thathttps://fedorahosted.org/389/ticket/48755 is merged, I would
like to discuss the way we handle CSN with relation to this master. As
I'm not an expert on this topic, I want to get the input of everyone
about this.

Following this document:
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html

As I understand it, after a full online init, the replica that consumed
the changes does not set it's CSN to match the CSN of the master that
sent the changes.

As a result, after the online init, this causes a number of changes to
be replicated from the sending master to the consumer. These are ignored
by the URP, and we continue.

However, in a number of cases these are *not* ignored, and have caused
us some bugs in replication in the past. We also have some failing
changes that are skipped, which could in certain circumstance lead to
inconsistency in replicas. We have gone to a lot of effort to be able to
skip changes, to handle the case above.

The reason was is that if there was a modrdn performed, and the entry ID
of the entry that was moved was less than the new parent ID, this *had*
to be skipped, so that after the online init the modrdn change was
replayed and applied to the consumer.

Since 48755 which sorts based on the parent ID, this seems to no longer
be an issue. So we don't need to have the master replay it's changelog
out to the consumer, because the consumer is now a literal clone of the
data.

So, is there a reason for us to leave the CSN of the consumer low to
allow this replay to occur? Or can we alter the behaviour of the
consumer to set it's CSN to the CSN of the sending master, so that we
don't need to replay these changes?

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill

[389-devel] Re: Replication after full online init

2016-07-04 Thread Ludwig Krispenz



On 07/04/2016 01:32 AM, William Brown wrote:

It's not the "post init" operations I'm worried about.

It's that operations that were part of the init to the consumer are
replayed from the changelog.

Operations that occurred after the init starts, definitely still need to
be replayed, and this makes sense.

Lets say we have:

1 - insert A
2 - insert ou=B
3 - modrdn A under ou=B
4 - insert C
xx <<-- We start to transmit the data here.

if we start the total update here, the supplier will send its RUV in the
start repl request, it will be set as RUV in the consumer after total
init is complete.
it skips to send the ruv entry

Are you sure? The behaviour that people are claiming to see would
contradict this behaviour.
yes. As I said, with this behaviour and with teh fix for 48755 there is 
still a potential error if the modrdn is done while the online init is 
in progress. So we would have to make the "people claim" more precise 
and investigate

Certainly there have been a number of fixes
in URP related to replaying modrdn's and related changed after an online
init 



Does that make sense?

yes, and I think that is what it is doing now

I don't think it is 
so what do you think the RUV of the consumer is after an online init ? 
it has to be set somehow and it is not random.




--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Replication after full online init

2016-07-01 Thread Ludwig Krispenz



On 07/01/2016 04:08 AM, William Brown wrote:

On Thu, 2016-06-30 at 14:53 -0700, Noriko Hosoi wrote:

On 06/30/2016 12:45 AM, Ludwig Krispenz wrote:

Hi William,

the reason that after  a total init the consumer does not have the
latest state of the supplier RUV and is receiving updates based on the
RUV at start of the total init is independent of the modrdn problem.
When a supplier is performing a total init it is still accepting
changes, the total init can take a while and there are scenarios where
an entry which is already sent is updated before total init finishes.
We cannot loose these changes.

OK...  Then, RUV needs to be created at the time when the supplier
starts online init?

The test case would be something like this?
1. run online init on the supplier.
2. do some operation like move entries against the supplier while the
online init is still running on the consumer.
3. do some operation which depends upon the previous operation done in
the step 2.
4. check the consumer is healthy or not.

Isn't it a timestamp issue from which operation should be replayed after
the total update?  Regardless of the way how to fix 48755, unless the
step 2 operation(s) are replayed after the online init is done, the
consumer could get broken/inconsistent?


It's not the "post init" operations I'm worried about.

It's that operations that were part of the init to the consumer are
replayed from the changelog.

Operations that occurred after the init starts, definitely still need to
be replayed, and this makes sense.

Lets say we have:

1 - insert A
2 - insert ou=B
3 - modrdn A under ou=B
4 - insert C
xx <<-- We start to transmit the data here.
if we start the total update here, the supplier will send its RUV in the 
start repl request, it will be set as RUV in the consumer after total 
init is complete.

it skips to send the ruv entry

so mods 1-4 will not be replayed

5 - modrdn C


Once the online init is complete, the master replays the log from event
1 -> 5 to the consumer, even though it should now be up to date at
position 4.

it is


Previously we could not guarantee this because in the scenario above, A
would have sorted before ou=B, by would not be able to be applied
because the consumer hadn't seen B yet. So after the init, the consumer
would have B and C, but not A, so we had to replay 1 -> 4 to fix this
up.

So I am suggesting that when we begin the online init we set the RUV of
the consumer to match the CSN of the master at the moment we begin the
transmission of data, so that we only need to replay event 5+, rather
than 1->5+

Does that make sense?

yes, and I think that is what it is doing now





--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Replication after full online init

2016-06-30 Thread Ludwig Krispenz

Hi William,

the reason that after a total init the consumer does not have the
latest state of the supplier RUV and is receiving updates based on the
RUV at start of the total init is independent of the modrdn problem.
When a supplier is performing a total init it is still accepting
changes, the total init can take a while and there are scenarios where
an entry which is already sent is updated before total init finishes. We
cannot loose these changes.
Therfor the update resolution/ entry state resolution on the consumer
side has to handle this, ignore changes already applied and apply new
changes. And it handles it, if there are bugs they have to be fixed.
Now, I am no longer sure if the fix for 48755 handles correctly all
modrdns received after the id list was prepared, the parentid might
change while the total init is on progress.
This brings up my origimal suggestion to handle the modrdn problems also
on the consumer side.

Ludwig

On 06/30/2016 02:34 AM, William Brown wrote:

Hi,

Now that https://fedorahosted.org/389/ticket/48755 is merged, I would
like to discuss the way we handle CSN with relation to this master. As
I'm not an expert on this topic, I want to get the input of everyone
about this.

Following this document:
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html

As I understand it, after a full online init, the replica that consumed
the changes does not set it's CSN to match the CSN of the master that
sent the changes.

As a result, after the online init, this causes a number of changes to
be replicated from the sending master to the consumer. These are ignored
by the URP, and we continue.

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review(2nd): #48402 allow plugins to detect a restore or import

2016-06-28 Thread Ludwig Krispenz

i also addressed the case of online import, which came up in the 
discussion during review.

Can you please review again:
https://fedorahosted.org/389/attachment/ticket/48402/0001-Ticket-48402-v3-allow-plugins-to-detect-a-restore-or.patch
Thanks,
Ludwig

On 02/17/2016 02:12 PM, Ludwig Krispenz wrote:

Hi,

following the suggestions by William, now import stops if the import 
file cannot be written (was the case already for the restore file)

Also logged meaning full message for PR_GetError return values

https://fedorahosted.org/389/attachment/ticket/48402/0001-Ticket-48402-v2-allow-plugins-to-detect-a-restore-or.patch 



On 02/02/2016 04:53 PM, Ludwig Krispenz wrote:

Please review ticket: https://fedorahosted.org/389/ticket/48402

Updated design page: 
http://www.port389.org/docs/389ds/design/detect-startup-after-import-or-restore.html


Patch: 
https://fedorahosted.org/389/attachment/ticket/48402/0001-Ticket-48402-allow-plugins-to-detect-a-restore-or-im.patch

--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org 


--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review (additional fixes): [389 Project] #48755: moving an entry could make the online init fail

2016-06-21 Thread Ludwig Krispenz

On 06/20/2016 05:55 PM, Noriko Hosoi wrote:

On 06/20/2016 02:20 AM, Ludwig Krispenz wrote:
I have a question about your tombstone entry... You are using 
entrydn instead of entryrdn?
no, sorry for the confusion, I was using ldapsearch to get teh 
tombstones, not dbscan
Hmmm, if that's the case, I need to learn why you don't have the 
parent-child relationship in your parentid index... 
yes, but that could be a problem of my instance, I will test again. I 
just had missed that the parentid index is added back for tombstones 
line 907 - 915. In that case I have no more concerns to use only the 
parentid index

Sorry for holding this up,
Ludwig

As I copied, tombstone entry should be in the index... :(
> *dn:* 
nsuniqueid=bd76ad01-322c11e6-a5389989-c3cc84f8,cn=x,ou=People,dc=example,dc=com

> entryid: 30
> parentid: 2

Is that the default configuration for IPA?  Does that mean IPA has 
no move issue?

  27 ldbm_back_delete( Slapi_PBlock *pb )
 781 if(create_tombstone_entry)
 903 if (entryrdn_get_switch()) /* subtree-rename: on */
* 907 /* To maintain tombstonenumsubordinates,**
** 908  * parentid is needed for tombstone, as well. 
*/**
** 913 retval = index_addordel_values_sv(be, 
LDBM_PARENTID_STR,**

** 914 svals, NULL, e->ep_id,**
** 915 BE_INDEX_ADD, );*

But in that case total update is supposed to have no change...

313 repl5_tot_run(Private_Repl_Protocol *prp)
441 if (is_entryrdn) {
// NEW BULK IMPORT
493 } else {
494 /* Original total update */
[...]
525 }

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review (additional fixes): [389 Project] #48755: moving an entry could make the online init fail

2016-06-20 Thread Ludwig Krispenz

On 06/17/2016 09:16 PM, Noriko Hosoi wrote:

On 06/17/2016 08:49 AM, Ludwig Krispenz wrote:

On 06/17/2016 05:31 PM, Noriko Hosoi wrote:

On 06/17/2016 12:17 AM, Ludwig Krispenz wrote:

Hi Noriko,

I still have a doubt on your fix. You now base the entries to be 
sent only on the parentid index, but when creating a tombstone, the 
entry is removed from the parentid index. So you will miss the 
tombstones in the total init.
Well, I don't think a tombstone entry is removed from the parentid 
index...
but in ldbm_back_delete() we remove the entryID from all inexes and 
if create_tombstone_entry add it back to specific indexes

This is an example of a tombstone in id2entry.db

id 16
rdn: nsuniqueid=bd90578a-2f6311e6-9346e70f-8f7f52e7,uid=tuser0
entryid: 16
parentid: 20

This is the corresponding item on parentid.db.  The entry 16 is 
still there as a child of parent id 20...

# dbscan -f parentid.db -k =20 -r -n
=20 3
10 16 19

was dbcache already flushed to disk when you did run dbscan ? I get
dn: 
nsuniqueid=bd76ad01-322c11e6-a5389989-c3cc84f8,cn=x,ou=People,dc=example,dc=com

entryid: 30
parentid: 2

dn: cn=y,ou=People,dc=example,dc=com
entryid: 31
parentid: 2

 dbscan -f /var/lib/dirsrv/slapd-elkris3/db/example/parentid.db -k =2 
-r -n

=2  1
31

I have a question about your tombstone entry...  You are using entrydn 
instead of entryrdn?
no, sorry for the confusion, I was using ldapsearch to get teh 
tombstones, not dbscan
> *dn:* 
nsuniqueid=bd76ad01-322c11e6-a5389989-c3cc84f8,cn=x,ou=People,dc=example,dc=com

> entryid: 30
> parentid: 2

Is that the default configuration for IPA?  Does that mean IPA has no 
move issue?

  27 ldbm_back_delete( Slapi_PBlock *pb )
 781 if(create_tombstone_entry)
 903 if (entryrdn_get_switch()) /* subtree-rename: on */
 907 /* To maintain tombstonenumsubordinates,
 908  * parentid is needed for tombstone, as well. */
 913 retval = index_addordel_values_sv(be, 
LDBM_PARENTID_STR,
 914   svals, 
NULL, e->ep_id,

 915 BE_INDEX_ADD, );

But in that case total update is supposed to have no change...

313 repl5_tot_run(Private_Repl_Protocol *prp)
441 if (is_entryrdn) {
// NEW BULK IMPORT
493 } else {
494 /* Original total update */
[...]
525 }

Thanks,
--noriko

Besides, the tombstone entry is found on the consumer server after 
the total update/bulk import...
# ldapsearch [...]  -b "dc=example,dc=com" 
"(|(objectclass=nstombstone)(objectclass=ldapsubentry))" dn

dn: cn=repl keep alive 1,dc=example,dc=com
dn: 
nsuniqueid=bd90578a-2f6311e6-9346e70f-8f7f52e7,uid=tuser0,ou=ou3,dc=example,dc=com

[...]

But obviously there are still issues, with which it does not pass 
the IPA CI test...  Continue looking into it...

Thanks,
--noriko

Ludwig

On 06/17/2016 03:05 AM, Noriko Hosoi wrote:

https://fedorahosted.org/389/ticket/48755

https://fedorahosted.org/389/attachment/ticket/48755/0001-Ticket-48755-moving-an-entry-could-make-the-online-i.3.patch
git patch file (master) -- additional fix to the server patch

https://fedorahosted.org/389/attachment/ticket/48755/0002-Ticket-48755-CI-test-test-case-for-ticket-48755.3.patch
git patch file (master) -- additional test case to the CI test patch

I also built copr build including the patch:
https://copr.fedorainfracloud.org/coprs/nhosoi/389-ds-base-1.3.5.7.f24/builds/

Martin, could it be possible to use this tentative build in the 
IPA acceptance test?  (I don't want to break IPA again...)

Thanks!
--noriko

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgerich

[389-devel] Re: Please review: ticket #48366 - proxyauth support does not work when bound as directory manager

2016-06-17 Thread Ludwig Krispenz


I added some clarifications to the ticket and here is a lib389 test case

https://fedorahosted.org/389/attachment/ticket/48366/0001-add-testcase-for-ticket-48366-proxyauth-for-root.patch

On 02/16/2016 03:35 PM, Ludwig Krispenz wrote:

https://fedorahosted.org/389/ticket/48366

https://fedorahosted.org/389/attachment/ticket/48366/0001-Ticket-48366-proxyauth-does-not-work-bound-as-direct.patch 


--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review (additional fixes): [389 Project] #48755: moving an entry could make the online init fail

2016-06-17 Thread Ludwig Krispenz



On 06/17/2016 02:36 PM, Martin Babinsky wrote:

On 06/17/2016 08:42 AM, Martin Babinsky wrote:

On 06/17/2016 03:05 AM, Noriko Hosoi wrote:

https://fedorahosted.org/389/ticket/48755

https://fedorahosted.org/389/attachment/ticket/48755/0001-Ticket-48755-moving-an-entry-could-make-the-online-i.3.patch 



git patch file (master) -- additional fix to the server patch

https://fedorahosted.org/389/attachment/ticket/48755/0002-Ticket-48755-CI-test-test-case-for-ticket-48755.3.patch 



git patch file (master) -- additional test case to the CI test patch

I also built copr build including the patch:
https://copr.fedorainfracloud.org/coprs/nhosoi/389-ds-base-1.3.5.7.f24/builds/ 




Martin, could it be possible to use this tentative build in the IPA
acceptance test?  (I don't want to break IPA again...)

Thanks!
--noriko


Hi Noriko,

I will run some of our replication CI tests with your copr build and get
back with results.



Our simple-replication CI test failed with your copr build and master 
branch freeipa (built today).


The previous DS build (389-ds-base-1.3.5.6-1.fc24.x86_64) worked just 
fine.


These are the errors/access logs from the failed test:

https://paste.fedoraproject.org/380434/15539814/
https://paste.fedoraproject.org/380435/61554061/

these seem to be from the replica, do you have the logs from master ?


I will do more investigation next week as we have to prioritize tasks 
heavily due to impending deadlines.




--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Idea to make replication a bit cleaner

2016-06-13 Thread Ludwig Krispenz


Hi German,

you are right that IPA is on the safe side, they maintain the last used 
replicaID and when creating a server instance only a higher replicaid is 
used, also when a server is removed, the removal triggers a cleanallruv, 
either from the script or by the topology plugin (>4.3).
This is because in IPA all server instance creation and removal is 
managed by IPA cammands.


This framework is not there by default for "admion managed" DS 
deployments, and that's what William wants to get improved.


William,
I'm not sure that the scenario you describe is really so bad as you 
think. If  a server is removed and the RID is not cleaned, it's 
component remains in the RUV, but this is just an overhead in examining 
ruvs, but should not block replication to continue. If a new server with 
the old, removed replicaID is installed, the ruv component should be 
reused, the URL replaced, and replication continue as if there haven't 
been updates for this replica ID for a long time. I'm saying "should", 
since there mioght be some cases where the changelog was purged and an 
anchor csn for the old/new replicaid cannot be found. So we need to do 
some tests and it would be good to make this safe.
One option would be to maintain already used replicaids, so at the init 
of a new server there could not only be a check for the same database 
generation, but also for a valid RUV
I think it is difficult to find a trigger for an automated cleanallruv, 
we would have to maintain something like a topology view of the 
deployment, like the topology plugin in IPA does


But it is definitely worth to think about solutions for thi sproblem

Ludwig


On 06/13/2016 10:21 AM, German Parente wrote:

Hi William,

I think this case is covered in IPA. I have never seen a new replica 
added with the same former ID of an old one.


The former ruvs are not cleaned automatically, though, in current 
versions and it's not a very severe issue now. There are also ipa 
commands to list and clean the ruvs.


I have also heard or read that in the dev versions (not still 
delivered), the cleaning is automatic, as you are proposing.


Thanks a lot.

regards,

German.


On Mon, Jun 13, 2016 at 7:21 AM, William Brown > wrote:


Hi,

I was discussing with some staff here in BNE about replication.

It seems a common case is that admins with 2 or 3 servers in MMR
(both DS and IPA) will do this:

* Setup all three masters A, B, C (replica id 1,2,3 respectively)
* Run them for a while in replication
* Remove C from replication
* Delete data, change the system
* Re-add C with the same replica id.

Supposedly this can cause duplicate RUV entries for id 3 in
masters A and B. Of course, this means that replication has all
kinds of insane issues at this point 


On one hand, this is the admins fault. But on the other, we should
handle this. Consider an admin who re-uses an IPA replica
setup file, without running CLEANALLRUV

So, an have some idea for this. Any change to a replication
agreement, should trigger a CLEANALLRUV, before we start the
agreement. This means on our local master we have removed the bad
RUV first, then we can add the RUV of the newly added master
when needed 

What do you think? I think that we must handle this better, and it
should be a non-issue to admins.


We can't prevent an admin from intentionally adding duplicate ID's
to the topology though. So making it so that the ID's are not
admin controlled would prevent this, but I haven't any good ideas
about this  (yet)




--
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


--
389-devel mailing list
389-devel@lists.fedoraproject.org


https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org




--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Please review: #48275 - search returns no entry when OR filter component contains non readable attribute

2016-05-25 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/48275
https://fedorahosted.org/389/attachment/ticket/48275/0001-Ticket-48275-correctly-handle-or-filters-with-compon.patch
https://fedorahosted.org/389/attachment/ticket/48275/0002-testcase-for-ticket-48275.patch

The fix passes the ported TET filter test and the ticket specific test, 
please take your time to review and comment, I'll be out a few days and 
answer whenI'm back


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael 
O'Neill
--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review: ticket #48766 Replication changelog can incorrectly skip over updates

2016-05-25 Thread Ludwig Krispenz

On 05/24/2016 07:00 PM, thierry bordaz wrote:

On 05/24/2016 05:24 PM, Ludwig Krispenz wrote:

On 05/24/2016 04:20 PM, thierry bordaz wrote:

Hi Ludwig,

Thanks for your explanation. The design looks very good. I think it
would be good to put into the code (especially
clcache_adjust_anchorcsn) the reference to the related design paragraph.

There is something I do not understand in clcache_skip_change.
My understanding is that this is the only place where the
consumer_maxcsn is updated.
But there are several conditions, if we decide to skip the update
that the consumer_maxcsn is not updated.

One of them is 'rid == buf->buf_consumer_rid'.
Does that mean that the consumer_maxcsn remains unchanged for the
RID of the consumer ?

the condition is:
if( rid == buf->buf_consumer_rid && buf->buf_ignoreConsumerRID)

so it will only be skipped if we determined that we don't need to
send anything for the consumers own rid

Ok. I get it. thanks.

An other question regarding the update of buf_consumer_ruv.
My understanding is that it contains the initial consumer RUV.
But later it is used in
clcache_refresh_consumer_maxcsns/clcache_refresh_local_maxcsn to fill
consumer_maxcsn.

consumer_maxcsn are updated with the non skipped updates to reflect
the current status of the consumer.
But when consumer_maxcsns/clcache_refresh_local_maxcsn are called my
understanding is that consumer_maxcsn are reset with buf_consumer_ruv
(initial consumer RUV).

Do I get it right ?

no :-) At least I think I have implemented it differently.
The consumer_maxcsn is set in /clcache_refresh_consumer_maxcsns(), which
is only called at the first buffer load, not at the relaod.
In //clcache_refresh_local_maxcsns it is only set if we have to add a
RID to the cscb list, but then no change for this rid was sent before.

thanks
thierry

On 05/24/2016 09:22 AM, Ludwig Krispenz wrote:

Hi,

On 05/23/2016 06:29 PM, thierry bordaz wrote:

On 05/23/2016 03:06 PM, Ludwig Krispenz wrote:
This is the latest version of the "changelog buffer processing"
fixes.

https://fedorahosted.org/389/ticket/48766

https://fedorahosted.org/389/attachment/ticket/48766/0001-reworked-clcach-buffer-code-following-design-at-http.patch

The background for the fix is here, I would like to get feedback
on this as well to clarify what is unclear
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html

Hello Ludwig,

I have not yet reviewed the patch. I was looking at the design.

Regarding your note:
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html#special-case-rid-of-the-consumer-in-the-current-replication-session.

If you refer to this part:

Special case: RID of the consumer in the current
replication session

If the consumer in the replication session is also a master its RID
will be contained at least in the consumerRUV. If it is also in the
supplier RUV the question is if it should be considered in the
decision if updates should be sent. Normally a master has the
latest changes applied to itself, so there would be no need to
check and send updates for its RID. But there can be scenarios
where this is not the case: if the consumer has been restored from
an older backup the latest csn for its own RID might be older than
changes available on other servers.

|NOTE: The current implementation ignores anchorCSNs based on the consumer RID.
If, by chance, the anchor csn used is older than this csn, the changes will be
sent, but they also ca nbe lost.
|

this referres to the "current" implementation before the fix, the
doc started as a post-design doc, and it shoul dbe correctedd
with the fix the if the supplier has newer changes for the
consumerRID than the consumer it will be reflected in the anchor
csn calculation.

It is said that the anchorCSN will not be the from the
consumerRID. What is the mechanism that guaranty that the consumer
will receive all the updates it was the originator ?

thanks
thierry
--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael
O'Neill

--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
389-devel mailing l

[389-devel] Re: Please review: ticket #48766 Replication changelog can incorrectly skip over updates

2016-05-24 Thread Ludwig Krispenz



On 05/24/2016 04:20 PM, thierry bordaz wrote:

Hi Ludwig,

Thanks for your explanation. The design looks very good. I think it 
would be good to put into the code (especially 
clcache_adjust_anchorcsn) the reference to the related design paragraph.


There is something I do not understand in clcache_skip_change.
My understanding is that this is the only place where the 
consumer_maxcsn is updated.
But there are several conditions, if we decide to skip the update that 
the consumer_maxcsn is not updated.

One of them is 'rid == buf->buf_consumer_rid'.
Does that mean that the consumer_maxcsn remains unchanged for the RID 
of the consumer ?


the condition is:
if( rid == buf->buf_consumer_rid && buf->buf_ignoreConsumerRID)

so it will only be skipped if we determined that we don't need to send 
anything for the consumers own rid




thanks
thierry




On 05/24/2016 09:22 AM, Ludwig Krispenz wrote:

Hi,

On 05/23/2016 06:29 PM, thierry bordaz wrote:



On 05/23/2016 03:06 PM, Ludwig Krispenz wrote:

This is the latest version of the "changelog buffer processing" fixes.


https://fedorahosted.org/389/ticket/48766

https://fedorahosted.org/389/attachment/ticket/48766/0001-reworked-clcach-buffer-code-following-design-at-http.patch 



The background for the fix is here, I would like to get feedback on 
this as well to clarify what is unclear
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html 





Hello Ludwig,

I have not yet reviewed the patch. I was looking at the design.

Regarding your note: 
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html#special-case-rid-of-the-consumer-in-the-current-replication-session.

If you refer to this part:


  Special case: RID of the consumer in the current
  replication session

If the consumer in the replication session is also a master its RID 
will be contained at least in the consumerRUV. If it is also in the 
supplier RUV the question is if it should be considered in the 
decision if updates should be sent. Normally a master has the latest 
changes applied to itself, so there would be no need to check and 
send updates for its RID. But there can be scenarios where this is 
not the case: if the consumer has been restored from an older backup 
the latest csn for its own RID might be older than changes available 
on other servers.


|NOTE: The current implementation ignores anchorCSNs based on the consumer RID. 
If, by chance, the anchor csn used is older than this csn, the changes will be 
sent, but they also ca nbe lost.
|

this referres to the "current" implementation before the fix, the doc 
started as a post-design doc, and it shoul dbe correctedd
with the fix the if the supplier has newer changes for the 
consumerRID than the consumer it will be reflected in the anchor csn 
calculation.





It is said that the anchorCSN will not be the from the consumerRID. 
What is the mechanism that guaranty that the consumer will receive 
all the updates it was the originator ?


thanks
thierry
--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org 



--
Red Hat GmbH,http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael 
O'Neill


--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org




--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael 
O'Neill

--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Initial review: ticket #48069 - CI test - filter

2016-05-24 Thread Ludwig Krispenz


Hi,
here is my current state of porting the TET filter suite,
https://fedorahosted.org/389/ticket/48069

After getting there I think it can probably be simplified, but before 
just going to change things I like to get your feedback and suggestions 
for improvements

Some issues I have:
- in the original TET suite there are tons of files data, messages, 
errors  I tried to integrate these into the tests and not rely on 
toom many files, but there are stilll two ldif fles in the data 
directory - what is best practice ?
- the MR filter tests in filter.sh and vfilter.sh are forced to PASS. I 
omitted them and have created a separate ticket: 
https://fedorahosted.org/389/ticket/48851


https://fedorahosted.org/389/attachment/ticket/48069/0001-add-costants-for-filter-test-suite.patch
https://fedorahosted.org/389/attachment/ticket/48069/0001-add-test-data-for-filter-test-suite.patch
https://fedorahosted.org/389/attachment/ticket/48069/0001-initial-port-of-TET-filter-tests.patch

https://fedorahosted.org/389/attachment/ticket/48069/0001-add-test-data-for-filter-test-suite.patch--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael 
O'Neill
--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review: ticket #48766 Replication changelog can incorrectly skip over updates

2016-05-24 Thread Ludwig Krispenz


Hi,

On 05/23/2016 06:29 PM, thierry bordaz wrote:



On 05/23/2016 03:06 PM, Ludwig Krispenz wrote:

This is the latest version of the "changelog buffer processing" fixes.


https://fedorahosted.org/389/ticket/48766

https://fedorahosted.org/389/attachment/ticket/48766/0001-reworked-clcach-buffer-code-following-design-at-http.patch 



The background for the fix is here, I would like to get feedback on 
this as well to clarify what is unclear
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html 





Hello Ludwig,

I have not yet reviewed the patch. I was looking at the design.

Regarding your note: 
http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html#special-case-rid-of-the-consumer-in-the-current-replication-session.

If you refer to this part:


 Special case: RID of the consumer in the current replication session

If the consumer in the replication session is also a master its RID will 
be contained at least in the consumerRUV. If it is also in the supplier 
RUV the question is if it should be considered in the decision if 
updates should be sent. Normally a master has the latest changes applied 
to itself, so there would be no need to check and send updates for its 
RID. But there can be scenarios where this is not the case: if the 
consumer has been restored from an older backup the latest csn for its 
own RID might be older than changes available on other servers.


|NOTE: The current implementation ignores anchorCSNs based on the consumer RID. 
If, by chance, the anchor csn used is older than this csn, the changes will be 
sent, but they also ca nbe lost.
|


this referres to the "current" implementation before the fix, the doc 
started as a post-design doc, and it shoul dbe correctedd
with the fix the if the supplier has newer changes for the consumerRID 
than the consumer it will be reflected in the anchor csn calculation.





It is said that the anchorCSN will not be the from the consumerRID. 
What is the mechanism that guaranty that the consumer will receive all 
the updates it was the originator ?


thanks
thierry
--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org 



--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael 
O'Neill

--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Please review: ticket #48766 Replication changelog can incorrectly skip over updates

2016-05-23 Thread Ludwig Krispenz


This is the latest version of the "changelog buffer processing" fixes.


https://fedorahosted.org/389/ticket/48766

https://fedorahosted.org/389/attachment/ticket/48766/0001-reworked-clcach-buffer-code-following-design-at-http.patch

The background for the fix is here, I would like to get feedback on this 
as well to clarify what is unclear

http://www.port389.org/docs/389ds/design/changelog-processing-in-repl-state-sending-updates.html

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael 
O'Neill
--
389-devel mailing list
389-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Please review: #48759: no plugin calls in tombstone purging

2016-03-15 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/48759

https://fedorahosted.org/389/attachment/ticket/48759/0001-add-testcase-for-ticket-48759.patch

https://fedorahosted.org/389/attachment/ticket/48759/0001-Ticket-48759-no-plugin-calls-in-tombstone-purging.patch

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael 
O'Neill
--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Please review(2nd): #48402 allow plugins to detect a restore or import

2016-02-17 Thread Ludwig Krispenz


Hi,

following the suggestions by William, now import stops if the import 
file cannot be written (was the case already for the restore file)

Also logged meaning full message for PR_GetError return values

https://fedorahosted.org/389/attachment/ticket/48402/0001-Ticket-48402-v2-allow-plugins-to-detect-a-restore-or.patch

On 02/02/2016 04:53 PM, Ludwig Krispenz wrote:

Please review ticket: https://fedorahosted.org/389/ticket/48402

Updated design page: 
http://www.port389.org/docs/389ds/design/detect-startup-after-import-or-restore.html


Patch: 
https://fedorahosted.org/389/attachment/ticket/48402/0001-Ticket-48402-allow-plugins-to-detect-a-restore-or-im.patch

--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: SASL/EXTERNAL bind mech issue

2016-02-16 Thread Ludwig Krispenz

what do you want to achieve, do you want to do client authentication via 
a certificate ?
you have to provide configuration info in ldap.conf or .ldaprc or 
environment variables, so that the openldap libs built with nss can 
access the client certificate, it has to be in a nss database.


with env it is something like this:

export LDAPTLS_CERT=Client-Cert
export LDAPTLS_KEY=/etc/dirsrv/slapd-EXAMPLE.COM/pwdfile.txt
export LDAPTLS_CACERTDIR=/etc/dirsrv/slapd-EXAMPLE.COM

LDAPTLS_CERT is the nickname of a certificate in the certificate database
LDAPTLS_KEY is a file containing the password for the key database


in a config file omit the "LDAP" part of the option name.



and are you running your search against a 389 server ? I think the file 
/etc/openldap/slapd.d/cn\=config.ldif is only relevant for openldap server.


Ludwig

On 02/15/2016 05:12 PM, Simon Pichugin wrote:

Hi team,

I am trying to set up SASL/EXTERNAL binding mechanism.
I perform all actions from our docs (Administration guide)

First, I've set up SSL/TLS on the clean instance:
1) Cert was created and imported
2) Trusted CA cert was imported too
3) cert8.db, key3.db, secmod.db were copied to /etc/openldap/certs/
4) Config was changed to accept SSL/TLS
5) Setup was tested and everything worked perfectly

Then client certificate was created and approved by our CA.

openssl x509 -in client_ds.crt -text
Certificate:
 Data:
 Version: 1 (0x0)
 Serial Number: 16371655739931625967 (0xe333ce279b9c09ef)
 Signature Algorithm: sha256WithRSAEncryption
 Issuer: C=CZ, ST=Moravia, L=Brno, O=Default Company Ltd, OU=Dev, 
CN=Simon
 Validity
 Not Before: Feb 12 13:51:50 2016 GMT
 Not After : Oct 21 13:51:50 2029 GMT
 Subject: C=CZ, L=Default City, O=example.com, CN=simon 
pichugin/emailAddress=spich...@redhat.com

After that certificate was imported to "userCertificate" attr of
our user (I've cut the attr output):

# spichugin, People, example.com
dn: uid=spichugin,ou=People,dc=example,dc=com
mail: spich...@redhat.com
uid: spichugin
givenName: simon
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
sn: pichugin
cn: simon pichugin
userPassword:: e1NTSEF9OVJhbUdER3prOE1JdENObnFJb3
userCertificate:: LS0tLS1CRUdJTiBDRVJUSUZJQ0FU
Next, /etc/dirsrv/slapd-stal/certmap.conf was modified with this contents:
certmap Example o=example.com
Example:DNComps
Example:FilterComps mail,cn
Also tried with this:
certmap Example cn=simon pichugin
Example:DNComps
Example:FilterComps mail,cn

Also I have added "olcTLSVerifyClient: demand" to 
/etc/openldap/slapd.d/cn\=config.ldif

/etc/openldap/ldap.conf contains only "TLS_CACERTDIR /etc/openldap/certs/", the 
rest options is by default

Then I've tested setup with this command:

[spichugi@rhel-ws ~]$ ldapsearch -H ldaps://rhel-ws.brq.redhat.com:636 -b 
"dc=example,dc=com" \
-Y EXTERNAL -U "dn:uid=spichugin,ou=People,dc=example,dc=com" -w Secret123 -d 1
ldap_url_parse_ext(ldaps://rhel-ws.brq.redhat.com:636)
ldap_create
ldap_url_parse_ext(ldaps://rhel-ws.brq.redhat.com:636/??base)
ldap_sasl_interactive_bind: user selected: EXTERNAL
ldap_int_sasl_bind: EXTERNAL
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP rhel-ws.brq.redhat.com:636
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying ::1 636
ldap_pvt_connect: fd: 3 tm: -1 async: 0
attempting to connect:
connect success
TLS: certdb config: configDir='/etc/openldap/certs/' tokenDescription='ldap(0)' 
certPrefix='' keyPrefix='' flags=readOnly
TLS: using moznss security dir /etc/openldap/certs/ prefix .
TLS: certificate 
[CN=rhel-ws.brq.redhat.com,OU=sdfsd,O=qwedasdf,L=VCrno,ST=Alabama,C=US] is valid
TLS certificate verification: subject: 
CN=rhel-ws.brq.redhat.com,OU=sdfsd,O=qwedasdf,L=VCrno,ST=Alabama,C=US, issuer: 
CN=Simon,OU=Dev,O=Default Company Ltd,L=Brno,ST=Moravia,C=CZ, cipher: AES-256, 
security level: high, secret key bits: 256, total key bits: 256, cache hits: 0, 
cache misses: 0, cache not reusable: 0
ldap_int_sasl_open: host=rhel-ws.brq.redhat.com
SASL/EXTERNAL authentication started
ldap_msgfree
ldap_err2string
ldap_sasl_interactive_bind_s: Unknown authentication method (-6)
 additional info: SASL(-4): no mechanism available:
ldap_free_connection 1 1
ldap_send_unbind
ber_flush2: 7 bytes to sd 3
ldap_free_connection: actually freed

Please, if someone has an idea what can be wrong, share it. :)

Thanks,
Simon
--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Please review: ticket #48366 - proxyauth support does not work when bound as directory manager

2016-02-16 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/48366

https://fedorahosted.org/389/attachment/ticket/48366/0001-Ticket-48366-proxyauth-does-not-work-bound-as-direct.patch
--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Please review: #48402 allow plugins to detect a restore or import

2016-02-02 Thread Ludwig Krispenz


Please review ticket: https://fedorahosted.org/389/ticket/48402

Updated design page: 
http://www.port389.org/docs/389ds/design/detect-startup-after-import-or-restore.html


Patch: 
https://fedorahosted.org/389/attachment/ticket/48402/0001-Ticket-48402-allow-plugins-to-detect-a-restore-or-im.patch

--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: Prereview for ticket #48402 , required by ticket #48380 SynRepl does not handle database re-initialization properly

2016-01-14 Thread Ludwig Krispenz



On 01/13/2016 11:59 PM, Noriko Hosoi wrote:

On 01/13/2016 02:34 PM, Mark Reynolds wrote:



On 01/13/2016 10:01 AM, Ludwig Krispenz wrote:
Ticket 48380 requires that sync repl handles database 
reinitializations properly, to be able to determine if cookies are 
presented are valid.
To achieve this plugins need to be able to detcet if the database is 
imported or restored and this is tarcked in ticket 48402.


Before implementing a fix, I would like to get feedback on the 
design: 
http://www.port389.org/docs/389ds/design/detect-startup-after-import-or-restore.html
The design looks good to me.  I really like the use of the "db event" 
file(like the guardian file).

+1

I also think it's nicely designed.

2 questions...  Who deletes the file and at what timing? 
the file should be deleted immediately after it is read and the flag 
set, so the "restore" file at the same time the gurduian file is read 
and the "import" file in ldbm:start_instance

If the file failed to be created or is deleted manually, what happens?
If it fails to be created, it can be at least logged, but then the 
server would not know that it was restored or imported - or, write the 
import/restore file at the beginning of import/restore and if fails,  
then don't do the restore/import.


If someone deletes the file manually, this is at the deleter's own risk


Thanks!
--noriko


Thanks,
Ludwig
--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org 


--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org 


--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Re: design review: don't call plugins for retro changelog

2016-01-14 Thread Ludwig Krispenz



On 01/14/2016 04:08 PM, Mark Reynolds wrote:



On 01/14/2016 09:42 AM, Ludwig Krispenz wrote:
We had many issues with the retro changelog plugin. The main reason 
is that the retro CL is a separate backend and if there is more than 
one regular backend it is easy to run into deadlocks, eg a change in 
backend A triggers and ADD in theh RCL, in the add a plugin might 
want to access backend B, but there was a change in backend B and it 
waits for the RCL lock and both threads are blocked.
All the scenrios so far could be resolved, by scoping the plugins to 
ignore changes in the retro CL, but it is tedious and in my opinion 
operations on the retro changelog should not be seen by plugins at all.
I propose a simple configuration an processing change to allow to 
ignore plugins for specific backends, please have a look at:


http://www.port389.org/docs/389ds/design/exclude-backends-from-plugin-operations.html 

Looks good.  Just to clarify when you say "Define a configuration 
parameter for backend entries:" do you mean "Define a configuration 
parameter for backend *plugins*:"? 

yes, it should be more clear, I mean something like:

dn: cn=changelog,cn=ldbm database,cn=plugins,cn=config

nsslapd-suffix: cn=changelog
nsslapd-cachesize: -1
nsslapd-no-plugins: on
nsslapd-cachememsize: 2097152
nsslapd-readonly: off


Regards,
Ludwig
--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org




--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org


--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Please review: Ticket 48341

2016-01-13 Thread Ludwig Krispenz


Hi,

to have my suggestion discussed by a wider audience, please have a look:

https://fedorahosted.org/389/ticket/48341
https://fedorahosted.org/389/attachment/ticket/48341/0001-Ticket-48341-deadlock-on-connection-mutex.patch

Ludwig
--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

[389-devel] Prereview for ticket #48402 , required by ticket #48380 SynRepl does not handle database re-initialization properly

2016-01-13 Thread Ludwig Krispenz

Ticket 48380 requires that sync repl handles database reinitializations 
properly, to be able to determine if cookies are presented are valid.
To achieve this plugins need to be able to detcet if the database is 
imported or restored and this is tarcked in ticket 48402.


Before implementing a fix, I would like to get feedback on the design: 
http://www.port389.org/docs/389ds/design/detect-startup-after-import-or-restore.html


Thanks,
Ludwig
--
389-devel mailing list
389-devel@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-devel@lists.fedoraproject.org

Re: [389-devel] Please review ticket 47976: deadlock in mep delete post op

2015-11-03 Thread Ludwig Krispenz


Hi Thierry,

we already had started to discuss on IRC, but here are my thoughts again.

Is it necessary to explicitely set the txn in the plugin ? The txn will 
be found when ldbm_back_delete() does dblayer_txn_begin(9 and it checks 
the per thread stack of txns.
In my opinion the real problem is not to set  the txn in id2entry, which 
will then try to read a locked page.


Ludwig

On 11/03/2015 05:40 PM, thierry bordaz wrote:

ticket https://fedorahosted.org/389/ticket/47976

fix 
https://fedorahosted.org/389/attachment/ticket/47976/0001-Ticket-47976-deadlock-in-mep-delete-post-op.patch


test case: 
https://fedorahosted.org/389/attachment/ticket/47976/0002-Ticket-47976-test-case.patch




--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel


--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

Re: [389-devel] Please review (memory leak): [389 Project] #48252: db2index creates index entry from deleted records

2015-09-15 Thread Ludwig Krispenz

Thanks for this, it not only fixes a memory leak but also a regression. 
with the previous version of the fix after the processing of the first 
tombstone nstombstone_vals was set an duse in deciding what to index :-)



On 08/28/2015 02:19 AM, Noriko Hosoi wrote:

https://fedorahosted.org/389/ticket/48252

https://fedorahosted.org/389/attachment/ticket/48252/0001-Ticket-48252-db2index-creates-index-entry-from-delet.2.patch 



Description: Commit ca4d38dff3bd820af2bf60c9bcc82fd64aac2556 fixing
ticket #48252 introduced memory leaks.
. nstombstone_vals needs to be allocated just once and freed at 
the end.
. when skipping to index an entry for being a tombstone, the entry 
needs

  to be freed.

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel


--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

[389-devel] Please review: #48258: dna plugin needs to handle binddn groups for authorization

2015-08-27 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/48258

https://fedorahosted.org/389/attachment/ticket/48258/0001-Ticket-48258-dna-plugin-needs-to-handle-binddn-group.patch
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

Re: [389-devel] Determine if operation is internal or not

2015-08-03 Thread Ludwig Krispenz


you can use:
slapi_operation_is_flag_set(pb-pb_op, OP_FLAG_INTERNAL);
or
slapi_op_internal(pb)

Ludwig
On 08/03/2015 08:35 AM, William Brown wrote:

Hi,

I can see that in a plugin we can determine if the operation is a replicate
operation with:

int is_repl = 0;
slapi_pblock_get(pb, SLAPI_IS_REPLICATED_OPERATION, is_repl);

Is there a way to determine if the operation is from internal to the ds instance
as well? I can't see anything obvious.

Sincerely,



--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

[389-devel] please review: #48149 ns-slapd double free or corruption crash

2015-05-12 Thread Ludwig Krispenz

This fix should resolve a crash if the fix for BDB is not available for 
a specific BDB version


https://fedorahosted.org/389/ticket/48149

fix:
https://fedorahosted.org/389/attachment/ticket/48149/0001-Ticket-48149-ns-slapd-double-free-or-corruption-cras.patch

test:
https://fedorahosted.org/389/attachment/ticket/48149/0002-Ticket-48149-test-for-ns-slapd-double-free-or-corrup.2.patch

--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

[389-devel] please review: #48175 Avoid using regex in ACL if possible.

2015-05-12 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/48175

https://fedorahosted.org/389/attachment/ticket/48175/0001-Ticket-48175-Avoid-using-regex-in-ACL-if-possible.patch
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

Re: [389-devel] please review #48136 - accept auxilliary objectclasse in replication agreements

2015-03-31 Thread Ludwig Krispenz


next try to address Mark's comments:
https://fedorahosted.org/389/attachment/ticket/48136/0001-Ticket-48136-v2v2-accept-auxilliary-objectclasse-in-.patch

On 03/30/2015 02:09 PM, Ludwig Krispenz wrote:

https://fedorahosted.org/389/ticket/48136

https://fedorahosted.org/389/attachment/ticket/48136/0001-Ticket-48136-accept-auxilliary-objectclasse-in-repli.patch 


--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel


--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

[389-devel] please review #48136 - accept auxilliary objectclasse in replication agreements

2015-03-30 Thread Ludwig Krispenz


https://fedorahosted.org/389/ticket/48136

https://fedorahosted.org/389/attachment/ticket/48136/0001-Ticket-48136-accept-auxilliary-objectclasse-in-repli.patch
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

1 2 >

1 - 100 of 154 matches

Mail list logo