[389-devel] Re: Config attribute - nsslapd-allowed-sasl-mechanisms - behaviour

[William Brown]

On Fri, 2017-04-21 at 07:55 +0200, Simon Pichugin wrote:
> On Fri, Apr 21, 2017 at 09:25:23AM +1000, William Brown wrote:
> > On Thu, 2017-04-20 at 22:54 +0200, Simon Pichugin wrote:
> > > Hello William,
> > > I think my question is for you in the first place.
> > > It regards the default attributes for cn=config feature.
> > > 
> > > Version tested: 389-ds-base-1.3.6.1-9.el7.x86_64
> > > 
> > > During TET troubleshooting I've faced two issues:
> > > 1. By default, we have:
> > > [root@qeos-126 dirsrv-tet-install]# ldapsearch -h localhost -p 389 -D 
> > > "cn=Directory manager"
> > > -w Secret123 -b "cn=config" "objectclass=*" | grep 
> > > nsslapd-allowed-sasl-mechanisms
> > > nsslapd-allowed-sasl-mechanisms:
> > > 
> > > Empty value.
> > > 
> > > We can modify it and set something. (I'll skip the output, it works as 
> > > expected.
> > > And after this, the server allows to do like this:
> > > [root@qeos-126 dirsrv-tet-install]# ldapmodify -h localhost -p 389 -D 
> > > "cn=Directory manager" -w Secret123
> > > dn: cn=config
> > > changetype: modify
> > > delete: nsslapd-allowed-sasl-mechanisms
> > > 
> > > modifying entry "cn=config"
> > > 
> > > [root@qeos-126 dirsrv-tet-install]# ldapsearch -h localhost -p 389 -D 
> > > "cn=Directory manager"
> > > -w Secret123 -b "cn=config" "objectclass=*" | grep 
> > > nsslapd-allowed-sasl-mechanisms
> > > nsslapd-allowed-sasl-mechanisms:
> > > 
> > 
> > > 
> > 
> > Pretty sure the SASL mechs can't be written to: they are returned from
> > https://pagure.io/389-ds-base/blob/master/f/ldap/servers/slapd/saslbind.c#_754
> >  
> > 
> > And that only allows setting from mechs discovered by cyrus-sasl, and
> > mechs that are from plugins that register to
> > slapi_register_supported_saslmechanism() . The allowed mechs parameter
> > is the list that is checked to see if we'll allow using it, and that's
> > called from ids_sasl_getopt. During the set in libglobs, it looks like
> > we check that the mech is a real name supported by SASL, so perhaps test
> > with values like PLAIN, GSSAPI instead? 
> > 
> > > 
> > > 2. Second one is a known issue, but still I'd like to clarify the 
> > > expected behaviour:
> > > [root@qeos-126 dirsrv-tet-install]# ldapsearch -h localhost -p 389 -D 
> > > "cn=Directory manager"
> > > -w Secret123 -b "cn=config" "objectclass=*" | grep 
> > > nsslapd-allowed-sasl-mechanisms
> > > nsslapd-allowed-sasl-mechanisms: A
> > > [root@qeos-126 dirsrv-tet-install]# ldapmodify -h localhost -p 389 -D 
> > > "cn=Directory manager" -w Secret123
> > > dn: cn=config
> > > changetype: modify
> > > add: nsslapd-allowed-sasl-mechanisms
> > > nsslapd-allowed-sasl-mechanisms: B
> > > [root@qeos-126 dirsrv-tet-install]# ldapsearch -h localhost -p 389 -D 
> > > "cn=Directory manager"
> > > -w Secret123 -b "cn=config" "objectclass=*" | grep 
> > > nsslapd-allowed-sasl-mechanisms
> > > nsslapd-allowed-sasl-mechanisms: B
> > > 
> > > So it wouldn't be a multivalued attribute? if we'll do the 'add' 
> > > operation, it would replace the existing value with a new.
> > > 
> > > Please, comment of a both cases. First looks more like a bug to me 
> > > though, and I will file it if you'll confirm it.
> > 
> > Anyway, it looks like in libglobs.c, that it expects a comma seperated
> > list, it's not multivalued. The reason is that this single attribute is
> > returned in  config_get_allowed_sasl_mechs(); during the ids_sasl_getopt
> > call, which SASL_CB_GETOPT expects a specific format.
> > 
> > I imagine this is because it's easier to store it as one line in the
> > config, and requires less parsing each SASL request to go from
> > multivalue to one line, so it's an efficiency thing. 
> > 
> > 
> > Does that help explain the answer to your problems? 
> 
> Partly, yes. My question was about possible regression.
> 
> We have a lot of tests for nsslapd-allowed-sasl-mechanisms in TET.
> It has raised two concerns I have now.
> 
> First, if we are trying to set 'empty value' with a replace operation, it 
> fails (and it is okay).
> 
> But now with your new cn=config feature we have 
> 'nsslapd-allo

[389-devel] Please review: 49229 49228

[William Brown]

https://pagure.io/389-ds-base/issue/49228

https://pagure.io/389-ds-base/issue/raw/files/eaba716a895b94acd5a3dbde186983d9b51a5ae164cdad21f1285e44b91c3c81-0001-Ticket-49228-Fix-SSE4.2-detection.patch

https://pagure.io/389-ds-base/issue/49229

https://pagure.io/389-ds-base/issue/raw/files/11631f959403a92ba617df8ca256a7bbee9cfc6b7ca2bb18986bdf634337a3fd-0001-Ticket-49229-Correct-issues-in-latest-commits.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: fixes for tickets that were commited

[William Brown]

https://pagure.io/389-ds-base/issue/49229

https://pagure.io/389-ds-base/issue/raw/files/11631f959403a92ba617df8ca256a7bbee9cfc6b7ca2bb18986bdf634337a3fd-0001-Ticket-49229-Correct-issues-in-latest-commits.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Build failed in Jenkins: 389-ds-base #1257

[William Brown]

On Thu, 2017-04-20 at 23:41 +, jenk...@fedoraproject.org wrote:
> See <https://jenkins.fedorainfracloud.org/job/389-ds-base/1257/changes>
> 
> Changes:
> 
> [William Brown] Ticket 49119 - Cleanup configure.ac options and defines

> 
> ldap/servers/slapd/pblock.c:2105:15: warning: unused variable ‘x’ 
> [-Wunused-variable]
> ldap/servers/slapd/pblock.c:3749:15: warning: unused variable ‘x’ 
> [-Wunused-variable]
> ldap/servers/slapd/task.c:1010:57: warning: passing argument 3 of 
> ‘slapi_pblock_set’ discards ‘const’ qualifier from pointer target type 
> [-Wdiscarded-qualifiers]
> ldap/servers/slapd/slapi-plugin.h:856:5: note: expected ‘void *’ but argument 
> is of type ‘const char *’
> ldap/servers/slapd/psearch.c:158:2: warning: ‘slapi_pblock_clone’ is 
> deprecated [-Wdeprecated-declarations]
> In file included from ldap/servers/slapd/psearch.c:24:0:
> ldap/servers/slapd/slap.h:1756:16: note: declared here
> 
> 

I'm working on a patch to address these issues :) 


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: memory leak in ldap-agent-bin

[William Brown]

https://pagure.io/389-ds-base/issue/49226

https://pagure.io/389-ds-base/issue/raw/files/db4643f56c085a8b664f1247cdadf26bed2cff6d4b6156b28c442af06abd2c76-0001-Ticket-49226-Memory-leak-in-ldap-agent-bin.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49223, change sds queue to pthread mutex

[William Brown]

https://pagure.io/389-ds-base/issue/49223

https://pagure.io/389-ds-base/issue/raw/files/7a02fe1522ab070097b8199d5cb246191c9aeb304877287f78289d7943eb565a-0003-Ticket-49223-Fix-sds-queue-locking.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49222 Fix test issues on rawhide

[William Brown]

https://pagure.io/389-ds-base/issue/49222

https://pagure.io/389-ds-base/issue/raw/files/e43131c8524043448e1e995a7c32ce3ae191ee271c6836e894a29b5a54d8f098-0004-Ticket-49222-Resolve-various-test-issues-on-rawhide.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review new Replication Diff Tool

[William Brown]

On Wed, 2017-04-12 at 17:02 -0400, Mark Reynolds wrote:
> Hello,
> 
> This is a beta version of a replication diff tool written in python. 
> 
> Design page (this needs updating - I hope to get that done tonight)
> 
> http://www.port389.org/docs/389ds/design/repl-diff-tool-design.html
> 
> 
> Current usage:
> 
>   -v, --verbose Verbose output
>   -o FILE, --outfile=FILE
> The output file
>   -D BINDDN, --binddn=BINDDN
> The Bind DN (REQUIRED)
>   -w BINDPW, --bindpw=BINDPW
> The Bind password (REQUIRED)
>   -h MHOST, --master_host=MHOST
> The Master host (default localhost)
>   -p MPORT, --master_port=MPORT
> The Master port (default 389)
>   -H RHOST, --replica_host=RHOST
> The Replica host (REQUIRED)
>   -P RPORT, --replica_port=RPORT
> The Replica port (REQUIRED)
>   -b SUFFIX, --basedn=SUFFIX
> Replicated suffix (REQUIRED)
>   -l LAG, --lagtime=LAG
> The amount of time to ignore inconsistencies
> (default
> 300 seconds)
>   -Z CERTDIR, --certdir=CERTDIR
> The certificate database directory for startTLS
> connections
>   -i IGNORE, --ignore=IGNORE
> Comma separated list of attributes to ignore
>   -M MLDIF, --mldif=MLDIF
> Master LDIF file (offline mode)
>   -R RLDIF, --rldif=RLDIF
> Replica LDIF file (offline mode)
> 
> |Examples: python repl-diff.py -D "cn=directory manager" -w PASSWORD -h
> localhost -p 389 -H remotehost -P  -b "dc=example,dc=com" ||python 
> repl-diff.py -D "cn=directory manager" -w PASSWORD -h localhost
> -p 389 -H remotehost -P  -b "dc=example,dc=com" -Z
> /etc/dirsrv/slapd-localhost|
> |python repl-diff.py -M /tmp/master.ldif -R /tmp/replica.ldif |
> 
> 
> How long the tool takes to run depends on the number of entries per
> database.  See performance numbers below 
> 
> Entries per Replica Time
> 
> -
> 100k40 seconds
> 500k3m 30secs
> 1 million   7m 30secs
> 2 million   14 minutes
> 10 million  ~70 minutes
> 
> 
> 
> I'd be very interested in feedback, RFE's, and bugs.

Hey mate, 

The tool looks great, awesome work on this. Really impressive that you
got it to 70 minutes for 10 million entries.

How responsive is the server during this process? We aren't going to
cause some odd resource exhaustion? 

import optparse

With python, optparse is deprecated. Can we use argparse instead? It's
nearly identical. Lots of examples of this in dsctl. 

With connect to replicas, some sites may only have ldaps (provided by a
load balancer). So our scripts should really be taking an LDAPurl, a
certdir, and a starttls flag. Because ldaps://localhost + certdir is a
valid option, but if we force call start_tls_s(), we break it. As well,
someone may use ldapi:// etc. It also saves on port options and more
flags to the cli because we can do ldap://localhost:30389 etc. 

Hope that helps, I'll be happy to review again later!

For now, I think our strategy with this should be to add it to
389-ds-base, and later we can move this into lib389 when we can. How
does that sound? 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49119 clean configure options

[William Brown]

https://pagure.io/389-ds-base/issue/49119

https://pagure.io/389-ds-base/issue/raw/files/b5673bc55b4fe94a630be6cdd1fb63509f05214dcc44630a5e4de48c805b1bdb-0001-Ticket-49119-Cleanup-configure.ac-options-and-define.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Low bound on import sizing

[William Brown]

https://pagure.io/389-ds-base/issue/49204

https://pagure.io/389-ds-base/issue/raw/files/439afb815888f065e5ad50db3918444da66758a6b85338265d8db85f115f9fce-0001-Ticket-49204-Fix-lower-bounds-on-import-autosize.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review: DSLdapObject compare function and user compare tests

[William Brown]

On Fri, 2017-04-07 at 06:10 +, Ankit Yadav wrote:
> Hello Everyone,
> 
> I have changed the compare function, took help from Entry.__eq__ function. 
> I have also updated the user_compare_test.py, Now it includes 4 assertions. 
> Details can be found in lib389/tests/idm/user_compare_test.py
> Also removed the 'DeepDiff' dependency.
> Please review this commit: 
> https://pagure.io/fork/ankity10/lib389/c/762ef9054f01837a33219aa2412d428f5d783c52?branch=ticket-1-config-compare
> 
> Thanks

Hi there,

This is a big improvement! Thanks for taking all my feedback.

When we supply patches for review, we generally like to supply them as
1, so we can see the whole change in one, rather than over two commits.
I've just updated our contributing guide, so it may be worth you reading
over it.

http://www.port389.org/docs/389ds/contributing.html

It has a section on how to squash commits together in git, as well as
our commit message formats we prefer.

21 + # check if RDN is same
22 + if obj1.rdn != obj2.rdn:
23 + return False

I think the question is if we want to check if two objects in different
subtrees have the same attributes, or if we want *true* object equality.

*true* object equality, you want to check nsUniqueId first, because
that's always going to be different between everything.

But if we want to just compare arbitrary objects, I think the rdn check
is better, and we need to ignore nsUniqueId.

For this purpose perhaps we want the "loose" equality, just checking if
attributes are the same. IE if I had:

cn=user1,ou=a
cn=user1,ou=b

They should compare the same, even though their nsUniqueId differs. 

For now I think this code is okay, but we should leave some comments
around this discussion there, and certainly document in the docstring
that this is a loose equality, not a strict "this object *is* this other
object". 


46   # removing _compate_exclude attrs from all attrs
47   for key in self._compare_exclude:
48 - attrs_dict.pop(key.lower(), None)
49 + del attrs_dict[key.lower()]

Rather than deleting these, why not use a set and do:

compare_attrs = set(attrs_dict.keys) - set(self._compare_exclude)

The issue with python is that sometimes delete and operations like that
are destructive and have weird side effects, so it's better to take a
functional approach and make a new set of attributes you want, rather
than modifying a set you already have. 

48   testuser1.delete()
49   testuser2.delete()

The test teardown deletes the database, so you don't need to delete the
users at the end :) 


If you squash this patch and the last one together now, and send again
for a quick check, I think you would be pretty close to having a working
object compare to merge :) 

Thanks heaps! 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: lib389 fixes to makefile and rpmspec

[William Brown]

https://pagure.io/lib389/issue/19

https://pagure.io/lib389/issue/raw/files/15c7b8fef1c1387d5091cc605199d451e083a21477e9db2519029ed70c030f0e-0001-Ticket-19-Missing-file-and-improve-make.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Build failed in Jenkins: 389-ds-base #1240

[William Brown]

> 
> ldap/servers/slapd/util.c:1492:65: warning: zero-length gnu_printf format 
> string [-Wformat-zero-length]
> ldap/servers/slapd/slapi-private.h:40:81: note: in definition of macro 
> ‘slapi_log_err’
> ldap/servers/slapd/util.c:1468:15: warning: ‘util_getvirtualmemsize’ defined 
> but not used [-Wunused-function]
> 

This looks like it's my fault: I'll fix it tomorrow morning. 


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: With NSS SQL db, ds will not start

[William Brown]

https://pagure.io/389-ds-base/issue/49041

https://pagure.io/389-ds-base/issue/raw/files/0b80b56b4f4c765518293a0263d321cde1b101c04c5aa57a1368164e0cde29d2-0001-Ticket-49041-SSL-fails-to-start-due-to-NSS-db-versio.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review: DSLdapObject compare function and user compare tests

[William Brown]

On Tue, 2017-04-04 at 23:02 +, Ankit Yadav wrote:
> Hello Everyone, 
> 
> I was working on issue #1 lib389 cn=config comparison, I have done some work 
> in that direction. I have added a compare function on DSLdapObject and have 
> written a test for comparing user object. Currently this test is not 
> complete, I am just printing the attributes values to verify the working of 
> comparison function.
> But comparison function is working, so let me know your feedback and how we 
> can proceed further with this?
> 
> Link to commit: 
> https://pagure.io/fork/ankity10/lib389/c/44b02ba4ddb54be12041052b5b75f17b9eaf6b8f?branch=ticket-1-config-compare
> 

Hey there,

Looks like a great start. A lot of comments though, but thanks for all
your effort and I hope these comments help you improve your code a lot.



27 + obj1_attrs_json = obj1.get_compare_attrs()
28 + obj2_attrs_json = obj2.get_compare_attrs()
29 + return DeepDiff(obj1_attrs_json, obj2_attrs_json)

I don't think you should convert these to json. You should be comparing
the dictionaries from the Entry directly. The JSON is only there for our
future admin server to represent the entries in a website - it should
never be used internally for operations in this library. JSON is also
non-deterministic garbage IMO, so lets avoid it "unless we need it". 

You already have:

42 + attrs_entry = self._instance.getEntry(self._dn,
ldap.SCOPE_BASE, "(objectclass=*)", ["*", "+"])42 +
attrs_entry = self._instance.getEntry(self._dn, ldap.SCOPE_BASE,
"(objectclass=*)", ["*", "+"])

So you should probably return either the entry data dict from here
instead, then compare on that an example of this here:

As "inspiration", look at:

https://pagure.io/lib389/blob/master/f/lib389/_entry.py#_91

It's worth exploring and reading the code for what you use to find gems
like this you may be able to copy or reuse. I think in this case, it's
better to make a new compare function based on this code rather than
trying to hack entry.__eq__ to work in this case, if that makes sense. 


 5 + from deepdiff import DeepDiff

Please don't add library dependencies, unless it exists as a package on
RHEL7, we can not support extra dependencies in our code. 

"""
yum search deepdiff
...
Warning: No matches found for: deepdiff
No matches found
"""


With these lines:

12   class DSLdapObject(DSLogging):
13 + compare_exclude = []

4   class UserAccount(DSLdapObject):
5 + compare_exclude = ['nsuniqueid']
6 + 

Make sure you put these attributes in __init__() of the objects as:

def __init__()
self._compare_exclude =  

The reason for this is that the top level attribute is global to all instances. 
So if any subclass overrides
the attribute, it causes the override of the parents attribute for ALL 
subclasses: this causes horrible
horrible issues that are near impossible to resolve. You need to make these 
self. to indicate
they are on the single instance only, which makes it work correctly.

As well, because these are internal, prefix the variable with an _ to say it's 
private. Therefore:

def __init__()
self._compare_exclude =  

Remember, especially on the UserAccount, to put the self._compare_exclude 
*after* the call
to super(UserAccount).__init__! , else the super call will just flush it back 
to the DSLdapObject
value. 



With your test case, you don't need:

57 + assert (len(users.list()) == 1)

Because if the user fails to create, we throw an exception from users.create, 
so the test
will fail at that stage. :) 


74 + print("testuser1 all attributes: ")

try to use 

log.[info,debug,error]() instead. This way based on the
DEBBUGING environment variable we see more or less output. We shouldn't
have "print" anywhere in the code base. 


88 + # We want to capture std out while running this module through
pytest, hence adding explicit test failure

Run py.test with "-s" to show the stdout during the test instead :) 


Remember, to test users that are the same and also different! Also, a
user compared to itself must be the same also, and a user compared to a
different object class should also be false! IE compare a user to a
group :) 


I hope that this advice helps you: I know it's a lot to change, but
thanks for your work! 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: CGroup memory limit detection

[William Brown]

https://pagure.io/389-ds-base/issue/48864

https://pagure.io/389-ds-base/issue/raw/files/5aebc50255a5665943528bde7f544c422ed9e9ee80c4b0193a41468d6740dff3-0001-Ticket-48864-Add-cgroup-memory-limit-detection-to-38.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] GSOC applications have closed,

[William Brown]

Hi,

GSOC applications have closed now: I really want to say thank you to
everyone who joined our lists, talked with us, and applied. I have been
really impressed by the high quality of students from around the world
that have shown an interest in this project. 

Please be patient with myself and the Fedora Project in the coming
weeks: Our next announcement about the project will on or after April
the 17th.

Thank you again,

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Remove vacuum lock on b+tree cow

[William Brown]

https://pagure.io/389-ds-base/issue/49153

https://pagure.io/389-ds-base/issue/raw/files/caa388aaee538a194605ae490b03e192d3df4bb276a676e5a6c72a224d1b3e52-0001-Ticket-49153-Remove-vacuum-lock-on-transaction-clean.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Clean up memory detection code

[William Brown]

https://pagure.io/389-ds-base/issue/48864

https://pagure.io/389-ds-base/issue/raw/files/a93d8c39090cea2ff30546b527016d763ecd3a90f4a76341fbdcbba529a5320c-0002-Ticket-48864-Cleanup-memory-detection-before-we-add-.patch
https://pagure.io/389-ds-base/issue/raw/files/a7f757134085798dc3484d924695aa0510eb1fc7c6b37134990c30eb71eb1eb2-0001-Ticket-48864-Cleanup-up-broken-format-macros-and-imp.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Add missing build requires to specfile

[William Brown]

https://pagure.io/389-ds-base/issue/49206

https://pagure.io/389-ds-base/issue/raw/files/e29c5479f901f48c46badb625e4bf70dcce2837f392f5ffa0b4946879e8e294e-0001-Ticket-49206-Add-missing-make-dependency.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: rename dsadm to dsctl

[William Brown]

https://pagure.io/lib389/issue/14


https://pagure.io/lib389/issue/raw/files/bb1d3128b660cb1b29a579013ad4906f48545b9ba17d99b29e91dfdd43a0983f-0001-Ticket-14-Remane-dsadm-to-dsctl.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 15 improve python instance creation

[William Brown]

https://pagure.io/lib389/issue/15

https://pagure.io/lib389/issue/raw/files/b13ac0d68059a8d22058166d1ba4c0fa87a42e7e0425cd236aec046497ff69bd-0001-Ticket-15-Improve-instance-configuration-ability.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49200 minimal dse.ldif

[William Brown]

https://pagure.io/389-ds-base/issue/49200

https://pagure.io/389-ds-base/issue/raw/files/67c5fa886814f41ab25a16bbf938b034d7404f032bd330499b24e92a14f728db-0001-Ticket-49200-provide-minimal-dse.ldif-for-python-ins.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: ds cli to init sample entries in a backend

[William Brown]

https://pagure.io/lib389/issue/13

https://pagure.io/lib389/issue/raw/files/ca18b107709e9e34e494398307424ac8c2c047efdb97cdfed3c16125f760e7cb-0001-Ticket-13-Add-init-function-to-create-new-domain-ent.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Build failed in Jenkins: 389-DS-NIGHTLY #192

[William Brown]

user2,dc=example,dc=com,password1}
> INFO:dirsrvtests.tests.tickets.ticket48228_test:  Bind as 
> {uid=user2,dc=example,dc=com,password2}
> INFO:dirsrvtests.tests.tickets.ticket48228_test:  Bind as 
> {uid=user2,dc=example,dc=com,password3}
> INFO:dirsrvtests.tests.tickets.ticket48228_test:  Bind as 
> {uid=user2,dc=example,dc=com,password4}
> INFO:dirsrvtests.tests.tickets.ticket48228_test:  Bind as 
> {uid=user2,dc=example,dc=com,password5}
> __ test_delete_entry 
> ___
> 
> topo = 
> test_entry = None
> 
> def test_delete_entry(topo, test_entry):
> """Check that entry deletion is replicated after delete operation
> 
> :ID: 18437262-9d6a-4b98-a47a-6182501ab9bc
> :feature: Multi master replication
> :setup: Four masters replication setup, an entry
> :steps: 1. Delete the entry from master1
> 2. Wait for replication to happen
> 3. Check entry on all other masters
> :expectedresults: Entry deletion should be replicated
> """
> 
> log.info('\''Deleting entry {} during the 
> test'\''.format(TEST_ENTRY_DN))
> topo.ms["master1"].delete_s(TEST_ENTRY_DN)
> 
> entries = get_repl_entries(topo, TEST_ENTRY_NAME, ["uid"])
> >   assert not entries, "Entry deletion {} wasn'\''t replicated 
> > successfully".format(TEST_ENTRY_DN)
> E   AssertionError: Entry deletion uid=mmrepl_test,dc=example,dc=com 
> wasn'\''t replicated successfully
> E   assert not [dn: uid=mmrepl_test,dc=example,dc=com\nuid: 
> mmrepl_test\n\n, dn: uid=mmrepl_test,dc=example,dc=com\nuid: mmrepl_test\n\n, 
> dn: uid=mmrepl_test,dc=example,dc=com\nuid: mmrepl_test\n\n]
> 
> <http://vm-058-081.abc.idm.lab.eng.brq.redhat.com:8080/job/389-DS-NIGHTLY/ws/source/389-ds-base/dirsrvtests/tests/suites/replication/acceptance_test.py>:154:
>  AssertionError
>  Captured stderr setup 
> -
> INFO:dirsrvtests.tests.suites.replication.acceptance_test:Adding entry 
> uid=mmrepl_test,dc=example,dc=com
> - Captured stderr call 
> -
> INFO:dirsrvtests.tests.suites.replication.acceptance_test:Deleting entry 
> uid=mmrepl_test,dc=example,dc=com during the test
> INFO:dirsrvtests.tests.suites.replication.acceptance_test:Wait for 
> replication to happen
> === 2 failed, 500 passed in 10973.19 seconds 
> ==='
> + '[' 1 -ne 0 ']'
> + echo CI Tests 'FAILED!'
> CI Tests FAILED!
> + MSG=FAILED
> + RC=1
> + sudo /usr/sbin/sendmail mreyno...@redhat.com firsty...@redhat.com
> + sudo rm -rf /var/tmp/slapd.vg.28550 /var/tmp/slapd.vg.50045 
> /var/tmp/slapd.vg.50147 /var/tmp/slapd.vg.58762
> + exit 1
> Build step 'Execute shell' marked build as failure
> ___
> 389-devel mailing list -- 389-devel@lists.fedoraproject.org
> To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49196 cache sanity check log levels

[William Brown]

https://pagure.io/389-ds-base/issue/49196

https://pagure.io/389-ds-base/issue/raw/files/9db0753f133daf14dd4cff435e5b6f488306948a368d1256e229ede526219a35-0001-Ticket-49196-Autotune-generates-crit-messages.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Lower default ioblock timeout

[William Brown]

https://pagure.io/389-ds-base/issue/49194

https://pagure.io/389-ds-base/issue/raw/files/58935336f6a6dbc90d8d8b2ad5642e4c3d72d6a45d63572dc9414fafe79f0c8a-0001-Ticket-49194-Lower-default-ioblock-timeout.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Restore lock counter due to ppc

[William Brown]

https://pagure.io/389-ds-base/issue/48989

https://pagure.io/389-ds-base/issue/raw/files/2b89fc694f537fc77e9966ca1430016e7af2a7b9b9f4d5372637aec57117758d-0001-Ticket-48989-Re-implement-lock-counter.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: gcc7 compiler warnings

[William Brown]

https://pagure.io/389-ds-base/issue/49193

https://pagure.io/389-ds-base/issue/raw/files/17085317b52fa3ce91a3c10cdb5d21c1b71e41079eb1940cf4a8302869067a37-0001-Ticket-49193-gcc7-warning-fixes.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: lfds711 upgrade

[William Brown]

https://pagure.io/389-ds-base/issue/49190

https://pagure.io/389-ds-base/issue/raw/files/fe52265594e30d95f90e630a463d1bc8d991310ea915087e98ec1ce0bc3f25a0-0001-Ticket-49190-Upgrade-lfds-to-7.1.1.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review 49186 : improve ns shutdown reliability

[William Brown]

https://pagure.io/389-ds-base/issue/49186

https://pagure.io/389-ds-base/issue/raw/files/2efd656894e25847cb1d0580999b6c21152efb3ced33412cd159b0073f8b021d-0001-Ticket-49186-Fix-NS-to-improve-shutdown-relability.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: fix definition for crc32c on non-sse4.2 hardware

[William Brown]

https://pagure.io/389-ds-base/issue/49187

https://pagure.io/389-ds-base/issue/raw/files/019d2d8c5c1f6c0841ff50d2897625454343c36fe01e54daf3050d34342f0608-0001-Ticket-49187-Fix-attribute-definition.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: improve counter overflow patch

[William Brown]

https://pagure.io/389-ds-base/issue/48989

https://pagure.io/389-ds-base/issue/raw/files/e44f83238e462ba9d0d5428c7cecb0df95cfe4602bf7ff2fc9c6b72f211d37dd-0001-Ticket-48989-Improve-counter-overflow-fix.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Please review: 49174 nsIdleTimeout of -1 causes connection to fail

[William Brown]

On Wed, 2017-03-22 at 14:16 +1000, William Brown wrote:
> https://pagure.io/389-ds-base/issue/49174
> 
> https://pagure.io/389-ds-base/issue/raw/files/72218d7fe921f50f3f86ce24ea9c5d592dc5c2fa8d5ae22fa5b11caf9d6c4a53-0001-Ticket-49174-nunc-stans-can-not-use-negative-timeout.patch
> 
> 


https://pagure.io/389-ds-base/issue/raw/files/508c41857f3d1963e7646dc9d2e01b249e6584f38230eb38f3aa4a07a9a9e549-0001-Ticket-49174-nunc-stans-can-not-use-negative-timeout.patch

Update to remove // comment. 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 1.3.5 branch, defaults.inf missing components

[William Brown]

https://pagure.io/389-ds-base/issue/49179

https://pagure.io/389-ds-base/issue/raw/files/382870e66a7e734a4d080d9ef7a31af1608efad7d05ca9ad205fddea5ad35549-0001-Ticket-49179-Missing-components-in-1.3.5-defaults.in.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review

[William Brown]

https://pagure.io/389-ds-base/issue/49177

https://pagure.io/389-ds-base/issue/raw/files/aa55a6c32e92e6c9af8af162c45c6419e1cff3abaa1ff95636fcaf1bf5fb89d8-0001-Ticket-49177-rpm-would-not-create-valid-pkgconfig-fi.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Review process suggestion

[William Brown]

Hi all,

I've noticed that sometimes during the review process that the following
happens:

Europe Afternoon: code is posted to review.
America Morning: code is acked
Europe Afternoon: code is merged

Australian Morning: "I have this comment, but the code is already
merged "

Then we need to make extra "fixup patches" instead.


I wonder if we could suggest that there is a mandatory 24 hour review
window. Code once posted for review and prior to merge must be:

* ack's from all reviewers
* online for 24 hours since submission before merge

We have a global team, and sometimes it's easy for us to forget this and
not let everyone have the chance to have their say. This review process
is really valuable to us all, so it's best if we can all input to this
as it raises the standard of our project.

I think the exception would be some kind of pressure (critical issues,
blocker, build chain break, broken nightly tests) where we would waive
the 24 hour review period.


Does this seem like a reasonable change we could make as a team to give
all of us the chance to review.

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Questions related to lib389 issue #1 cn=config

[William Brown]

his command in updated lib389 repo ==>
> >>
> >> sudo PYTHONPATH=`pwd` py.test -s lib389/tests/cli/conf_backend.py
> >>
> >> step 6: Got some INFO logs and and 2 errors
> >>
> >> Errors:
> >> This error two times:
> >>
> >> * error start ===*
> >> *self = ,
> >> section = 'slapd'*
> >> *option = 'pid_file', raw = False, vars = None*
> >>
> >> *def get(self, section, option, raw=False, vars=None):*
> >> *"""Get an option value for a given section.*
> >>
> >> *If `vars' is provided, it must be a dictionary. The option
> >> is looked up*
> >> *in `vars' (if provided), `section', and in `defaults' in
> >> that order.*
> >>
> >> *All % interpolations are expanded in the return values,
> >> unless the*
> >> *optional argument `raw' is true. Values for interpolation
> >> keys are*
> >> *looked up in the same manner as the option.*
> >>
> >> *The section DEFAULT is special.*
> >> *"""*
> >> *sectiondict = {}*
> >> *try:*
> >> *sectiondict = self._sections[section]*
> >> *except KeyError:*
> >> *if section != DEFAULTSECT:*
> >> *raise NoSectionError(section)*
> >> *# Update with the entry specific variables*
> >> *vardict = {}*
> >> *if vars:*
> >> *for key, value in vars.items():*
> >> *vardict[self.optionxform(key)] = value*
> >> *d = _Chainmap(vardict, sectiondict, self._defaults)*
> >> *option = self.optionxform(option)*
> >> *try:*
> >> *value = d[option]*
> >> *except KeyError:*
> >> *>   raise NoOptionError(option, section)*
> >> *E   NoOptionError: No option 'pid_file' in section: 'slapd'*
> >>
> >> */usr/lib64/python2.7/ConfigParser.py:618: NoOptionError*
> >> * error ends *
> >>
> >> After removing all the instances this is what I am getting every time.
> >> What could be the possible reason for this error?
> >>
> >> Regards,
> >> Ankit Yadav.
> >>
> >>
> >>
> >>
> >> On 16 March 2017 at 12:18, Ankit Yadav <ankit...@gmail.com> wrote:
> >>
> >>>
> >>> Sorry I was travelling so was unable to try that.
> >>>
> >>> I have tried this command several times. Then I got a error like this:
> >>>
> >>> > except KeyError:
> >>> > >   raise NoOptionError(option, section)
> >>> > E   NoOptionError: No option 'pid_file' in section: 'slapd'
> >>>
> >>> This shows my 389-ds-base is not installed properly so,
> >>> I have purged all the ds instances.
> >>> Now I am reinstalling the 389-ds-base and then try once again.
> >>>
> >>> I will update you about this once I am done.
> >>>
> >>> Regards,
> >>> Ankit Yadav.
> >>>
> >>>
> >>>
> >>>
> >>> On 15 March 2017 at 13:48, William Brown <wibr...@redhat.com> wrote:
> >>>
> >>>> On Wed, 2017-03-15 at 12:50 +0530, Ankit Yadav wrote:
> >>>> > I have uninstalled that package.
> >>>> > There were some issues with my installation but now I am getting some
> >>>> > different errors.
> >>>> >
> >>>>
> >>>> That means you already have the instance of that name: The error is
> >>>> still a bit rough.
> >>>>
> >>>> Try:
> >>>>
> >>>> sudo /sbin/remove-ds.pl -i slapd-standalone
> >>>>
> >>>> Then run the test again.
> >>>>
> >>>> --
> >>>> Sincerely,
> >>>>
> >>>> William Brown
> >>>> Software Engineer
> >>>> Red Hat, Australia/Brisbane
> >>>>
> >>>>
> >>>> ___
> >>>> 389-devel mailing list -- 389-devel@lists.fedoraproject.org
> >>>> To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
> >>>>
> >>>>
> >>>
> >>
> >
> ___
> 389-devel mailing list -- 389-devel@lists.fedoraproject.org
> To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Questions related to lib389 issue #1 cn=config

[William Brown]

On Wed, 2017-03-15 at 12:50 +0530, Ankit Yadav wrote:
> I have uninstalled that package.
> There were some issues with my installation but now I am getting some
> different errors.
> 

That means you already have the instance of that name: The error is
still a bit rough.

Try:

sudo /sbin/remove-ds.pl -i slapd-standalone

Then run the test again. 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] please review: allow ds and lib389 to work with schema in 1.3.6

[William Brown]

https://pagure.io/389-ds-base/issue/49172

https://pagure.io/389-ds-base/issue/raw/files/c31ec8fd731b2c66dad5c91e80838cd6404eface75db88844832be40a9d56ff1-0001-Ticket-49172-Allow-lib389-to-read-system-schema-and-.patch

https://pagure.io/389-ds-base/issue/raw/files/c804231cbd852551b7eb65eaea3fb3775968e1ccc105a5a80c80aca20afc6d5b-0001-Ticket-49172-Fix-test-schema-files.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] urgent: please review, nunc-stans timeout is not correctly handled

[William Brown]

https://pagure.io/389-ds-base/issue/49171

https://pagure.io/389-ds-base/issue/raw/files/1b2ca3d697463730a0e867be14206b5ad5155073c6f7dcead57258adb2a56463-0001-Ticket-49171-Nunc-Stans-incorrectly-reports-a-timeou.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Urgent: please review external bind fix

[William Brown]

https://pagure.io/389-ds-base/issue/49165

https://pagure.io/389-ds-base/issue/raw/files/9c8c82e1a43e1adf854e0857e5b12ad6de4c7bdc1f1836c1c6ae064ed57ee37b-0001-Ticket-49165-pw_verify-did-not-handle-external-auth.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Questions related to lib389 issue #1 cn=config

[William Brown]

On Mon, 2017-03-13 at 06:06 +, Ankit Yadav wrote:
> Hello Everyone,
> 
> I am currently working on one of the issue of lib389 issue #1 cn=config. 
> Below are some steps that I did to help myself and a question related to 
> these steps.
> 
> lib389 issue #1 Description: 
> Once we have enough plugin and index code, we should be able to use dsconf to 
> compare the state of two instances. This will let us check for differing 
> plugin, index, configuration values between two servers.
> Additionally, we would be able to mask "known different" values, such as fs 
> paths, hostnames, etc.
> This will help in migrations and upgrades of DS.
> 
> I have an instance of 389-ds installed with this configuration:
> 
> computer name = local.com
> system user = dirsrv
> system group = dirsrv
> network port = 389
> directory server identifier = local
> suffix dc=local
> cn = Directory Manager
> 
> To see configuration of server I tried this command : $ ldapsearch -H ldap:// 
> -x -s base -b "" -LLL "+"
> 
> I got this output: 
> 
> dn:
> creatorsName: cn=server,cn=plugins,cn=config
> modifiersName: cn=server,cn=plugins,cn=config
> createTimestamp: 20170313052148Z
> modifyTimestamp: 20170313052148Z
> nsUniqueId: f4bebe00-07ac11e7-8000-
> namingContexts: dc=local
> nsBackendSuffix: userRoot:dc=local
> subschemaSubentry: cn=schema
> supportedExtension: 2.16.840.1.113730.3.5.7
> supportedExtension: 2.16.840.1.113730.3.5.8
> supportedExtension: 1.3.6.1.4.1.4203.1.11.3
> supportedExtension: 2.16.840.1.113730.3.5.3
> supportedExtension: 2.16.840.1.113730.3.5.12
> supportedExtension: 2.16.840.1.113730.3.5.5
> supportedExtension: 2.16.840.1.113730.3.5.6
> supportedExtension: 2.16.840.1.113730.3.5.9
> supportedExtension: 2.16.840.1.113730.3.5.4
> supportedExtension: 2.16.840.1.113730.3.6.5
> supportedExtension: 2.16.840.1.113730.3.6.6
> supportedExtension: 2.16.840.1.113730.3.6.7
> supportedExtension: 2.16.840.1.113730.3.6.8
> supportedExtension: 1.3.6.1.4.1.4203.1.11.1
> supportedControl: 2.16.840.1.113730.3.4.2
> supportedControl: 2.16.840.1.113730.3.4.3
> supportedControl: 2.16.840.1.113730.3.4.4
> supportedControl: 2.16.840.1.113730.3.4.5
> supportedControl: 1.2.840.113556.1.4.473
> supportedControl: 2.16.840.1.113730.3.4.9
> supportedControl: 2.16.840.1.113730.3.4.16
> supportedControl: 2.16.840.1.113730.3.4.15
> supportedControl: 2.16.840.1.113730.3.4.17
> supportedControl: 2.16.840.1.113730.3.4.19
> supportedControl: 1.3.6.1.1.13.1
> supportedControl: 1.3.6.1.1.13.2
> supportedControl: 1.3.6.1.4.1.42.2.27.8.5.1
> supportedControl: 1.3.6.1.4.1.42.2.27.9.5.2
> supportedControl: 1.2.840.113556.1.4.319
> supportedControl: 1.3.6.1.4.1.42.2.27.9.5.8
> supportedControl: 1.3.6.1.4.1.4203.666.5.16
> supportedControl: 2.16.840.1.113730.3.4.14
> supportedControl: 2.16.840.1.113730.3.4.20
> supportedControl: 1.3.6.1.4.1.1466.29539.12
> supportedControl: 2.16.840.1.113730.3.4.12
> supportedControl: 2.16.840.1.113730.3.4.18
> supportedControl: 2.16.840.1.113730.3.4.13
> supportedFeatures: 1.3.6.1.4.1.4203.1.5.1
> supportedSASLMechanisms: EXTERNAL
> supportedSASLMechanisms: SCRAM-SHA-1
> supportedSASLMechanisms: GSS-SPNEGO
> supportedSASLMechanisms: GSSAPI
> supportedSASLMechanisms: DIGEST-MD5
> supportedSASLMechanisms: CRAM-MD5
> supportedSASLMechanisms: PLAIN
> supportedSASLMechanisms: LOGIN
> supportedSASLMechanisms: ANONYMOUS
> supportedLDAPVersion: 2
> supportedLDAPVersion: 3
> vendorName: 389 Project
> vendorVersion: 389-Directory/1.3.5.15 B2016.308.2026
> 
> Are the above attributes and those configuration attributes which we are 
> trying to compare in this issue are same?

This output you are seeing is the rootDSE: It describes the features and
supported components of the server.

You will eventually be looking at cn=config, but for now your first
check to make the comparison work should be looking at objects from
lib389/idm/group.py for example. Make two groups in dc=example,dc=com,
then compare them.

You'll need a reproducable environment, so first, look at creating a
py.test case in lib389/tests/, and getting that to work, to build and
tear down an instance. We have plenty of examples in that folder of how
to do this. 

From there you can create the groups, then work on the comparison. 

I hope that helps you, 


> 
> Sincerely,
> 
> William Brown
> Software Engineer
> Red Hat, Australia/Brisbane
> 


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: ns-slapd with setcap net_bind

[William Brown]

https://pagure.io/389-ds-base/issue/48432

https://pagure.io/389-ds-base/issue/raw/files/0fb8acffb0a8758a9c1620b6d8456b94a8125d972a34d55f14f522ce835ed864-0001-Ticket-48432-Linux-capabilities-on-ns-slapd.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: selinux policy

[William Brown]

https://pagure.io/389-ds-base/issue/49151

https://pagure.io/389-ds-base/issue/raw/files/23fa6477de5c63a6fd8d5914c5b6b6db81118b49645aab4352f884f414fc7e61-0001-Ticket-49151-Remove-defunct-selinux-policy.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] please review 49160 sds copyright and benchmark

[William Brown]

https://pagure.io/389-ds-base/issue/49160

https://pagure.io/389-ds-base/issue/raw/files/5dcf794618dbecb08e5541d6bd9762c60c0d432099c328b5ac7d533d00b88435-0001-Ticket-49160-Fix-sds-benchmark-and-copyright.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] 389 ds Announcement - Merge of Nunc Stans with Directory Server

[William Brown]

The 389 Directory Server team have decided that in the interest of
maintenance and to keep with the direction of the project, to merge the
source code of Nunc Stans into the 389 DS repository.

This software will be found in the following location after this change:

https://pagure.io/389-ds-base/blob/master/f/src/nunc-stans 

The repository for Nunc Stans will be preserved for historical reasons,
with the "master" commit being a single README explaining where Nunc
Stans can now be found.

-- For developers of Directory Server:

You no longer need to build external nunc-stans and related components.
They will be part of the Directory Server build process. You should
remove your references to nunc-stans and it's repository. You may wish
to remove libnunc-stans.so* from your library paths where you make
install into. 

-- For maintainers of Directory Server:

Nunc Stans was not considered ready for versions of DS 1.3.5 and lower.
We have decided to make it the default event handler in DS 1.3.6. This
merge of Nunc Stans with DS will take place for the DS 1.3.6 release. No
configuration of Directory Server is needed to enable this change.

This means you do not need to add or maintain extra libraries for
Directory Server in your distributions. Nunc Stans will come as part of
our source code in DS, and will automatically be built and installed
during the make process. Patches will be considered "whole" units that
encapsulate any fix to Nunc Stans and Directory Server kept in
synchronous. 

You will need to ensure your platform package builds correctly include
the extra libraries in the system shared library directory.

-- For developers consuming Nunc Stans:

Nunc Stans lower than 0.2.1 are not considered to be stable, and should
not be used.

Directory Server 1.3.6 will contain what was "0.2.2". In order to
continue to use this, you should use the 389 Directory Server source, or
depend on your distribution's 389-ds-base-libs package or similar. It is
recommended that you use the distributed nunc-stans pkgconfig
information in your build process to correctly detect and include this
library.

-- For Directory Server admins using Nunc Stans

Nunc Stans lower than 0.2.1 are not considered to be stable, and should
not be used with Directory Server. We advise upgrade to Directory Server
1.3.6 immediately if you are using this configuration.

Directory Server 1.3.6 will contain what was "0.2.2". It is enabled by
default, so there is no more configuration option for enable/disable of
this feature. In the future our existing poll() based mechanism will be
removed. 


If you have questions about this change, please contact the team on the
developer mailing list (389-devel@lists.fedoraproject.org)

References:

https://pagure.io/389-ds-base/issue/49006
https://pagure.io/389-ds-base/issue/49139
http://www.port389.org/docs/389ds/design/nunc-stans.html
http://www.port389.org/docs/389ds/design/nunc-stans-workers.html


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Review: RFC submission contents

[William Brown]

https://pagure.io/389-ds-base/issue/48707

https://pagure.io/389-ds-base/issue/raw/files/1c4ff57f06f97521d0a9334fe1a8a015a79cf255b5aff5394d9ab4ed2e3778bb-0001-Ticket-48707-Update-rfc-to-accomodate-that-authid-is.patch

https://tools.ietf.org/html/draft-wibrown-ldapssotoken-02

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Discuss: local vs remote be flag

[William Brown]

On Mon, 2017-02-27 at 17:51 -0800, Noriko Hosoi wrote:
> Thank you, William!
> 
> On 02/27/2017 03:35 PM, William Brown wrote:
> > On Mon, 2017-02-27 at 12:23 -0800, Noriko Hosoi wrote:
> >> Hi William,
> >>
> >> A couple of questions (as usual... :)
> >>
> >> On 02/26/2017 03:51 PM, William Brown wrote:
> >>> Hi,
> >>>
> >>> I think that we should have a flag of a backend as local or remote for
> >>> DS.
> >> Does the remote backend mean the referral and/or the chaining backend?
> >> Or could there be more?
> > At the moment I only meant referral and chaining are "remote". LDBM is
> > local.
> Ok...  I thought the chaining backend could be tricky. :)  For instance, 
> the frontend server could have all the ACIs not just for the local 
> backend as well as for the chained backend servers...  If that's the 
> case, as long as the entry exists, the acl could be handled locally, but 
> maybe that's not the main issue at all for now... :p  So, please never mind.

Well, this could be the case, but the logic still stands when you think
about the process, and how we get entries from the backend and do the
aci checks  

> >>> The reason for this is to move the ACI chechk outside of LDBM into
> >>> do_() but only for local backends.
> >> The outside means pre- backend or post- backend or both?
> >>
> >> If ACI contains attribute value, how it'd be taken care of?
> > That is a good question.
> >
> > Right now we perform the acl checks in ldbm_.c, generally before we
> > call any pre operation plugins. Before this, the backend retrieves the
> > entry from the cache as needed.
> >
> > I'm suggesting that instead of keeping all this logic in ldbm, we
> > separate it up so that in rough pseudo code:
> >
> > do_
> > if local_be:
> >  e = be_get_entry()
> >  plugin_call_acl_plugin(e, op, ...)
> >  plugin_call_plugins(e, ...)
> >  be_commit_(e, mods )
> > else:
> >  forward_remote_operation
> >
> > The idea is that we can seperate the operation logic of the server
> > ( plugin calls, acl checks) from the backend.
> How about evaluating the ACLs for the changes made by the operation?  
> be_get_entry returns the entry on which operation is already done?  
> (Unless being add, moved, renamed, you may not be able to check the 
> ACL?)  Isn't it sometimes quite expensive?

This was a generic template, for a specific op, we can still do this. If
you look at ldbm_add.c, the acl check takes the pblock with mods, the
entry, etc. So really, this still works, because in the do_add for
example, we have the mods in the pb, and then we just step through the
entry retrival and check.


> > Right now, LDBM is really
> > tied into how an operation flows and operates, and it shouldn't be
> > involved at all. LDBM should be responsible for a cache, and how entries
> > are on disk, how we apply filters and indexes to that.
> >
> > Where as the server outside should direct plugin operations, access
> > control and others.
> >
> > Think about if we want to implement another backend type: Well we would
> > have to duplicate all these calls to plugin hooks, acls, etc, and they
> > may be out of order, or introduce other subtle issues.
> >
> > So having the local vs remote flag, really allows us to start to unpick
> > this, because we can make the checks happen in the do_ component -
> > This means implementing a new backend will be easier for us in the
> > future, and we can confine our server logic to one place.
> +1  I think that's a good plan.

Yeah, it's just the specifics of how we achieve it. This idea of a local
vs remote flag could even perhaps be an "ACL check flag", and we have
others as needed. That way we can start to clean up our logic a bit. The
do_ functions are really large and so I think this would help to
break up the steps nicely, same with the ldbm_ functions. 

> >
> >>> This way when we support multiple local backend types (which we do
> >>> already with fedse etc), instead of each backend suppling the need to
> >>> call acl's, we can do it once in the do_ at the correct step.
> >>>
> >>> We don't need this for remote backends as we can not guarantee we parse
> >>> the ACI and that adds extra complexity to the situation. We assume the
> >>> remote backend does the ACI check for us via some kind of passthrough
> >>> bind, or other means.
> >>>
> >>> Additionally, this fla

[389-devel] Review: certdetection breaks some tests

[William Brown]

https://pagure.io/lib389/issue/4

https://pagure.io/lib389/issue/raw/files/6c8f9ff82540af2043294576bbe525eb7bb465bb14d03eca0b6a9e87838d33f1-0001-Ticket-4-Cert-detection-breaks-some-tests.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Discuss: local vs remote be flag

[William Brown]

On Mon, 2017-02-27 at 12:23 -0800, Noriko Hosoi wrote:
> Hi William,
> 
> A couple of questions (as usual... :)
> 
> On 02/26/2017 03:51 PM, William Brown wrote:
> > Hi,
> >
> > I think that we should have a flag of a backend as local or remote for
> > DS.
> Does the remote backend mean the referral and/or the chaining backend?  
> Or could there be more?

At the moment I only meant referral and chaining are "remote". LDBM is
local. 

> > The reason for this is to move the ACI chechk outside of LDBM into
> > do_() but only for local backends.
> The outside means pre- backend or post- backend or both?
> 
> If ACI contains attribute value, how it'd be taken care of?

That is a good question. 

Right now we perform the acl checks in ldbm_.c, generally before we
call any pre operation plugins. Before this, the backend retrieves the
entry from the cache as needed. 

I'm suggesting that instead of keeping all this logic in ldbm, we
separate it up so that in rough pseudo code:

do_
if local_be:
e = be_get_entry()
plugin_call_acl_plugin(e, op, ...)
plugin_call_plugins(e, ...)
be_commit_(e, mods )
else:
forward_remote_operation 

The idea is that we can seperate the operation logic of the server
( plugin calls, acl checks) from the backend. Right now, LDBM is really
tied into how an operation flows and operates, and it shouldn't be
involved at all. LDBM should be responsible for a cache, and how entries
are on disk, how we apply filters and indexes to that. 

Where as the server outside should direct plugin operations, access
control and others.

Think about if we want to implement another backend type: Well we would
have to duplicate all these calls to plugin hooks, acls, etc, and they
may be out of order, or introduce other subtle issues.

So having the local vs remote flag, really allows us to start to unpick
this, because we can make the checks happen in the do_ component -
This means implementing a new backend will be easier for us in the
future, and we can confine our server logic to one place.

> > This way when we support multiple local backend types (which we do
> > already with fedse etc), instead of each backend suppling the need to
> > call acl's, we can do it once in the do_ at the correct step.
> >
> > We don't need this for remote backends as we can not guarantee we parse
> > the ACI and that adds extra complexity to the situation. We assume the
> > remote backend does the ACI check for us via some kind of passthrough
> > bind, or other means.
> >
> > Additionally, this flag would allow other plugins to distinguish if they
> > should operate or not on the operation - We may not want to apply
> > memberof to a remote BE for example.
> >
> >
> >
> > ___
> > 389-devel mailing list -- 389-devel@lists.fedoraproject.org
> > To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org
> 
> 
> _______
> 389-devel mailing list -- 389-devel@lists.fedoraproject.org
> To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Discuss: local vs remote be flag

[William Brown]

Hi,

I think that we should have a flag of a backend as local or remote for
DS.

The reason for this is to move the ACI chechk outside of LDBM into
do_() but only for local backends.

This way when we support multiple local backend types (which we do
already with fedse etc), instead of each backend suppling the need to
call acl's, we can do it once in the do_ at the correct step.

We don't need this for remote backends as we can not guarantee we parse
the ACI and that adds extra complexity to the situation. We assume the
remote backend does the ACI check for us via some kind of passthrough
bind, or other means. 

Additionally, this flag would allow other plugins to distinguish if they
should operate or not on the operation - We may not want to apply
memberof to a remote BE for example. 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49142 bytes vs unicode in test causes failure.

[William Brown]

https://pagure.io/389-ds-base/issue/49142

https://pagure.io/389-ds-base/issue/raw/files/9a99355426a501aa28bd90c43c9ee9be6cc762888d3db96a6041ddc90fae8278-0001-Ticket-49142-bytes-vs-unicode-in-plugin-tests.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Prevent systemd killing us during large cl replay

[William Brown]

https://pagure.io/389-ds-base/issue/49138

https://pagure.io/389-ds-base/issue/raw/files/f940e785a77b0dd8a9f986e47fd6daee15627dd75c8542b0b8f4116fa970498e-0001-Ticket-49138-Increase-systemd-timout.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review; fix test 48212

[William Brown]

https://pagure.io/389-ds-base/issue/49140


https://pagure.io/389-ds-base/issue/raw/files/1c3fbba12dcf959014f2e2e295e706f94922771bdc5663b44cf4577b6b7c7564-0001-Ticket-49140-Remove-legacy-inst-reference-in-test.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Build failed in Jenkins: 389-DS-NIGHTLY #165

[William Brown]

> >>
> > I know that everyone is busy.
> >
> > But in our team we have a policy that new patches should not be pushed
> > if tests are not green.
> Your tests are obviously more stable than ours.  In fact our "nightly
> tests" are a fairly new process.  We were even hesitant to start doing
> this due to the inconsistency of some of the tests.  Like I said before,
> our testing framework has also underwent some heavy construction
> recently.  Anyway, we will try and do better. 
> 
> For now I will remove the reports from the public mailing list so you,
> and others, are not spammed with the daily failures.  Once we get this
> sorted out I will restore those settings.
> 

Just submitted patches for all three issues.


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Remove hardcoded elements from dblock test

[William Brown]

https://pagure.io/389-ds-base/issue/49134

https://pagure.io/389-ds-base/issue/raw/files/bfd0e7407d8f0898afceb38436a196cb2a473c9a729782e049cc72cbdf8eccfd-0001-Ticket-49134-Remove-hardcoded-elements-from-db-lock-.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: SDN premangaling fix

[William Brown]

https://pagure.io/389-ds-base/issue/49086

https://pagure.io/389-ds-base/issue/raw/files/5bac3f1381e0bb05f37ce59978a42805453ce8655c397230b4ef7ea25040f002-0001-Ticket-49086-SDN-premangaling-broken-after-SASL-chan.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] lib389 pytest check version relies on root

[William Brown]

https://pagure.io/lib389/issue/2

https://pagure.io/lib389/issue/raw/files/d4c7c7d291da39439a49742c5af6d41485f681f35fe45dca42be9c456ee97386-0001-Ticket-2-pytest-mark-with-version-relies-on-root.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review, NS22 add ns_job_done docs for persist

[William Brown]

https://pagure.io/nunc-stans/issue/22

https://pagure.io/nunc-stans/issue/raw/files/2e5e61d638558a1db80f4212fa9ba5fe2aa7cab7aa5e9ac9ad4213d97cc0660c-0001-Ticket-22-Document-that-we-must-call-ns_job_done-on-.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: NS80, decouple NSPR for mutex and thread

[William Brown]

https://pagure.io/nunc-stans/issue/80

https://pagure.io/nunc-stans/issue/raw/files/243764927dafe3b5dadc8e59dba85c41befa5238e39a297eb0883e42a55b6baf-0001-Ticket-80-Decouple-nunc-stans-from-NSPR.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: GSoc 2017:Regarding the project developing administrative tools for 389 directory server

[William Brown]

On Mon, 2017-02-13 at 01:51 -0800, Asantha Thilina wrote:
> Hi all,
> 
> I am final year software engineering student at srilanka institute of
> information technology and i would like to contribute to project *developing
> administrative tools for 389 directory server* for GSoC and i would like to
> know more about the project and would be grateful if someone can guide me
> on this

Hi there,

I'm the developer who submitted to the Fedora GSOC project.

If you want to know more I would advise that you read about dsadm and
dsconf on our wiki:

http://www.port389.org/docs/389ds/design/dsadm-dsconf.html

A lot of work has already gone into the project, and can be found in the
lib389 repository. This is here:

https://pagure.io/lib389/tree/master

This is a system able to setup and build new Directory Server instances.
The command line tools are found:

https://pagure.io/lib389/blob/master/f/cli
https://pagure.io/lib389/blob/master/f/lib389/cli_conf
https://pagure.io/lib389/blob/master/f/lib389/tests/cli

These work by consuming our LDAP orm system which is implemented here:

https://pagure.io/lib389/blob/master/f/lib389/_mapped_object.py

And an example of the consumption of this is:

https://pagure.io/lib389/blob/master/f/lib389/plugins.py


* What would be your assigned task:

-- We would be looking for you to setup and deploy your own ldap server,
and understand how the plugins work.
-- We would then want you to add the components to plugins.py, so that
tests and the cli can manipulate those plugins.
-- You would be wiring in the plugin management command to cli_conf
-- finally, and most importantly, the cli and plugins.py code is fully
unit tested. 

I hope this helps you,

___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Q. on lib389 Re: Please review: #49121 ns-slapd crashes in ldif_sput due to the output buf size is less than the real size.

[William Brown]

On Mon, 2017-02-13 at 09:31 -0800, Noriko Hosoi wrote:
> On 02/13/2017 08:51 AM, Lukas Slebodnik wrote:
> > On (13/02/17 08:38), Noriko Hosoi wrote:
> >> On 02/12/2017 06:14 PM, William Brown wrote:
> >>> On Sun, 2017-02-12 at 17:42 -0800, Noriko Hosoi wrote:
> >>>> On 02/12/2017 03:51 PM, Noriko Hosoi wrote:
> >>>>> https://pagure.io/389-ds-base/issue/49121
> >>>>>
> >>>>> https://pagure.io/389-ds-base/issue/raw/files/d857ff4919940bcebeae870774896783f7b6e86ce08a2c2e924610ac2335f8de-0001-Ticket-49121-ns-slapd-crashes-in-ldif_sput-due-to-th.patch
> >>>>>
> >>>>> These odd » characters are shown at some part of the patch.  I wonder
> >>>>> what causes this display issue...  And non-ascii characters are not
> >>>>> printed correctly although they are in the "View Raw" mode.  Please
> >>>>> see the utf8str.txt file.
> >>>>>
> >>>>> 277  @@ -1499,30 +1506,36 @@ entry2str_internal_size_attrlist( const
> >>>>> Slapi_Attr *attrlist, int entry2str_ctrl
> >>>>> 280   »   »   /* Count the space required for the present and 
> >>>>> deleted values */
> >>>>> 281  -» » elen+= entry2str_internal_size_valueset(a->a_type,
> >>>>> >a_present_values,
> >>>>> 282  -» » » » » » » » » » » » entry2str_ctrl, attribute_state,
> >>>>> 283  -» » » » » » » » » » » » VALUE_PRESENT); 305  +» » elen += 
> >>>>> entry2str_internal_size_valueset(a, a->a_type,
> >>>>> >a_present_values,
> >>>>> 306  +» » entry2str_ctrl, attribute_state, VALUE_PRESENT);
> >>>>>
> >>>>>
> >>>>> Note: The test build was blessed by the bug reporter.
> >>>> Thanks to William for his reviews.  I've update the patch based on his
> >>>> suggestion.
> >>>>
> >>>> I also have 2 lib389 patches (attached to this email).  Is lib389 still
> >>>> in fedorahosted?  I cloned pagure.io/lib389.git, but I found it empty...
> >>>>
> >>>> 1) could you please review the patches?
> >>> I'm happy with the dbscan change
> >>>
> >>> I don't use the valgrind wrapper myself (it should disable transparently
> >>> when ASAN is enabled).
> >> Are there any chance to support both valgrind and ASAN?  Well, I'm not at 
> >> the
> >> position to insist anything any more here :p, but they are just tools and
> >> supporting both of them is not a bad idea, is it?
> > FYI:
> > My experienceis that you should use either ASAN or valgrind.
> > They do not work well togeteher. That's the same for other sanitizers.

They *can not* be operated together. They both try to intercept malloc,
and they break each other. 

> I see...  Thanks, Lukas!
> 
> The current test scripts call valgrind APIs only if it's built without 
> ASAN enabled as follows.  That is, if ASAN is enabled in the CI, the 
> valgrind check won't be executed.  Probably, we could move this 
> "has_asan" check to the inside of the valgrind APIs and keep the APIs in 
> lib389?

I thought I had already done that? It really only affects me anyway. 

> 
> *if not topology_m2.ms["master1"].has_asan():*
>  results_file = valgrind_get_results_file(topology_m2.ms["master1"])
> 
> Another question is, when the CI test enables ASAN, is this test case -- 
> checking the valgrind result and determining the test case succeeded or 
> failed -- still valid?  Or should we rewrite it based upon the ASAN 
> syntax (if any) later?

We can use both in the way here, I don't see a reason to change. In the
ASAN mode, the DS exits with a fail code, which crashes the test, even
if the valgrind wrapper "passes".

> 
> Thanks,
> --noriko
> >
> > ASAN is faster but valgrind should catch more bugs.
> > Of course it can change in future :-)

I disagree, but that's what we are entitled to do.

ASAN is easier to setup (compile only, no need to hack scripts),
triggers errors faster (lib389 picks up errors immediately, no need to
parse output) gives better traces (shows where the leak occured, where
it was allocated, and can even show memory addresses for you to set
watch points on to debug) and it's faster. 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Improve test assertions in nunc-stans

[William Brown]

https://pagure.io/nunc-stans/issue/79

https://pagure.io/nunc-stans/issue/raw/files/b638149eb2f3fd76c29d5b5c2e9dafb698538e10accc8fb9837d8d906a2da9e2-0001-Ticket-79-Add-assertion-that-event-is-not-registerd-.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Q. on lib389 Re: Please review: #49121 ns-slapd crashes in ldif_sput due to the output buf size is less than the real size.

[William Brown]

On Sun, 2017-02-12 at 17:42 -0800, Noriko Hosoi wrote:
> On 02/12/2017 03:51 PM, Noriko Hosoi wrote:
> >
> > https://pagure.io/389-ds-base/issue/49121
> >
> > https://pagure.io/389-ds-base/issue/raw/files/d857ff4919940bcebeae870774896783f7b6e86ce08a2c2e924610ac2335f8de-0001-Ticket-49121-ns-slapd-crashes-in-ldif_sput-due-to-th.patch
> >
> > These odd » characters are shown at some part of the patch.  I wonder 
> > what causes this display issue...  And non-ascii characters are not 
> > printed correctly although they are in the "View Raw" mode.  Please 
> > see the utf8str.txt file.
> >
> > 277  @@ -1499,30 +1506,36 @@ entry2str_internal_size_attrlist( const 
> > Slapi_Attr *attrlist, int entry2str_ctrl
> > 280   »   »   /* Count the space required for the present and 
> > deleted values */
> > 281  -» » elen+= entry2str_internal_size_valueset(a->a_type, 
> > >a_present_values,
> > 282  -» » » » » » » » » » » » entry2str_ctrl, attribute_state,
> > 283  -» » » » » » » » » » » » VALUE_PRESENT); 305  +» » elen += 
> > entry2str_internal_size_valueset(a, a->a_type, 
> > >a_present_values,
> > 306  +» » entry2str_ctrl, attribute_state, VALUE_PRESENT);
> >
> >
> > Note: The test build was blessed by the bug reporter.
> 
> Thanks to William for his reviews.  I've update the patch based on his 
> suggestion.
> 
> I also have 2 lib389 patches (attached to this email).  Is lib389 still 
> in fedorahosted?  I cloned pagure.io/lib389.git, but I found it empty...
> 
> 1) could you please review the patches?

I'm happy with the dbscan change

I don't use the valgrind wrapper myself (it should disable transparently
when ASAN is enabled). I'm not sure about the check for sbin dir though,
because it shouldn't matter what we use. Is there a reason you need to
check for root with just those dirs? Is it related to where the DS
instance is from (prefix build vs rpm)

> 2) if they look fine, is it ok to push them to fedorahosted git repository?

I would ask Mark this, he knows the most about the situation. 


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: object management tool

[William Brown]

https://fedorahosted.org/389/ticket/49126

https://fedorahosted.org/389/attachment/ticket/49126/0001-Ticket-49126-DIT-management-tool.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Memberof: blocking direct writes to memberof attr in add/mod

[William Brown]

What do you think of this?

https://fedorahosted.org/389/ticket/47557#comment:15

If you want, I can write up a full design doc. I think this is the best
way, rather than each plugin trying to intercept and block the
operation, if we provide a generic way to register and block these. 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: pre-release bump to nunc-stans to avoid a build issue

[William Brown]

https://pagure.io/nunc-stans/issue/83

https://pagure.io/nunc-stans/issue/raw/files/eaf3ee7335e79b82f54cea8a91c6689f1d04bac7723d81781af3cedfcb6288a5-0001-Ticket-83-bump-version-to-master-to-0.2.2-pre.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: plugin compat tests

[William Brown]

https://fedorahosted.org/389/ticket/49086

https://fedorahosted.org/389/attachment/ticket/49086/0001-Ticket-49086-public-api-compatability-test-for-SDN-c.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: pblock analytics

[William Brown]

https://fedorahosted.org/389/ticket/49116

https://fedorahosted.org/389/attachment/ticket/49116/0001-Ticket-49116-Pblock-usage-analytics.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] 49086: Target spec and target address

[William Brown]

Hi,

While looking into the failure of test 48272 (addn), I realised that we
were using both target_address and target_spec in the operation struct.
The pblock is a bit inconsistent about their usage also.

I've attached a list of the places we use them. 

I think that this is a mistake, we really only need "one" target value
(probably target_spec). 

I want to propose that we drop target_address for target_spec, update
the pblock to use operation_set_target_spec as needed. 

However, I don't know the full history of these values, so I would like
your input to this design. Is there a difference between target_spec and
target_address?

Thanks,


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane
[william@rei 13:15] ~/development/389ds/ds I0> grep -r -n -e 'target_address' 
ldap/servers/plugins/replication/replutil.c:440:if 
(op->target_address.uniqueid == NULL)
ldap/servers/plugins/replication/replutil.c:447:if (op->target_address.sdn 
== NULL)
ldap/servers/plugins/replication/windows_inc_protocol.c:1099:return (strcmp 
(op->target_address.uniqueid, START_ITERATION_ENTRY_UNIQUEID) == 0);
ldap/servers/plugins/replication/windows_inc_protocol.c:1341:   
entry.op->target_address.uniqueid, csn_str,
ldap/servers/plugins/replication/windows_inc_protocol.c:1355:   
entry.op->target_address.uniqueid, csn_str,
ldap/servers/plugins/replication/windows_inc_protocol.c:1366:   
entry.op->target_address.uniqueid, csn_str,
ldap/servers/plugins/replication/windows_inc_protocol.c:1381:   
entry.op->target_address.uniqueid, csn_str);
ldap/servers/plugins/replication/repl5_plugins.c:1136:  
op_params->target_address.uniqueid = slapi_ch_strdup (uniqueid);
ldap/servers/plugins/replication/repl5_plugins.c:1167:  
REPL_GET_DN(_params->target_address),
ldap/servers/plugins/replication/repl5_plugins.c:1168:  
op_params->target_address.uniqueid,
ldap/servers/plugins/replication/repl5_plugins.c:1177:  
slapi_ch_free((void**)_params->target_address.uniqueid);
ldap/servers/plugins/replication/repl5_plugins.c:1193:  const char *dn 
= op_params ? REPL_GET_DN(_params->target_address) : "unknown";
ldap/servers/plugins/replication/repl5_plugins.c:1194:  Slapi_DN *sdn = 
op_params ? (_params->target_address)->sdn : NULL;
ldap/servers/plugins/replication/repl5_plugins.c:1195:  char *uniqueid 
= op_params ? op_params->target_address.uniqueid : "unknown";
ldap/servers/plugins/replication/windows_protocol_util.c:1550:  local_dn = 
slapi_sdn_dup( op->target_address.sdn );
ldap/servers/plugins/replication/windows_protocol_util.c:1559:  rc = 
windows_get_local_entry_by_uniqueid(prp, op->target_address.uniqueid, 
_entry, 0);
ldap/servers/plugins/replication/windows_protocol_util.c:1561:  rc = 
windows_get_local_tombstone_by_uniqueid(prp, op->target_address.uniqueid, 
_entry);
ldap/servers/plugins/replication/windows_protocol_util.c:1570:  
  op->target_address.uniqueid, _entry, 1 /* is_global */);
ldap/servers/plugins/replication/windows_protocol_util.c:1577:  
REPL_GET_DN(>target_address));
ldap/servers/plugins/replication/windows_protocol_util.c:1590:  
REPL_GET_DN(>target_address));
ldap/servers/plugins/replication/windows_protocol_util.c:1596:  
REPL_GET_DN(>target_address), "ours");
ldap/servers/plugins/replication/windows_protocol_util.c:1615:  
REPL_GET_DN(>target_address), is_ours ? "ours" : "not ours", 
ldap/servers/plugins/replication/windows_protocol_util.c:1629:  
REPL_GET_DN(>target_address),
ldap/servers/plugins/replication/windows_protocol_util.c:1637:  
REPL_GET_DN(>target_address), slapi_sdn_get_dn(remote_dn));
ldap/servers/plugins/replication/repl5_inc_protocol.c:1339: if 
(create_NSDS50ReplUpdateInfoControl(op->target_address.uniqueid,
ldap/servers/plugins/replication/repl5_inc_protocol.c:1353: 
op2string(op->operation_type), 
REPL_GET_DN(>target_address),
ldap/servers/plugins/replication/repl5_inc_protocol.c:1387: 

REPL_GET_DN(>target_address),
ldap/servers/plugins/replication/repl5_inc_protocol.c:1392: 
return_value = conn_send_add(prp->conn, 
REPL_GET_DN(>target_address),
ldap/servers/plugins/replication/repl5_inc_protocol.c:1412: 
   

[389-devel] 49086: Target spec and target address

[William Brown]

Hi,

While looking into the failure of test 48272 (addn), I realised that we
were using both target_address and target_spec in the operation struct.
The pblock is a bit inconsistent about their usage also.

I've attached a list of the places we use them. 

I think that this is a mistake, we really only need "one" target value
(probably target_spec). 

I want to propose that we drop target_address for target_spec, update
the pblock to use operation_set_target_spec as needed. 

However, I don't know the full history of these values, so I would like
your input to this design. Is there a difference between target_spec and
target_address?

Thanks,

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane
[william@rei 13:15] ~/development/389ds/ds I0> grep -r -n -e 'target_address' 
ldap/servers/plugins/replication/replutil.c:440:if 
(op->target_address.uniqueid == NULL)
ldap/servers/plugins/replication/replutil.c:447:if (op->target_address.sdn 
== NULL)
ldap/servers/plugins/replication/windows_inc_protocol.c:1099:return (strcmp 
(op->target_address.uniqueid, START_ITERATION_ENTRY_UNIQUEID) == 0);
ldap/servers/plugins/replication/windows_inc_protocol.c:1341:   
entry.op->target_address.uniqueid, csn_str,
ldap/servers/plugins/replication/windows_inc_protocol.c:1355:   
entry.op->target_address.uniqueid, csn_str,
ldap/servers/plugins/replication/windows_inc_protocol.c:1366:   
entry.op->target_address.uniqueid, csn_str,
ldap/servers/plugins/replication/windows_inc_protocol.c:1381:   
entry.op->target_address.uniqueid, csn_str);
ldap/servers/plugins/replication/repl5_plugins.c:1136:  
op_params->target_address.uniqueid = slapi_ch_strdup (uniqueid);
ldap/servers/plugins/replication/repl5_plugins.c:1167:  
REPL_GET_DN(_params->target_address),
ldap/servers/plugins/replication/repl5_plugins.c:1168:  
op_params->target_address.uniqueid,
ldap/servers/plugins/replication/repl5_plugins.c:1177:  
slapi_ch_free((void**)_params->target_address.uniqueid);
ldap/servers/plugins/replication/repl5_plugins.c:1193:  const char *dn 
= op_params ? REPL_GET_DN(_params->target_address) : "unknown";
ldap/servers/plugins/replication/repl5_plugins.c:1194:  Slapi_DN *sdn = 
op_params ? (_params->target_address)->sdn : NULL;
ldap/servers/plugins/replication/repl5_plugins.c:1195:  char *uniqueid 
= op_params ? op_params->target_address.uniqueid : "unknown";
ldap/servers/plugins/replication/windows_protocol_util.c:1550:  local_dn = 
slapi_sdn_dup( op->target_address.sdn );
ldap/servers/plugins/replication/windows_protocol_util.c:1559:  rc = 
windows_get_local_entry_by_uniqueid(prp, op->target_address.uniqueid, 
_entry, 0);
ldap/servers/plugins/replication/windows_protocol_util.c:1561:  rc = 
windows_get_local_tombstone_by_uniqueid(prp, op->target_address.uniqueid, 
_entry);
ldap/servers/plugins/replication/windows_protocol_util.c:1570:  
  op->target_address.uniqueid, _entry, 1 /* is_global */);
ldap/servers/plugins/replication/windows_protocol_util.c:1577:  
REPL_GET_DN(>target_address));
ldap/servers/plugins/replication/windows_protocol_util.c:1590:  
REPL_GET_DN(>target_address));
ldap/servers/plugins/replication/windows_protocol_util.c:1596:  
REPL_GET_DN(>target_address), "ours");
ldap/servers/plugins/replication/windows_protocol_util.c:1615:  
REPL_GET_DN(>target_address), is_ours ? "ours" : "not ours", 
ldap/servers/plugins/replication/windows_protocol_util.c:1629:  
REPL_GET_DN(>target_address),
ldap/servers/plugins/replication/windows_protocol_util.c:1637:  
REPL_GET_DN(>target_address), slapi_sdn_get_dn(remote_dn));
ldap/servers/plugins/replication/repl5_inc_protocol.c:1339: if 
(create_NSDS50ReplUpdateInfoControl(op->target_address.uniqueid,
ldap/servers/plugins/replication/repl5_inc_protocol.c:1353: 
op2string(op->operation_type), 
REPL_GET_DN(>target_address),
ldap/servers/plugins/replication/repl5_inc_protocol.c:1387: 

REPL_GET_DN(>target_address),
ldap/servers/plugins/replication/repl5_inc_protocol.c:1392: 
return_value = conn_send_add(prp->conn, 
REPL_GET_DN(>target_address),
ldap/servers/plugins/replication/repl5_inc_protocol.c:1412: 
   

[389-devel] Please review: 49027 - Do not store clear on secfailure

[William Brown]

https://fedorahosted.org/389/ticket/49027

https://fedorahosted.org/389/attachment/ticket/49027/0001-Ticket-49027-on-secfailure-do-not-store-cleartext-pa.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Nunc Stans, make tevent optional

[William Brown]

https://pagure.io/nunc-stans/issue/70

https://pagure.io/nunc-stans/issue/raw/files/8dd0fb698209522beda23f835614a7c448f35763e6dae7cb770888d59acc48e9-0001-Ticket-70-Make-tevent-an-optional-component-of-nunc-.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 48787, FreeBSD support for Directory Server

[William Brown]

https://fedorahosted.org/389/ticket/48797

https://fedorahosted.org/389/attachment/ticket/48797/0001-Ticket-48797-Add-freebsd-support-to-ns-slapd-main.2.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: DS autotuning by default

[William Brown]

https://fedorahosted.org/389/ticket/49028

https://fedorahosted.org/389/attachment/ticket/49028/0001-Ticket-49028-Autosize-database-cache-by-default.patch



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Nunc-Stans 75, 76

[William Brown]

Hi,

This is a more complex review than normal due to the addition of a new
dependency.

https://pagure.io/nunc-stans/issue/75

https://pagure.io/nunc-stans/issue/76

Related tickets:

https://pagure.io/nunc-stans/issue/67

https://pagure.io/nunc-stans/issue/40

https://pagure.io/nunc-stans/issue/73


I want to propose that we adopt a new library I have created, libsds,
for consumption in nunc-stans and soon directory server. 

https://pagure.io/libsds

libsds contains a single threaded queue, mutex protected queue, lfds
queue wrapper, single thread b+tree, and soon a copy on write b+tree and
lru based on it.

The benefit of this abstraction is:

* reduce the need to add data structures to Directory Server. We have
many custom structures in various modules (bst in valueset, avl for aci,
etc). 
* Testing. The structures in ds are tested only through "production
use". libsds comes with extensive cmocka test suites. 
* Build process, lfds makes the nunc-stans build very complex and
fragile. Libsds wraps this much better, and makes the process very
stable and clean.
* Abstraction. Improvements to libsds will benefit all of DS, rather
than just small parts we have to hand hold at the moment.
* Portability. Libsds supports detection of arch and platform, meaning
when we can not provide a lock free queue, we fall back to the mutex
queue seamlessly.
* Less duplication. The lfds queue requires some handholding to work.
libsds wraps this up. Additionally, encourages us to use these
structures if they are as easy as "init(), push(), pop()". 

As far as Directory Server is concerned, we would bundle and build
libsds just like we currently do for nunc-stans. 

A large benefit of this abstraction is portability, allowing support of
nunc-stans on other platforms like FreeBSD, linux arm64, or other cpus. 

Here is the libsds source:

https://pagure.io/libsds/tree/master

Here is the patch to allow nunc-stans to use this:

https://pagure.io/nunc-stans/issue/raw/files/3f1bb8192449719c74e7c1f1787a8b37f69df3204920ccb5395f090323173db5-0001-Ticket-73-67-40-Replace-nunc-stans-datastructures-wi.patch

https://pagure.io/nunc-stans/issue/raw/files/862f38cbd70574adbc78e7f973776d5a9df0a0d89277c3c30ef413a444c40343-0002-Ticket-73-67-40-Replace-lfds-wrapper.patch

Finally, the patch to allow nunc-stans on FreeBSD

https://pagure.io/nunc-stans/issue/raw/files/ff0a32398f22b3009f01ed3898931719ba1529e721c79d9f4aa6f240d51ed6d8-0003-Ticket-75-Port-nunc-stans-to-freebsd.patch


After this, an extra patch will be needed for Directory Server to
support building a bundle of libsds. I want to use the libsds queue for
the DS logging subsystem soon and the b+tree for replacement of the
conntable in connection.c, so I would really like to see this included.

Thanks,

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: lib389 49084 and 48413

[William Brown]

https://fedorahosted.org/389/ticket/48413

https://fedorahosted.org/389/attachment/ticket/48413/0001-Ticket-48413-Improvements-to-lib389-for-rest.patch

https://fedorahosted.org/389/ticket/49084

https://fedorahosted.org/389/attachment/ticket/49084/0001-Ticket-49084-Configshell-for-cli-tools.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review 49066

[William Brown]

https://fedorahosted.org/389/ticket/49066

https://fedorahosted.org/389/attachment/ticket/49066/0001-Ticket-49066-Memory-leaks-in-server-part-2.2.patch
-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Readme for nunc-stans for pagure

[William Brown]

https://pagure.io/nunc-stans/issue/72


https://pagure.io/nunc-stans/issue/raw/files/52eb1aca39cb95dd608f30fa6ab523b7f2705afdea6e1e7ceb0e2c78ffbf1577-0001-Ticket-72-Add-readme-to-pagure.patch


-- 
William Brown <will...@blackhats.net.au>


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49066 memory leak resolutions

[William Brown]

https://fedorahosted.org/389/ticket/49066

https://fedorahosted.org/389/attachment/ticket/49066/0001-Ticket-49066-Memory-leaks-in-server-part-2.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 48835 - Fix rpm builds for RHEL7

[William Brown]

https://fedorahosted.org/389/ticket/48835

https://fedorahosted.org/389/attachment/ticket/48835/0002-Ticket-48835-package-tests-into-python-site-packages.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49068 fix std c99 on 1.3.5

[William Brown]

https://fedorahosted.org/389/ticket/49068

https://fedorahosted.org/389/attachment/ticket/49068/0001-Ticket-49068-1.3.5-use-std-c99.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Various tickets

[William Brown]

https://fedorahosted.org/389/ticket/49041

https://fedorahosted.org/389/ticket/49066

https://fedorahosted.org/389/ticket/48835


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review patch 1: fix memory leaks

[William Brown]

https://fedorahosted.org/389/ticket/49066

https://fedorahosted.org/389/attachment/ticket/49066/0001-Ticket-49066-Memory-leaks-in-server.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Move DS tests in RPM to proper python locations

[William Brown]

https://fedorahosted.org/389/ticket/48835

https://fedorahosted.org/389/attachment/ticket/48835/0001-Ticket-48835-Tests-with-setup.py.in.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49052 system breaks asan builds

[William Brown]

https://fedorahosted.org/389/ticket/49052

https://fedorahosted.org/389/attachment/ticket/49052/0001-Ticket-49052-Environment-quoting-on-fedora-causes-ds.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Build failed in Jenkins: 389-DS-NIGHTLY #149

[William Brown]

On Thu, 2016-12-01 at 09:16 +1000, William Brown wrote:
> > CRITICAL:stress_tests:DeleteUsers: failed to delete 
> > (uid=person3,dc=example,dc=com) error: Can'\''t contact LDAP server
> > CRITICAL:stress_tests:DeleteUsers: failed to delete 
> > (uid=employee4,dc=example,dc=com) error: Can'\''t contact LDAP server
> > CRITICAL:stress_tests:DeleteUsers: failed to delete 
> > (uid=entry4,dc=example,dc=com) error: Can'\''t contact LDAP server
> > Exception in thread Thread-37:
> > Traceback (most recent call last):
> >   File "/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner
> > self.run()
> >   File 
> > "<http://vm-058-081.abc.idm.lab.eng.brq.redhat.com:8080/job/389-DS-NIGHTLY/ws/source/ds/dirsrvtests/tests/suites/dynamic-plugins/stress_tests.py;,>
> >  line 92, in run
> > assert False
> > AssertionError: assert False
> > 
> 
> 
> I will also be investigating this error today. 
> 

https://fedorahosted.org/389/ticket/48894



-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Build failed in Jenkins: 389-DS-NIGHTLY #149

[William Brown]

> CRITICAL:stress_tests:DeleteUsers: failed to delete 
> (uid=person3,dc=example,dc=com) error: Can'\''t contact LDAP server
> CRITICAL:stress_tests:DeleteUsers: failed to delete 
> (uid=employee4,dc=example,dc=com) error: Can'\''t contact LDAP server
> CRITICAL:stress_tests:DeleteUsers: failed to delete 
> (uid=entry4,dc=example,dc=com) error: Can'\''t contact LDAP server
> Exception in thread Thread-37:
> Traceback (most recent call last):
>   File "/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner
> self.run()
>   File 
> "<http://vm-058-081.abc.idm.lab.eng.brq.redhat.com:8080/job/389-DS-NIGHTLY/ws/source/ds/dirsrvtests/tests/suites/dynamic-plugins/stress_tests.py;,>
>  line 92, in run
> assert False
> AssertionError: assert False
> 


I will also be investigating this error today. 


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49002 remove memsets where possible

[William Brown]

https://fedorahosted.org/389/ticket/49002

https://fedorahosted.org/389/attachment/ticket/49002/0001-Ticket-49002-Remove-memset-on-allocation.patch




-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Allow DS to consume schema from /usr and /etc in parallel

[William Brown]

https://fedorahosted.org/389/ticket/48163

https://fedorahosted.org/389/attachment/ticket/48163/0001-Ticket-48163-Read-schema-from-multiple-locations.4.patch

https://fedorahosted.org/389/attachment/ticket/48163/0002-Ticket-48163-Re-space-schema.c.2.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: lib389 extended op and sasl fixes

[William Brown]

https://fedorahosted.org/389/ticket/49056

https://fedorahosted.org/389/attachment/ticket/49056/0001-Ticket-48707-Implement-draft-wibrown-ldapssotoken-01.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review, 49054 fix compiler warnings

[William Brown]

https://fedorahosted.org/389/ticket/49054

https://fedorahosted.org/389/attachment/ticket/49054/0001-Ticket-49054-Fix-sasl_map-unused-paramater-compiler-.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 48894 entry state delete

[William Brown]

https://fedorahosted.org/389/ticket/48894#comment:9

https://fedorahosted.org/389/attachment/ticket/48894/0001-Ticket-48894-Issues-with-delete-of-entrywsi-with-lar.patch
-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Autotune threads 49021

[William Brown]

https://fedorahosted.org/389/ticket/49021

https://fedorahosted.org/389/attachment/ticket/49021/0001-Ticket-49021-Automatic-thread-tuning.2.patch

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: 49042 Fix nightly tests

[William Brown]

https://fedorahosted.org/389/ticket/49042

https://fedorahosted.org/389/attachment/ticket/49042/0001-Ticket-49042-Test-failure-that-expects-old-default.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Build failed in Jenkins: 389-DS-NIGHTLY #138

[William Brown]

Hi,

> topology =  0x7fee22ec8e90>
> attr = '\''nsslapd-dbcachesize'\'', expected_value = '\''1000'\'', 
> required = False
> 
> def _check_configured_value(topology, attr=DBLOCK_ATTR_CONFIG, 
> expected_value=None, required=False):
> entries = topology.standalone.search_s(ldbm_config, ldap.SCOPE_BASE, 
> '\''cn=config'\'')
> if required:
> assert(entries[0].hasValue(attr))
> elif entries[0].hasValue(attr):
> >   assert(entries[0].getValue(attr) == expected_value)
> E   assert '\''33554432'\'' == '\''1000'\''
> E - 33554432
> E + 1000
> 

I'm fixing this test now: It's a bit silly, but I broke it because I
increased this default value to make the server perform better ...

Patch shortly. 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Extending the pblock

[William Brown]

On Mon, 2016-11-21 at 12:53 -0500, Mark Reynolds wrote:
> 
> On 11/20/2016 08:17 PM, William Brown wrote:
> > Hi,
> >
> > I have to add some new getters to pblock.c, but I think we should talk
> > about how to add these.
> >
> > Right now, we have a nearly ~2000 line case switch statement in
> > slapi_pblock_get(pb, TYPE, *void). No matter how we cut it, this is
> > pretty insane.
> :-)  Yeah its a bit much at the moment.
> >  We have huge amounts of defines in slapi-plugin.h for
> > this, and we can't expose those to other langs (python, rust), because
> > they are in a C header file.
> >
> > As well, case switch statements that large, at some point, it's going to
> > be inefficient to access. 
> >
> > I think that we should try to break this up, it's just a bit much at
> > this point.
> >
> > So for new additions I would like to use:
> >
> > slapi_pblock_get_(pblock, out) {
> > *out = pb->item
> > }
> 
> This works for me, but if I may, I propose that we tweak the naming
> style to:
> 
> slapi_pb_get_(Slapi_PBlock *pb)
> slapi_pb_set_(Slapi_PBlock *pb, ...)
> 
> This way we differentiate the new functions, and two, it helps keep the
> function name from getting too long.

Okay, that could be the way to go.

I need to get something from the connection, so maybe I'll wire up a
patch as a demo of what I want to achieve. I certainly don't want to
extend the pblock, if anything I want to get us to break it down and
compact it. 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Please review: Increase default memory sizing

[William Brown]

https://fedorahosted.org/389/ticket/49042

https://fedorahosted.org/389/attachment/ticket/49042/0002-Ticket-49042-Increase-cache-defaults-slightly.patch


-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org

[389-devel] Re: Urgent review: 49041 ssl fails to start on f24

[William Brown]

> >>
> >> Any advice would be greatly appreciated...
> > My curiosity is only around how William found this bug in the first
> > place and what makes it so urgent.
> Ok.  Thanks, Rob!  Yes, I'm curious, too. :)

Please see

https://fedorahosted.org/389/ticket/49041#comment:3

The issue is *clearly* a DS behavioural bug where we do NOT detect the
presence of the sqlite formatted DB correctly. We then incorrectly
create invalid BDB files, and it "masks" the SQL format  from the server
start up.

As a result, after a server restart your certificates "vanish" and SSL
fails to start. (They are still in key4/cert9, but key3/cert8 are broken
and used preferentially).

That's why it's urgent ;) 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Brisbane


signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org