Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-21 Thread Martin Kosek
On 11/20/2012 09:24 PM, Rob Crittenden wrote:
> Martin Kosek wrote:
>> On 11/16/2012 05:18 PM, Rob Crittenden wrote:
>>> Nalin Dahyabhai wrote:
 On Thu, Nov 15, 2012 at 11:53:44PM -0500, Rob Crittenden wrote:
> In order for this to work you'll need to apply the last two patches
> (both 0001) to slapi-nis and spin it up yourself, otherwise you'll
> have serious deadlock issues. I know this is extra work but this
> patch is potentially disruptive so I figure the earlier it is out
> the better.
>
> Noriko/Rich/Nalin, can you guys review the slapi-nis pieces? I may
> have been too aggressive in my cleanup.
>
> Noriko/Rich, can you review the 389-ds plugin parts of my 1072 patch?
>
> Once we have an official slapi-nis build with these patches we'll
> need to set the minimum n-v-r in our spec file.

 Rob, the original patch was already applied.  I since reworked large
 parts of how it was organized to make it easier for me to read, and
 tagged the result as 0.43.  Have you tested the IPA changes in
 combination with the 0.44 builds from either ipa-devel or Fedora 18's
 updates-testing repository?

 Nalin

>>>
>>> I tested the 0.44 build and things are looking good. I'm not deadlocking, 
>>> so I
>>> guess that's the desired out come :-)
>>>
>>> I bulk loaded a few thousand users and groups and tested the compat and NIS
>>> plugins and the data appears correct.
>>>
>>> Updated patch with minimum n-v-r in spec attached.
>>>
>>> rob
>>>
>>
>> Good job, this closes a lot of tickets! I am now also able to run tests 
>> without
>> having to set wait_for_attr env config first!
>>
>> The patch generally seems to work OK, I tried it even with a replica without
>> transactions enabled and so far so good. I have found just few issues:
>>
>> 1) Patch needs a mild rebase (spec file conflict)
> 
> Done.
> 
>>
>> 2) One Unit test failure slipped:
>>
>> ==
>> FAIL: test_permission[22]: permission_find: Search for permissions by attr 
>> with
>> a limit of 1 (truncated)
> 
> I hadn't noticed this one because the memberof for the role wasn't being
> updated. I added a task that runs in the updates and that fixed it.
> 
>> 3) Question - what is the reason of keeping wait_for_attr sections in our 
>> code?
>> I may miss something, but I see no difference with api.env.wait_for_attr
>> enabled... AFAIU, there should not even be any difference as all attributes
>> should be filled in one transaction, so I would rather remove this flag from
>> both code and man page.
> 
> Was just hedging my bets. You're right though, it makes no sense when we have
> transactions. I've removed it.
> 
>> 4) nsslapd-pluginbetxn is not set for schema compatibility plugin after 
>> upgrade:
>>
>> # Schema Compatibility, plugins, config
>> dn: cn=Schema Compatibility,cn=plugins,cn=config
>> nsslapd-pluginId: schema-compat-plugin
>> cn: Schema Compatibility
>> objectClass: top
>> objectClass: nsSlapdPlugin
>> objectClass: extensibleObject
>> nsslapd-pluginDescription: Schema Compatibility Plugin
>> nsslapd-pluginEnabled: on
>> nsslapd-pluginPath: /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
>> nsslapd-pluginVersion: 0.44 (betxn support available and enabled by default)
>> nsslapd-pluginVendor: redhat.com
>> nsslapd-pluginType: object
>> nsslapd-pluginInitfunc: schema_compat_plugin_init
>>
>> This is supposed to be enabled by default, judging by nsslapd-pluginVersion
>> description, but this may create an inconsistency between new installs and
>> upgraded IPA servers.
>>
>> The same issue applies to IPA server with NIS plugin enabled.
> 
> Yeah, I fixed the ldif but had not added an update for these.
> 
> I had to declare a new option for the updater, onlyifexist, which will force a
> value in an attribute only if the entry already exists. I need this to be sure
> that the only value for betxn is on.
> 
> rob
> 

Thanks. I think the patch works fine now.

ACK, pushed to master.

Martin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-20 Thread Rob Crittenden

Martin Kosek wrote:

On 11/16/2012 05:18 PM, Rob Crittenden wrote:

Nalin Dahyabhai wrote:

On Thu, Nov 15, 2012 at 11:53:44PM -0500, Rob Crittenden wrote:

In order for this to work you'll need to apply the last two patches
(both 0001) to slapi-nis and spin it up yourself, otherwise you'll
have serious deadlock issues. I know this is extra work but this
patch is potentially disruptive so I figure the earlier it is out
the better.

Noriko/Rich/Nalin, can you guys review the slapi-nis pieces? I may
have been too aggressive in my cleanup.

Noriko/Rich, can you review the 389-ds plugin parts of my 1072 patch?

Once we have an official slapi-nis build with these patches we'll
need to set the minimum n-v-r in our spec file.


Rob, the original patch was already applied.  I since reworked large
parts of how it was organized to make it easier for me to read, and
tagged the result as 0.43.  Have you tested the IPA changes in
combination with the 0.44 builds from either ipa-devel or Fedora 18's
updates-testing repository?

Nalin



I tested the 0.44 build and things are looking good. I'm not deadlocking, so I
guess that's the desired out come :-)

I bulk loaded a few thousand users and groups and tested the compat and NIS
plugins and the data appears correct.

Updated patch with minimum n-v-r in spec attached.

rob



Good job, this closes a lot of tickets! I am now also able to run tests without
having to set wait_for_attr env config first!

The patch generally seems to work OK, I tried it even with a replica without
transactions enabled and so far so good. I have found just few issues:

1) Patch needs a mild rebase (spec file conflict)


Done.



2) One Unit test failure slipped:

==
FAIL: test_permission[22]: permission_find: Search for permissions by attr with
a limit of 1 (truncated)


I hadn't noticed this one because the memberof for the role wasn't being 
updated. I added a task that runs in the updates and that fixed it.



3) Question - what is the reason of keeping wait_for_attr sections in our code?
I may miss something, but I see no difference with api.env.wait_for_attr
enabled... AFAIU, there should not even be any difference as all attributes
should be filled in one transaction, so I would rather remove this flag from
both code and man page.


Was just hedging my bets. You're right though, it makes no sense when we 
have transactions. I've removed it.



4) nsslapd-pluginbetxn is not set for schema compatibility plugin after upgrade:

# Schema Compatibility, plugins, config
dn: cn=Schema Compatibility,cn=plugins,cn=config
nsslapd-pluginId: schema-compat-plugin
cn: Schema Compatibility
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject
nsslapd-pluginDescription: Schema Compatibility Plugin
nsslapd-pluginEnabled: on
nsslapd-pluginPath: /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
nsslapd-pluginVersion: 0.44 (betxn support available and enabled by default)
nsslapd-pluginVendor: redhat.com
nsslapd-pluginType: object
nsslapd-pluginInitfunc: schema_compat_plugin_init

This is supposed to be enabled by default, judging by nsslapd-pluginVersion
description, but this may create an inconsistency between new installs and
upgraded IPA servers.

The same issue applies to IPA server with NIS plugin enabled.


Yeah, I fixed the ldif but had not added an update for these.

I had to declare a new option for the updater, onlyifexist, which will 
force a value in an attribute only if the entry already exists. I need 
this to be sure that the only value for betxn is on.


rob

>From f2b27db177a8301d4a035bce7f0db7af36bbe669 Mon Sep 17 00:00:00 2001
From: Rob Crittenden 
Date: Thu, 15 Nov 2012 21:38:26 -0500
Subject: [PATCH] Enable transactions by default, make password and modrdn
 TXN-aware

The password and modrdn plugins needed to be made transaction aware
for the pre and post operations.

Remove the reverse member hoop jumping. Just fetch the entry once
and all the memberof data is there (plus objectclass).

Fix some unit tests that are failing because we actually get the data
now due to transactions.

Add small bit of code in user plugin to retrieve the user again
ala wait_for_attr but in the case of transactions we need do it only
once.

Deprecate wait_for_attr code.

Add a memberof fixup task for roles.

https://fedorahosted.org/freeipa/ticket/1263
https://fedorahosted.org/freeipa/ticket/1891
https://fedorahosted.org/freeipa/ticket/2056
https://fedorahosted.org/freeipa/ticket/3043
https://fedorahosted.org/freeipa/ticket/3191
https://fedorahosted.org/freeipa/ticket/3046
---
 daemons/ipa-slapi-plugins/ipa-modrdn/ipa_modrdn.c  | 26 --
 .../ipa-slapi-plugins/ipa-modrdn/modrdn-conf.ldif  |  2 +-
 .../ipa-pwd-extop/ipa_pwd_extop.c  | 21 
 daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd.h   |  2 +
 .../ipa-pwd-extop/ipapwd_prepost.c | 24 +
 .../ipa-pwd-extop/pwd-extop-conf.

Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-20 Thread Rob Crittenden

Martin Kosek wrote:

On 11/20/2012 04:27 PM, Nalin Dahyabhai wrote:

On Tue, Nov 20, 2012 at 02:08:04PM +0100, Martin Kosek wrote:

4) nsslapd-pluginbetxn is not set for schema compatibility plugin after upgrade:

# Schema Compatibility, plugins, config
dn: cn=Schema Compatibility,cn=plugins,cn=config
nsslapd-pluginId: schema-compat-plugin
cn: Schema Compatibility
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject
nsslapd-pluginDescription: Schema Compatibility Plugin
nsslapd-pluginEnabled: on
nsslapd-pluginPath: /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
nsslapd-pluginVersion: 0.44 (betxn support available and enabled by default)
nsslapd-pluginVendor: redhat.com
nsslapd-pluginType: object
nsslapd-pluginInitfunc: schema_compat_plugin_init

This is supposed to be enabled by default, judging by nsslapd-pluginVersion
description, but this may create an inconsistency between new installs and
upgraded IPA servers.

The same issue applies to IPA server with NIS plugin enabled.


Which version of IPA is it that starts explicitly configuring
"nsslapd-pluginbetxn" values for plugins?


We explicitly started to set it to off in
ea4f60b15a2743eb61f27ccd33d7bed17552eade, i.e. FreeIPA 3.0.x in F18. The plan
is to set it to on with Rob's patch 1072 in FreeIPA 3.1.



For Fedora, at least, are there cases where we're going from a version
that didn't configure that setting to a version that does configure it,
as an update within a single release?  If not, I can make the default
change depending on which release we're building for, and we'll be fine.
If that sort of upgrade is expected, though, the package will probably
need to start conflicting with versions of IPA that don't configure
"nsslapd-pluginbetxn" one way or the other, because there's no default
value that's guaranteed to be safe.

Nalin



Such update may make it more bulletproof. Bug I think that we should be OK as
long as the default betxn support is enabled in Fedora 18 or later (which it
is) because there should be no FreeIPA 2.x release (nsslapd-pluginbetxn not
set) there.

Martin



I had intended to always enable it with this patch (but I screwed it up 
somehow). I'm fine having this as an IPA responsibility. If you want to 
make it the default in slapi-nis too that would be fine, and is probably 
a good idea since transactions are enabled by default in 1.3, but I want 
to be explicit in IPA.


rob

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-20 Thread Martin Kosek
On 11/20/2012 04:27 PM, Nalin Dahyabhai wrote:
> On Tue, Nov 20, 2012 at 02:08:04PM +0100, Martin Kosek wrote:
>> 4) nsslapd-pluginbetxn is not set for schema compatibility plugin after 
>> upgrade:
>>
>> # Schema Compatibility, plugins, config
>> dn: cn=Schema Compatibility,cn=plugins,cn=config
>> nsslapd-pluginId: schema-compat-plugin
>> cn: Schema Compatibility
>> objectClass: top
>> objectClass: nsSlapdPlugin
>> objectClass: extensibleObject
>> nsslapd-pluginDescription: Schema Compatibility Plugin
>> nsslapd-pluginEnabled: on
>> nsslapd-pluginPath: /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
>> nsslapd-pluginVersion: 0.44 (betxn support available and enabled by default)
>> nsslapd-pluginVendor: redhat.com
>> nsslapd-pluginType: object
>> nsslapd-pluginInitfunc: schema_compat_plugin_init
>>
>> This is supposed to be enabled by default, judging by nsslapd-pluginVersion
>> description, but this may create an inconsistency between new installs and
>> upgraded IPA servers.
>>
>> The same issue applies to IPA server with NIS plugin enabled.
> 
> Which version of IPA is it that starts explicitly configuring
> "nsslapd-pluginbetxn" values for plugins?

We explicitly started to set it to off in
ea4f60b15a2743eb61f27ccd33d7bed17552eade, i.e. FreeIPA 3.0.x in F18. The plan
is to set it to on with Rob's patch 1072 in FreeIPA 3.1.

> 
> For Fedora, at least, are there cases where we're going from a version
> that didn't configure that setting to a version that does configure it,
> as an update within a single release?  If not, I can make the default
> change depending on which release we're building for, and we'll be fine.
> If that sort of upgrade is expected, though, the package will probably
> need to start conflicting with versions of IPA that don't configure
> "nsslapd-pluginbetxn" one way or the other, because there's no default
> value that's guaranteed to be safe.
> 
> Nalin
> 

Such update may make it more bulletproof. Bug I think that we should be OK as
long as the default betxn support is enabled in Fedora 18 or later (which it
is) because there should be no FreeIPA 2.x release (nsslapd-pluginbetxn not
set) there.

Martin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-20 Thread Nalin Dahyabhai
On Tue, Nov 20, 2012 at 02:08:04PM +0100, Martin Kosek wrote:
> 4) nsslapd-pluginbetxn is not set for schema compatibility plugin after 
> upgrade:
> 
> # Schema Compatibility, plugins, config
> dn: cn=Schema Compatibility,cn=plugins,cn=config
> nsslapd-pluginId: schema-compat-plugin
> cn: Schema Compatibility
> objectClass: top
> objectClass: nsSlapdPlugin
> objectClass: extensibleObject
> nsslapd-pluginDescription: Schema Compatibility Plugin
> nsslapd-pluginEnabled: on
> nsslapd-pluginPath: /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
> nsslapd-pluginVersion: 0.44 (betxn support available and enabled by default)
> nsslapd-pluginVendor: redhat.com
> nsslapd-pluginType: object
> nsslapd-pluginInitfunc: schema_compat_plugin_init
> 
> This is supposed to be enabled by default, judging by nsslapd-pluginVersion
> description, but this may create an inconsistency between new installs and
> upgraded IPA servers.
> 
> The same issue applies to IPA server with NIS plugin enabled.

Which version of IPA is it that starts explicitly configuring
"nsslapd-pluginbetxn" values for plugins?

For Fedora, at least, are there cases where we're going from a version
that didn't configure that setting to a version that does configure it,
as an update within a single release?  If not, I can make the default
change depending on which release we're building for, and we'll be fine.
If that sort of upgrade is expected, though, the package will probably
need to start conflicting with versions of IPA that don't configure
"nsslapd-pluginbetxn" one way or the other, because there's no default
value that's guaranteed to be safe.

Nalin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-20 Thread Martin Kosek
On 11/16/2012 05:18 PM, Rob Crittenden wrote:
> Nalin Dahyabhai wrote:
>> On Thu, Nov 15, 2012 at 11:53:44PM -0500, Rob Crittenden wrote:
>>> In order for this to work you'll need to apply the last two patches
>>> (both 0001) to slapi-nis and spin it up yourself, otherwise you'll
>>> have serious deadlock issues. I know this is extra work but this
>>> patch is potentially disruptive so I figure the earlier it is out
>>> the better.
>>>
>>> Noriko/Rich/Nalin, can you guys review the slapi-nis pieces? I may
>>> have been too aggressive in my cleanup.
>>>
>>> Noriko/Rich, can you review the 389-ds plugin parts of my 1072 patch?
>>>
>>> Once we have an official slapi-nis build with these patches we'll
>>> need to set the minimum n-v-r in our spec file.
>>
>> Rob, the original patch was already applied.  I since reworked large
>> parts of how it was organized to make it easier for me to read, and
>> tagged the result as 0.43.  Have you tested the IPA changes in
>> combination with the 0.44 builds from either ipa-devel or Fedora 18's
>> updates-testing repository?
>>
>> Nalin
>>
> 
> I tested the 0.44 build and things are looking good. I'm not deadlocking, so I
> guess that's the desired out come :-)
> 
> I bulk loaded a few thousand users and groups and tested the compat and NIS
> plugins and the data appears correct.
> 
> Updated patch with minimum n-v-r in spec attached.
> 
> rob
> 

Good job, this closes a lot of tickets! I am now also able to run tests without
having to set wait_for_attr env config first!

The patch generally seems to work OK, I tried it even with a replica without
transactions enabled and so far so good. I have found just few issues:

1) Patch needs a mild rebase (spec file conflict)

2) One Unit test failure slipped:

==
FAIL: test_permission[22]: permission_find: Search for permissions by attr with
a limit of 1 (truncated)
--
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
  File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 249, in

func = lambda: self.check(nice, **test)
  File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 266, in 
check
self.check_output(nice, cmd, args, options, expected, extra_check)
  File "/root/freeipa-master/tests/test_xmlrpc/xmlrpc_test.py", line 304, in
check_output
assert_deepequal(expected, got, nice)
  File "/root/freeipa-master/tests/util.py", line 335, in assert_deepequal
assert_deepequal(e_sub, g_sub, doc, stack + (key,))
  File "/root/freeipa-master/tests/util.py", line 323, in assert_deepequal
assert_deepequal(e_sub, g_sub, doc, stack + (i,))
  File "/root/freeipa-master/tests/util.py", line 329, in assert_deepequal
doc, sorted(missing), sorted(extra), expected, got, stack
AssertionError: assert_deepequal: dict keys mismatch.
  test_permission[22]: permission_find: Search for permissions by attr with a
limit of 1 (truncated)
  missing keys = []
  extra keys = ['memberindirect']
  expected = {'dn': ipapython.dn.DN('cn=Modify HBAC
rule,cn=permissions,cn=pbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com'), 'cn':
[u'Modify HBAC rule'], 'member_privilege': [u'HBAC Administrator'], 'subtree':
u'ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com',
'attrs': [u'servicecategory', u'sourcehostcategory', u'cn', u'description',
u'ipaenabledflag', u'accesstime', u'usercategory', u'hostcategory',
u'accessruletype', u'sourcehost'], 'permissions': [u'write']}
  got = {'dn': u'cn=Modify HBAC
rule,cn=permissions,cn=pbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com', 'cn':
(u'Modify HBAC rule',), 'member_privilege': (u'HBAC Administrator',),
'subtree':
u'ldap:///ipauniqueid=*,cn=hbac,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com',
'memberindirect': (u'cn=IT Security
Specialist,cn=roles,cn=accounts,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com',),
'attrs': (u'servicecategory', u'sourcehostcategory', u'cn', u'description',
u'ipaenabledflag', u'accesstime', u'usercategory', u'hostcategory',
u'accessruletype', u'sourcehost'), 'permissions': (u'write',)}
  path = ('result', 0)


3) Question - what is the reason of keeping wait_for_attr sections in our code?
I may miss something, but I see no difference with api.env.wait_for_attr
enabled... AFAIU, there should not even be any difference as all attributes
should be filled in one transaction, so I would rather remove this flag from
both code and man page.

4) nsslapd-pluginbetxn is not set for schema compatibility plugin after upgrade:

# Schema Compatibility, plugins, config
dn: cn=Schema Compatibility,cn=plugins,cn=config
nsslapd-pluginId: schema-compat-plugin
cn: Schema Compatibility
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject
nsslapd-pluginDescription: Schema Compatibility Plugin
nsslapd-pluginEnabled: on
nsslapd-pluginPath: /usr/lib

Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-16 Thread Noriko Hosoi

(2012/11/16 16:09), Rich Megginson wrote:

On 11/15/2012 09:53 PM, Rob Crittenden wrote:
This patch enables transaction support in 389-ds-base and fixes a few 
transaction issues within IPA.


This converts parts of the password and modrnd plugins to support 
transactions. The password plugin still largely runs as 
non-transactional because extop plugins aren't supported in 
transactions yet.


I've left the wait_for_attr code in place for now but on reflection 
we should probably remove it. I'll leave that up to the reviewer, but 
I can't see the need for it any more.


In order for this to work you'll need to apply the last two patches 
(both 0001) to slapi-nis and spin it up yourself, otherwise you'll 
have serious deadlock issues. I know this is extra work but this 
patch is potentially disruptive so I figure the earlier it is out the 
better.


Noriko/Rich/Nalin, can you guys review the slapi-nis pieces? I may 
have been too aggressive in my cleanup.


Noriko/Rich, can you review the 389-ds plugin parts of my 1072 patch?

ack

Your patch looks good to me, too.
Thank you so much!
--noriko


Once we have an official slapi-nis build with these patches we'll 
need to set the minimum n-v-r in our spec file.


rob




___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-16 Thread Rich Megginson

On 11/15/2012 09:53 PM, Rob Crittenden wrote:
This patch enables transaction support in 389-ds-base and fixes a few 
transaction issues within IPA.


This converts parts of the password and modrnd plugins to support 
transactions. The password plugin still largely runs as 
non-transactional because extop plugins aren't supported in 
transactions yet.


I've left the wait_for_attr code in place for now but on reflection we 
should probably remove it. I'll leave that up to the reviewer, but I 
can't see the need for it any more.


In order for this to work you'll need to apply the last two patches 
(both 0001) to slapi-nis and spin it up yourself, otherwise you'll 
have serious deadlock issues. I know this is extra work but this patch 
is potentially disruptive so I figure the earlier it is out the better.


Noriko/Rich/Nalin, can you guys review the slapi-nis pieces? I may 
have been too aggressive in my cleanup.


Noriko/Rich, can you review the 389-ds plugin parts of my 1072 patch?

ack


Once we have an official slapi-nis build with these patches we'll need 
to set the minimum n-v-r in our spec file.


rob


___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-16 Thread Rob Crittenden

Nalin Dahyabhai wrote:

On Thu, Nov 15, 2012 at 11:53:44PM -0500, Rob Crittenden wrote:

In order for this to work you'll need to apply the last two patches
(both 0001) to slapi-nis and spin it up yourself, otherwise you'll
have serious deadlock issues. I know this is extra work but this
patch is potentially disruptive so I figure the earlier it is out
the better.

Noriko/Rich/Nalin, can you guys review the slapi-nis pieces? I may
have been too aggressive in my cleanup.

Noriko/Rich, can you review the 389-ds plugin parts of my 1072 patch?

Once we have an official slapi-nis build with these patches we'll
need to set the minimum n-v-r in our spec file.


Rob, the original patch was already applied.  I since reworked large
parts of how it was organized to make it easier for me to read, and
tagged the result as 0.43.  Have you tested the IPA changes in
combination with the 0.44 builds from either ipa-devel or Fedora 18's
updates-testing repository?

Nalin



I tested the 0.44 build and things are looking good. I'm not 
deadlocking, so I guess that's the desired out come :-)


I bulk loaded a few thousand users and groups and tested the compat and 
NIS plugins and the data appears correct.


Updated patch with minimum n-v-r in spec attached.

rob
>From eaba4afdf902432a8b5e84ff841cf3b74a85d08b Mon Sep 17 00:00:00 2001
From: Rob Crittenden 
Date: Thu, 15 Nov 2012 21:38:26 -0500
Subject: [PATCH] Enable transactions by default, make password and modrdn
 TXN-aware

The password and modrdn plugins needed to be made transaction aware
for the pre and post operations.

Remove the reverse member hoop jumping. Just fetch the entry once
and all the memberof data is there (plus objectclass).

Fix some unit tests that are failing because we actually get the data
now due to transactions.

Add small bit of code in user plugin to retrieve the user again
ala wait_for_attr but in the case of transactions we need do it only
once.

https://fedorahosted.org/freeipa/ticket/1263
https://fedorahosted.org/freeipa/ticket/1891
https://fedorahosted.org/freeipa/ticket/2056
https://fedorahosted.org/freeipa/ticket/3043
https://fedorahosted.org/freeipa/ticket/3191
https://fedorahosted.org/freeipa/ticket/3046
---
 daemons/ipa-slapi-plugins/ipa-modrdn/ipa_modrdn.c  | 26 --
 .../ipa-slapi-plugins/ipa-modrdn/modrdn-conf.ldif  |  2 +-
 .../ipa-pwd-extop/ipa_pwd_extop.c  | 21 
 daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd.h   |  2 +
 .../ipa-pwd-extop/ipapwd_prepost.c | 24 +
 .../ipa-pwd-extop/pwd-extop-conf.ldif  |  1 +
 freeipa.spec.in|  5 +-
 install/share/nis.uldif|  1 +
 install/share/schema_compat.uldif  |  1 +
 install/updates/10-disable-betxn.update| 37 -
 install/updates/10-enable-betxn.update | 43 
 install/updates/Makefile.am|  2 +-
 ipalib/plugins/baseldap.py | 60 +++---
 ipalib/plugins/user.py |  5 ++
 ipaserver/install/dsinstance.py|  4 --
 tests/test_xmlrpc/test_automember_plugin.py| 10 
 tests/test_xmlrpc/test_nesting.py  |  1 +
 tests/test_xmlrpc/test_permission_plugin.py|  2 +-
 tests/test_xmlrpc/test_privilege_plugin.py |  7 +++
 tests/test_xmlrpc/test_role_plugin.py  |  5 ++
 20 files changed, 156 insertions(+), 103 deletions(-)
 delete mode 100644 install/updates/10-disable-betxn.update
 create mode 100644 install/updates/10-enable-betxn.update

diff --git a/daemons/ipa-slapi-plugins/ipa-modrdn/ipa_modrdn.c b/daemons/ipa-slapi-plugins/ipa-modrdn/ipa_modrdn.c
index 70a4ea82144829015e663c37212af7196a9ad72a..6cec5f242b7d23d3752e5bc30c67e034abc96abb 100644
--- a/daemons/ipa-slapi-plugins/ipa-modrdn/ipa_modrdn.c
+++ b/daemons/ipa-slapi-plugins/ipa-modrdn/ipa_modrdn.c
@@ -201,6 +201,12 @@ ipamodrdn_init(Slapi_PBlock *pb)
 {
 int status = EOK;
 char *plugin_identity = NULL;
+Slapi_Entry *plugin_entry = NULL;
+char *plugin_type = NULL;
+int delfn = SLAPI_PLUGIN_POST_DELETE_FN;
+int mdnfn = SLAPI_PLUGIN_POST_MODRDN_FN;
+int modfn = SLAPI_PLUGIN_POST_MODIFY_FN;
+int addfn = SLAPI_PLUGIN_POST_ADD_FN;
 
 LOG_TRACE("--in-->\n");
 
@@ -213,6 +219,18 @@ ipamodrdn_init(Slapi_PBlock *pb)
 PR_ASSERT(plugin_identity);
 setPluginID(plugin_identity);
 
+if ((slapi_pblock_get(pb, SLAPI_PLUGIN_CONFIG_ENTRY, &plugin_entry) == 0) &&
+plugin_entry &&
+(plugin_type = slapi_entry_attr_get_charptr(plugin_entry, "nsslapd-plugintype")) &&
+plugin_type && strstr(plugin_type, "betxn"))
+{
+addfn = SLAPI_PLUGIN_BE_TXN_POST_ADD_FN;
+mdnfn = SLAPI_PLUGIN_BE_TXN_POST_MODRDN_FN;
+delfn = SLAPI_PLUGIN_BE_TXN_POST_DELETE_FN;
+modfn = SLAPI_PLUGIN_BE_TXN_POST_MODIFY_FN;
+}
+slapi_ch_free_s

Re: [Freeipa-devel] [PATCH] 1072 enable transaction support

2012-11-16 Thread Nalin Dahyabhai
On Thu, Nov 15, 2012 at 11:53:44PM -0500, Rob Crittenden wrote:
> In order for this to work you'll need to apply the last two patches
> (both 0001) to slapi-nis and spin it up yourself, otherwise you'll
> have serious deadlock issues. I know this is extra work but this
> patch is potentially disruptive so I figure the earlier it is out
> the better.
> 
> Noriko/Rich/Nalin, can you guys review the slapi-nis pieces? I may
> have been too aggressive in my cleanup.
> 
> Noriko/Rich, can you review the 389-ds plugin parts of my 1072 patch?
> 
> Once we have an official slapi-nis build with these patches we'll
> need to set the minimum n-v-r in our spec file.

Rob, the original patch was already applied.  I since reworked large
parts of how it was organized to make it easier for me to read, and
tagged the result as 0.43.  Have you tested the IPA changes in
combination with the 0.44 builds from either ipa-devel or Fedora 18's
updates-testing repository?

Nalin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel