Re: [Freeipa-devel] [WIP] Thin client

[Jan Cholasta]

On 26.5.2016 18:43, Martin Basti wrote:



On 26.05.2016 11:21, Martin Basti wrote:



On 26.05.2016 07:15, Jan Cholasta wrote:

On 25.5.2016 18:17, Martin Basti wrote:



On 25.05.2016 16:07, Jan Cholasta wrote:

On 25.5.2016 15:03, David Kupka wrote:

On 18/05/16 08:01, Jan Cholasta wrote:

On 16.5.2016 16:35, Martin Basti wrote:



On 09.05.2016 14:07, Jan Cholasta wrote:

On 6.5.2016 14:32, Martin Basti wrote:



On 28.04.2016 14:45, Jan Cholasta wrote:

Hi,

I have pushed my thin client WIP branch to GitHub:
.

All commits up to "ipalib: use relative imports for cross-plugin
imports" should be good for review. The rest is subject to
change
(WARNING: I will force push into this branch).

Honza


I did not test it yet, I just checked the code

* automount: do not inherit automountlocation_tofiles from
LDAPQuery *
LGTM

* dns: move all dnsrecord code called on client to a single
class *
LGTM

* dns: do not rely on server data structures in code called on
client *
1)
you forgot to increment VERSION


This was deliberate, as it will no longer be necessary to bump
VERSION
for backward compatible changes after this whole patchset is
merged.
But we're not there yet, so fixed.


How we should handle VERSION after your patches?


Otherwise LGTM

* otptoken: fix import of DN *
ACK

* otptoken_yubikey: fix otptoken_add_yubikey arguments *
1)
you forgot to increment VERSION


Fixed.



2)
Did you find out why this was issue?
-del answer['value']# Why does this cause an error if
omitted?
 -del answer['summary']  # Why does this cause an
error if
omitted?


The command definition was not complete, it was missing
has_output.



Otherwise LGTM

* vault: move client-side code to the module level *
LGTM

* vault: copy arguments of client commands from server
counterparts *
1)
you forgot to increment Version


Fixed.



* ipalib: use relative imports for cross-plugin imports *
1) Missing explanation for future generations why this change is
needed
in commit message


Fixed.



The other commits I will check later.
Martin^2



Okay. Thanks.



* frontend: remove the unused Command.soft_validate method
LGTM

* frontend: perform argument value validation only on server
LGTM

* frontend: do not forward argument defaults to server
I'm not a fan of returning  None in promt_param function, but I
havent
found anything better to use.


It always returned None for unset params.


LGTM

* ipalib: optimize API.txt
this contains a lot of black magic, but because this is mainly
copy of
current to proper places, LGTM


It's actually mostly cut-n-paste.



* makeaci: load additional plugins using API.add_module
Looks good, but I miss explanation why is this change needed


The next change would be impossible without this.



* plugable: replace API.import_plugins with new API.add_package
LGTM


* ipalib, ipaserver: migrate all plugins to Registry-based
registration
ACK

* ipalib, ipaserver: fix incorrect API.register calls in docstrings
LGTM


* plugable: remove the unused deprecated API.register method
LGTM


* plugable: switch API to Registry-based plugin discovery
LGTM

* frontend: merge baseldap.CallbackRegistry into Command
LGTM

*frontend: move the interactive_prompt callback type to Command
 LGTM

Martin^2







Hello,
first batch of 30 patches from Honza's trac-4739 branch
(https://github.com/jcholast/freeipa/commits/trac-4739) is ready
to be
pushed into master.
All upto "frontend: allow commands to have an argument named
`name`" got
over numerous test cycles in last week+ and is working as
expected
now, ACK.


Thanks for the review.



Honzo, please rebase and push them, thanks!



Attaching the patches for reference.

Pushed to master: 2b50fc617024f18a81e97f30f75ed1b9221c4055



Guys, you forgot to test it with newer pylint.


I don't see any "BuildRequires: newer pylint" in the spec file.



Patch attached.


LGTM, although the commit message is wrong - this is not related to
thin client at all, PublicError and PublicMessage had .kw since forever.


updated commit message






Can I push that fix for pylint?


Sure, ACK.



I found something suspicious in tests, did anything in IPA messages
change? I suspect that this is related to client_patches.


Yes, 'data' is a dict which contains structured message data. I did not 
see these specific failures before, though. Just add it to expected 
results where missing.




E   AssertionError: assert_deepequal: dict keys mismatch.
E 0026: permission_find: Search for u'testperm' with a
limit of 1 (truncated) with members
E missing keys = []
E extra keys = [u'data']
E expected = {'message': u'Search result has been
truncated: Configured size limit exceeded', 'code': 13017, 'type':
u'warning', 'name': u'SearchResultTruncated'}
E got = {u'data': {u'reason': u'Configured size limit
exceeded'}, u'message': u'Search result has 

Re: [Freeipa-devel] [PATCH 0033] Fix CA being presented as running even if it weren't

[Stanislav Laznicka]

Self NACK. I should not post patches when tired, sorry. Minor fix is 
attached.



On 05/26/2016 07:21 PM, Stanislav Laznicka wrote:

Hello,

Please, see the attached patch. Fixes 
https://fedorahosted.org/freeipa/ticket/5898


Standa





From b42146384771d95761cbeaab516f559ee87b66cc Mon Sep 17 00:00:00 2001
From: Stanislav Laznicka 
Date: Thu, 26 May 2016 15:24:15 +0200
Subject: [PATCH] Fixes CA always being presented as running

Even after manually stopping the pki-tomcatd service instance the
service the is_running() method would still return True.

https://fedorahosted.org/freeipa/ticket/5898
---
 ipaplatform/redhat/services.py | 18 ++
 ipaserver/install/service.py   |  4 ++--
 2 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/ipaplatform/redhat/services.py b/ipaplatform/redhat/services.py
index 92dae452a31a0b3680e9c407eccb120881cc9e25..6f8ac2cb41c8b716d8fa13869565d9dc047d2cc4 100644
--- a/ipaplatform/redhat/services.py
+++ b/ipaplatform/redhat/services.py
@@ -222,6 +222,24 @@ class RedHatCAService(RedHatService):
 if wait:
 self.wait_until_running()
 
+def is_running(self, instance_name=""):
+if instance_name:
+return super(RedHatCAService, self).is_running(instance_name)
+while True:
+try:
+status = dogtag.ca_status()
+except Exception as e:
+root_logger.debug(
+'Failed to check CA status: {err}'.format(err=e)
+)
+return False
+if status == 'running':
+return True
+elif status == 'starting':
+time.sleep(1)
+else:
+return False
+
 
 # Function that constructs proper Red Hat OS family-specific server classes for
 # services of specified name
diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py
index 40767acd57d5e1fa8126144ca64f6951848ce214..e1361c2fb339c73dd36028257ed862d02d9411e3 100644
--- a/ipaserver/install/service.py
+++ b/ipaserver/install/service.py
@@ -346,8 +346,8 @@ class Service(object):
 def restart(self, instance_name="", capture_output=True, wait=True):
 self.service.restart(instance_name, capture_output=capture_output, wait=wait)
 
-def is_running(self):
-return self.service.is_running()
+def is_running(self, instance_name=""):
+return self.service.is_running(instance_name)
 
 def install(self):
 self.service.install()
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH 0033] Fix CA being presented as running even if it weren't

[Stanislav Laznicka]

Hello,

Please, see the attached patch. Fixes 
https://fedorahosted.org/freeipa/ticket/5898


Standa

From ba7ecd6eed5cb2d70dbe684539d927c44e4c11b6 Mon Sep 17 00:00:00 2001
From: Stanislav Laznicka 
Date: Thu, 26 May 2016 15:24:15 +0200
Subject: [PATCH] Fixes CA always being presented as running

Even after manually stopping the pki-tomcatd service instance the
service the is_running() method would still return True.

https://fedorahosted.org/freeipa/ticket/5898
---
 ipaplatform/redhat/services.py | 18 ++
 ipaserver/install/service.py   |  4 ++--
 2 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/ipaplatform/redhat/services.py b/ipaplatform/redhat/services.py
index 92dae452a31a0b3680e9c407eccb120881cc9e25..7f54f76fa04bbcfafd57ef7cda48cce5c197cd4b 100644
--- a/ipaplatform/redhat/services.py
+++ b/ipaplatform/redhat/services.py
@@ -222,6 +222,24 @@ class RedHatCAService(RedHatService):
 if wait:
 self.wait_until_running()
 
+def is_running(self, instance_name=""):
+if instance_name:
+super(RedHatCAService, self).is_running(instance_name)
+while True:
+try:
+status = dogtag.ca_status()
+except Exception as e:
+root_logger.debug(
+'Failed to check CA status: {err}'.format(err=e)
+)
+return False
+if status == 'running':
+return True
+elif status == 'starting':
+time.sleep(1)
+else:
+return False
+
 
 # Function that constructs proper Red Hat OS family-specific server classes for
 # services of specified name
diff --git a/ipaserver/install/service.py b/ipaserver/install/service.py
index 40767acd57d5e1fa8126144ca64f6951848ce214..e1361c2fb339c73dd36028257ed862d02d9411e3 100644
--- a/ipaserver/install/service.py
+++ b/ipaserver/install/service.py
@@ -346,8 +346,8 @@ class Service(object):
 def restart(self, instance_name="", capture_output=True, wait=True):
 self.service.restart(instance_name, capture_output=capture_output, wait=wait)
 
-def is_running(self):
-return self.service.is_running()
+def is_running(self, instance_name=""):
+return self.service.is_running(instance_name)
 
 def install(self):
 self.service.install()
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCHES 0089-0093] Authentication Indicators

[Martin Basti]

On 26.05.2016 17:36, Nathaniel McCallum wrote:

Martin, can we get patches 1-4 pushed? I'll submit patch 5 again to the
list after a rebase for further discussion.

Here it is, pushed to master:
* cd9bc84240c99ed744e5ee44db18d925a5292ffd Rename syncreq.[ch] to 
otpctrl.[ch]
* 168a6c7d4778a2a3c729e3ac24e4ad9dfacb46c0 Ensure that ipa-otpd bind 
auths validate an OTP
* 204200d73bb135cb7b9b31b8f1ba5268d73094a5 Return password-only preauth 
if passwords are allowed
* 8f356a4305a9aa74aacae36806d6e8ed1b765245 Enable authentication 
indicators for OTP and RADIUS




On Wed, 2016-05-25 at 13:32 +0200, Sumit Bose wrote:

On Tue, May 24, 2016 at 12:21:43PM -0400, Nathaniel McCallum wrote:

New versions again. This time I just removed the stray "TODO:
assign
OID" line in the commit as it no longer applies.

ACK to patches 1-4.

Patch 5 can be committed independently and needs some additional
discussion, see below.

bye,
Sumit


On Tue, 2016-05-24 at 12:08 -0400, Nathaniel McCallum wrote:

I have attached new versions of the patches. Comments below.

On Tue, 2016-05-24 at 15:25 +0200, Sumit Bose wrote:

On Thu, May 12, 2016 at 05:33:26PM -0400, Nathaniel McCallum
wrote:

On Fri, 2016-05-06 at 14:44 +0200, Sumit Bose wrote:

On Wed, May 04, 2016 at 05:33:55PM -0400, Nathaniel
McCallum
wrote:

...


 From c9e2b50248493fb5a283cf8c88c8e20c312d6348 Mon Sep 17
00:00:00
2001
From: Nathaniel McCallum 
Date: Wed, 4 May 2016 17:08:45 -0400
Subject: [PATCH 5/5] Enable service authentication indicator
management


For me the patch looks good, but it would be nice if someone
more
used
to our usage of python can have a short look to see if all
conventioens
are met. ACK for the functionality.

I would like for us to merge the first four patches first and
then
concentrate on this one.

In particular, the following issue needs to be discussed. We
currently
only have two, hard-coded indicator values: otp and radius. Thus,
we
use a StrEnum for this property. However, in the long-term, I'd
like
to
have more flexibility; such as per-token indicators. This implies
String.

Is there some way to do StrEnum now and easily migrate to String
later?
I think this will break API. Thoughts?



--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [WIP] Thin client

[Martin Basti]

On 26.05.2016 11:21, Martin Basti wrote:



On 26.05.2016 07:15, Jan Cholasta wrote:

On 25.5.2016 18:17, Martin Basti wrote:



On 25.05.2016 16:07, Jan Cholasta wrote:

On 25.5.2016 15:03, David Kupka wrote:

On 18/05/16 08:01, Jan Cholasta wrote:

On 16.5.2016 16:35, Martin Basti wrote:



On 09.05.2016 14:07, Jan Cholasta wrote:

On 6.5.2016 14:32, Martin Basti wrote:



On 28.04.2016 14:45, Jan Cholasta wrote:

Hi,

I have pushed my thin client WIP branch to GitHub:
.

All commits up to "ipalib: use relative imports for cross-plugin
imports" should be good for review. The rest is subject to 
change

(WARNING: I will force push into this branch).

Honza


I did not test it yet, I just checked the code

* automount: do not inherit automountlocation_tofiles from
LDAPQuery *
LGTM

* dns: move all dnsrecord code called on client to a single 
class *

LGTM

* dns: do not rely on server data structures in code called on
client *
1)
you forgot to increment VERSION


This was deliberate, as it will no longer be necessary to bump
VERSION
for backward compatible changes after this whole patchset is 
merged.

But we're not there yet, so fixed.


How we should handle VERSION after your patches?


Otherwise LGTM

* otptoken: fix import of DN *
ACK

* otptoken_yubikey: fix otptoken_add_yubikey arguments *
1)
you forgot to increment VERSION


Fixed.



2)
Did you find out why this was issue?
-del answer['value']# Why does this cause an error if
omitted?
 -del answer['summary']  # Why does this cause an 
error if

omitted?


The command definition was not complete, it was missing 
has_output.




Otherwise LGTM

* vault: move client-side code to the module level *
LGTM

* vault: copy arguments of client commands from server
counterparts *
1)
you forgot to increment Version


Fixed.



* ipalib: use relative imports for cross-plugin imports *
1) Missing explanation for future generations why this change is
needed
in commit message


Fixed.



The other commits I will check later.
Martin^2



Okay. Thanks.



* frontend: remove the unused Command.soft_validate method
LGTM

* frontend: perform argument value validation only on server
LGTM

* frontend: do not forward argument defaults to server
I'm not a fan of returning  None in promt_param function, but I 
havent

found anything better to use.


It always returned None for unset params.


LGTM

* ipalib: optimize API.txt
this contains a lot of black magic, but because this is mainly 
copy of

current to proper places, LGTM


It's actually mostly cut-n-paste.



* makeaci: load additional plugins using API.add_module
Looks good, but I miss explanation why is this change needed


The next change would be impossible without this.



* plugable: replace API.import_plugins with new API.add_package
LGTM


* ipalib, ipaserver: migrate all plugins to Registry-based
registration
ACK

* ipalib, ipaserver: fix incorrect API.register calls in docstrings
LGTM


* plugable: remove the unused deprecated API.register method
LGTM


* plugable: switch API to Registry-based plugin discovery
LGTM

* frontend: merge baseldap.CallbackRegistry into Command
LGTM

*frontend: move the interactive_prompt callback type to Command
 LGTM

Martin^2







Hello,
first batch of 30 patches from Honza's trac-4739 branch
(https://github.com/jcholast/freeipa/commits/trac-4739) is ready 
to be

pushed into master.
All upto "frontend: allow commands to have an argument named 
`name`" got
over numerous test cycles in last week+ and is working as 
expected

now, ACK.


Thanks for the review.



Honzo, please rebase and push them, thanks!



Attaching the patches for reference.

Pushed to master: 2b50fc617024f18a81e97f30f75ed1b9221c4055



Guys, you forgot to test it with newer pylint.


I don't see any "BuildRequires: newer pylint" in the spec file.



Patch attached.


LGTM, although the commit message is wrong - this is not related to 
thin client at all, PublicError and PublicMessage had .kw since forever.



updated commit message






Can I push that fix for pylint?

I found something suspicious in tests, did anything in IPA messages 
change? I suspect that this is related to client_patches.


E   AssertionError: assert_deepequal: dict keys mismatch.
E 0026: permission_find: Search for u'testperm' with a 
limit of 1 (truncated) with members

E missing keys = []
E extra keys = [u'data']
E expected = {'message': u'Search result has been 
truncated: Configured size limit exceeded', 'code': 13017, 'type': 
u'warning', 'name': u'SearchResultTruncated'}
E got = {u'data': {u'reason': u'Configured size limit 
exceeded'}, u'message': u'Search result has been truncated: Configured 
size limit exceeded', u'code': 13017, u'type': u'warning', u'name': 
u'SearchResultTruncated'}

E path = ('messages', 0)



E   

Re: [Freeipa-devel] [PATCHES 0089-0093] Authentication Indicators

[Nathaniel McCallum]

Martin, can we get patches 1-4 pushed? I'll submit patch 5 again to the
list after a rebase for further discussion.

On Wed, 2016-05-25 at 13:32 +0200, Sumit Bose wrote:
> On Tue, May 24, 2016 at 12:21:43PM -0400, Nathaniel McCallum wrote:
> > New versions again. This time I just removed the stray "TODO:
> > assign
> > OID" line in the commit as it no longer applies.
> 
> ACK to patches 1-4.
> 
> Patch 5 can be committed independently and needs some additional
> discussion, see below.
> 
> bye,
> Sumit
> 
> > 
> > On Tue, 2016-05-24 at 12:08 -0400, Nathaniel McCallum wrote:
> > > I have attached new versions of the patches. Comments below.
> > > 
> > > On Tue, 2016-05-24 at 15:25 +0200, Sumit Bose wrote:
> > > > On Thu, May 12, 2016 at 05:33:26PM -0400, Nathaniel McCallum
> > > > wrote:
> > > > > On Fri, 2016-05-06 at 14:44 +0200, Sumit Bose wrote:
> > > > > > On Wed, May 04, 2016 at 05:33:55PM -0400, Nathaniel
> > > > > > McCallum
> > > > > > wrote:
> 
> ...
> 
> > > > > From c9e2b50248493fb5a283cf8c88c8e20c312d6348 Mon Sep 17
> > > > > 00:00:00
> > > > > 2001
> > > > > From: Nathaniel McCallum 
> > > > > Date: Wed, 4 May 2016 17:08:45 -0400
> > > > > Subject: [PATCH 5/5] Enable service authentication indicator
> > > > > management
> > > > > 
> > > > 
> > > > For me the patch looks good, but it would be nice if someone
> > > > more
> > > > used
> > > > to our usage of python can have a short look to see if all
> > > > conventioens
> > > > are met. ACK for the functionality.
> > > 
> > > I would like for us to merge the first four patches first and
> > > then
> > > concentrate on this one.
> > > 
> > > In particular, the following issue needs to be discussed. We
> > > currently
> > > only have two, hard-coded indicator values: otp and radius. Thus,
> > > we
> > > use a StrEnum for this property. However, in the long-term, I'd
> > > like
> > > to
> > > have more flexibility; such as per-token indicators. This implies
> > > String.
> > > 
> > > Is there some way to do StrEnum now and easily migrate to String
> > > later?
> > > I think this will break API. Thoughts?
> > > 

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH 0486, 0487] Update zanata config

[Martin Basti]

https://fedorahosted.org/freeipa/ticket/5915

Patches attached

From f45f57256342dac4b4a5bbbef7241d60371195ca Mon Sep 17 00:00:00 2001
From: Martin Basti 
Date: Thu, 26 May 2016 14:56:16 +0200
Subject: [PATCH] Set proper zanata project-version

https://fedorahosted.org/freeipa/ticket/5915
---
 zanata.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/zanata.xml b/zanata.xml
index f954047360a9501df13bcf23bf4687757b987be7..736efff2575d9d2fa848c29ae91f5865f1ddda9e 100644
--- a/zanata.xml
+++ b/zanata.xml
@@ -2,7 +2,7 @@
 http://zanata.org/namespace/config/;>
   https://fedora.zanata.org/
   freeipa
-  master
+  ipa-4-2
   gettext
   .
   .
-- 
2.5.5

From 41ab3228c28fa85117d0f340557f6ef382b6c8bb Mon Sep 17 00:00:00 2001
From: Martin Basti 
Date: Thu, 26 May 2016 15:41:50 +0200
Subject: [PATCH] Set proper zanata project-version

https://fedorahosted.org/freeipa/ticket/5915
---
 zanata.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/zanata.xml b/zanata.xml
index f954047360a9501df13bcf23bf4687757b987be7..c0a2f91e504a434e66a37338193d2e9035b1f6de 100644
--- a/zanata.xml
+++ b/zanata.xml
@@ -2,7 +2,7 @@
 http://zanata.org/namespace/config/;>
   https://fedora.zanata.org/
   freeipa
-  master
+  ipa-4-3
   gettext
   .
   .
-- 
2.5.5

From 59a64881b539bd5354d8489bb584de50844b83e7 Mon Sep 17 00:00:00 2001
From: Martin Basti 
Date: Thu, 26 May 2016 14:41:57 +0200
Subject: [PATCH] Translations: remove deprecated locale configuration

zanata.xml should not contain , because this tag is deprecated
and locales are handled/configured on the server side. Also locale
mappings are done on server side.

https://fedorahosted.org/freeipa/ticket/5915
---
 zanata.xml | 20 
 1 file changed, 20 deletions(-)

diff --git a/zanata.xml b/zanata.xml
index f954047360a9501df13bcf23bf4687757b987be7..d39a5936268313c1c5ecf002110d8e3c678f331b 100644
--- a/zanata.xml
+++ b/zanata.xml
@@ -7,25 +7,5 @@
   .
   .
 
-  
-bn-IN
-ca
-cs
-de
-es
-eu
-fr
-hi
-hu
-id
-ja
-kn
-nl
-pl
-ru
-tg
-uk
-zh-CN
-  
   
 
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Reviving FreeIPA translations

[Martin Basti]

On 26.05.2016 16:21, Martin Basti wrote:



On 26.05.2016 16:15, Yuri Chornoivan wrote:
написане Thu, 26 May 2016 16:51:47 +0300, Martin Basti 
:





On 16.05.2016 08:37, Martin Kosek wrote:

On 05/15/2016 09:34 PM, Yuri Chornoivan wrote:
написане Sun, 15 May 2016 21:51:45 +0300, Martin Kosek 
:



On 05/14/2016 01:29 PM, Yuri Chornoivan wrote:
написане Sat, 14 May 2016 12:57:13 +0300, Jérôme Fenal 
:



2016-05-13 13:32 GMT+02:00 Martin Kosek :


Hello,

As you may or may not know, Tomas Babej left the FreeIPA team 
as a Red Hat
employee, which of course does not mean he cannot contribute 
to the FreeIPA

project, but that he won't have as much time for contributions as
previously.

One of Tomas' responsibilities was taking care of FreeIPA 
translations
(Translation Maintainer role). As far as I know, there 2 main 
tasks

associated
with the Translation Maintainer role:

1) Periodically uploading new upstream strings to the FreeIPA 
translation

platform of choice, which is the Fedora Zanata instance:
https://fedora.zanata.org/project/view/freeipa
The upload should happen periodically, on the right occasions, 
so that the
translators (especially the French and Ukrainian translations 
which have

100%
translated) have sufficient time to translate strings for the 
next version

and
do not have to translate it all in couple days before release. 
(This was

one of
the feedback I heard recently).

2) Downloading translated strings, Tomas added a short HowTo 
to our

Release page:
http://www.freeipa.org/page/Release#Translations

We will need a new volunteer who would help doing 1) and 2) 
for the planned
releases and making sure this process runs. The first task 
would be

uploading
current strings in master as the next release is FreeIPA 4.4 
planned for

June,
so it may be nice to already upload new strings we have in 
FreeIPA already

to
Zanata, so that they can be translated in sufficient time.

Volunteer(s)?

As part of the learning process, I think it would be useful to 
do more
documentation of the steps taken in every translation 
life-cycle, current

HowTo
in Release page is rather vague. I for example did not find 
information

how to
work with translation versions that I saw defined in Zanata 
(branching may

work
similarly as in current FreeIPA git).


​Thanks to the two volunteers​!

Looking forward to see this happen!

To reiterate on Martin K. message on uploads, I'd really like 
to see
regular strings uploads to the master branch in Zanata, say 
once a week or
every two weeks, so that translators can work on smaller 
strings batches.

"Release early, release oftem" :)

And strings that would be translated twice in a short time span 
wouldn't be
entirely lost because they may stay in the Zanata translation 
memory and
could help translators finalizing dot releases if the specific 
branches in

Zanata.

​And if ​we can see the upload to master soon, translators can 
start

working now before the branch for the 4.4 June release.

​Regards,

J.

Hi,

Similar thoughts here.

Thanks for feedback!

Just a note on branches, I think that it is worth to keep the 
translation just
for the current release because keeping several branches on 
Zanata (or any
other translation platform) is proved to be not effective from 
both sides,

developers and translators.
I see. My expectation would be that these branches would be only 
used if there
is a bug in the translation, not for active translation. The 
thing is, strings
in master may change or may be deleted, so they may no longer be 
applied to the
branched FreeIPA x.y.z releases. So practically, we would not be 
able to update

the translations for branched release once we branch.

Do you see that expected and acceptable?
Just two examples from the projects that I am involved (three 
polar examples).


KDE (Desktop environment):
1. Developed translation infrastructure (dedicated server,
specifically-tailored software (Lokalize)).

2. Four translation branches (stable and trunk for Qt4 and Qt5-based
applications).

3. Automatic message extraction every 24 hours.

4. Inbuild translation integration for releases.

All this needs attention and strict release rules to keep 
everything in sync.


Inkscape (SVG editor):
1. No specific infrastructure.

2. Translation branches are not strict (translators should guess 
what and where

they are translating).

3. Manual extraction from time to time.

4. No specific integration or QA.

Medium attention paid from the Inkscape developers.

GnuPG (encryption framework):
1. No specific infrastructure.

2. Strict branches (1.4, 2.0 and 2.1) but no syncing (should be 
performed by

translators manually).

3. Manual extraction. No strict release schedule. You are lucky if 
you send

your translation in time.

4. Manual integration by Werner when he finds it necessary. ;)

Minimum attention paid from the developer.

I think that FreeIPA in the sense 

Re: [Freeipa-devel] Reviving FreeIPA translations

[Martin Basti]

On 26.05.2016 16:15, Yuri Chornoivan wrote:
написане Thu, 26 May 2016 16:51:47 +0300, Martin Basti 
:





On 16.05.2016 08:37, Martin Kosek wrote:

On 05/15/2016 09:34 PM, Yuri Chornoivan wrote:
написане Sun, 15 May 2016 21:51:45 +0300, Martin Kosek 
:



On 05/14/2016 01:29 PM, Yuri Chornoivan wrote:
написане Sat, 14 May 2016 12:57:13 +0300, Jérôme Fenal 
:



2016-05-13 13:32 GMT+02:00 Martin Kosek :


Hello,

As you may or may not know, Tomas Babej left the FreeIPA team 
as a Red Hat
employee, which of course does not mean he cannot contribute to 
the FreeIPA

project, but that he won't have as much time for contributions as
previously.

One of Tomas' responsibilities was taking care of FreeIPA 
translations
(Translation Maintainer role). As far as I know, there 2 main 
tasks

associated
with the Translation Maintainer role:

1) Periodically uploading new upstream strings to the FreeIPA 
translation

platform of choice, which is the Fedora Zanata instance:
https://fedora.zanata.org/project/view/freeipa
The upload should happen periodically, on the right occasions, 
so that the
translators (especially the French and Ukrainian translations 
which have

100%
translated) have sufficient time to translate strings for the 
next version

and
do not have to translate it all in couple days before release. 
(This was

one of
the feedback I heard recently).

2) Downloading translated strings, Tomas added a short HowTo to 
our

Release page:
http://www.freeipa.org/page/Release#Translations

We will need a new volunteer who would help doing 1) and 2) for 
the planned
releases and making sure this process runs. The first task 
would be

uploading
current strings in master as the next release is FreeIPA 4.4 
planned for

June,
so it may be nice to already upload new strings we have in 
FreeIPA already

to
Zanata, so that they can be translated in sufficient time.

Volunteer(s)?

As part of the learning process, I think it would be useful to 
do more
documentation of the steps taken in every translation 
life-cycle, current

HowTo
in Release page is rather vague. I for example did not find 
information

how to
work with translation versions that I saw defined in Zanata 
(branching may

work
similarly as in current FreeIPA git).


​Thanks to the two volunteers​!

Looking forward to see this happen!

To reiterate on Martin K. message on uploads, I'd really like to 
see
regular strings uploads to the master branch in Zanata, say once 
a week or
every two weeks, so that translators can work on smaller strings 
batches.

"Release early, release oftem" :)

And strings that would be translated twice in a short time span 
wouldn't be
entirely lost because they may stay in the Zanata translation 
memory and
could help translators finalizing dot releases if the specific 
branches in

Zanata.

​And if ​we can see the upload to master soon, translators can 
start

working now before the branch for the 4.4 June release.

​Regards,

J.

Hi,

Similar thoughts here.

Thanks for feedback!

Just a note on branches, I think that it is worth to keep the 
translation just
for the current release because keeping several branches on 
Zanata (or any
other translation platform) is proved to be not effective from 
both sides,

developers and translators.
I see. My expectation would be that these branches would be only 
used if there
is a bug in the translation, not for active translation. The thing 
is, strings
in master may change or may be deleted, so they may no longer be 
applied to the
branched FreeIPA x.y.z releases. So practically, we would not be 
able to update

the translations for branched release once we branch.

Do you see that expected and acceptable?
Just two examples from the projects that I am involved (three polar 
examples).


KDE (Desktop environment):
1. Developed translation infrastructure (dedicated server,
specifically-tailored software (Lokalize)).

2. Four translation branches (stable and trunk for Qt4 and Qt5-based
applications).

3. Automatic message extraction every 24 hours.

4. Inbuild translation integration for releases.

All this needs attention and strict release rules to keep 
everything in sync.


Inkscape (SVG editor):
1. No specific infrastructure.

2. Translation branches are not strict (translators should guess 
what and where

they are translating).

3. Manual extraction from time to time.

4. No specific integration or QA.

Medium attention paid from the Inkscape developers.

GnuPG (encryption framework):
1. No specific infrastructure.

2. Strict branches (1.4, 2.0 and 2.1) but no syncing (should be 
performed by

translators manually).

3. Manual extraction. No strict release schedule. You are lucky if 
you send

your translation in time.

4. Manual integration by Werner when he finds it necessary. ;)

Minimum attention paid from the developer.

I think that FreeIPA in the sense of translation handling should be 
something

Re: [Freeipa-devel] Reviving FreeIPA translations

[Yuri Chornoivan]

написане Thu, 26 May 2016 16:51:47 +0300, Martin Basti :




On 16.05.2016 08:37, Martin Kosek wrote:

On 05/15/2016 09:34 PM, Yuri Chornoivan wrote:
написане Sun, 15 May 2016 21:51:45 +0300, Martin Kosek  
:



On 05/14/2016 01:29 PM, Yuri Chornoivan wrote:
написане Sat, 14 May 2016 12:57:13 +0300, Jérôme Fenal  
:



2016-05-13 13:32 GMT+02:00 Martin Kosek :


Hello,

As you may or may not know, Tomas Babej left the FreeIPA team as a  
Red Hat
employee, which of course does not mean he cannot contribute to  
the FreeIPA

project, but that he won't have as much time for contributions as
previously.

One of Tomas' responsibilities was taking care of FreeIPA  
translations

(Translation Maintainer role). As far as I know, there 2 main tasks
associated
with the Translation Maintainer role:

1) Periodically uploading new upstream strings to the FreeIPA  
translation

platform of choice, which is the Fedora Zanata instance:
https://fedora.zanata.org/project/view/freeipa
The upload should happen periodically, on the right occasions, so  
that the
translators (especially the French and Ukrainian translations  
which have

100%
translated) have sufficient time to translate strings for the next  
version

and
do not have to translate it all in couple days before release.  
(This was

one of
the feedback I heard recently).

2) Downloading translated strings, Tomas added a short HowTo to our
Release page:
http://www.freeipa.org/page/Release#Translations

We will need a new volunteer who would help doing 1) and 2) for  
the planned

releases and making sure this process runs. The first task would be
uploading
current strings in master as the next release is FreeIPA 4.4  
planned for

June,
so it may be nice to already upload new strings we have in FreeIPA  
already

to
Zanata, so that they can be translated in sufficient time.

Volunteer(s)?

As part of the learning process, I think it would be useful to do  
more
documentation of the steps taken in every translation life-cycle,  
current

HowTo
in Release page is rather vague. I for example did not find  
information

how to
work with translation versions that I saw defined in Zanata  
(branching may

work
similarly as in current FreeIPA git).


​Thanks to the two volunteers​!

Looking forward to see this happen!

To reiterate on Martin K. message on uploads, I'd really like to see
regular strings uploads to the master branch in Zanata, say once a  
week or
every two weeks, so that translators can work on smaller strings  
batches.

"Release early, release oftem" :)

And strings that would be translated twice in a short time span  
wouldn't be
entirely lost because they may stay in the Zanata translation  
memory and
could help translators finalizing dot releases if the specific  
branches in

Zanata.

​And if ​we can see the upload to master soon, translators can start
working now before the branch for the 4.4 June release.

​Regards,

J.

Hi,

Similar thoughts here.

Thanks for feedback!

Just a note on branches, I think that it is worth to keep the  
translation just
for the current release because keeping several branches on Zanata  
(or any
other translation platform) is proved to be not effective from both  
sides,

developers and translators.
I see. My expectation would be that these branches would be only used  
if there
is a bug in the translation, not for active translation. The thing  
is, strings
in master may change or may be deleted, so they may no longer be  
applied to the
branched FreeIPA x.y.z releases. So practically, we would not be able  
to update

the translations for branched release once we branch.

Do you see that expected and acceptable?
Just two examples from the projects that I am involved (three polar  
examples).


KDE (Desktop environment):
1. Developed translation infrastructure (dedicated server,
specifically-tailored software (Lokalize)).

2. Four translation branches (stable and trunk for Qt4 and Qt5-based
applications).

3. Automatic message extraction every 24 hours.

4. Inbuild translation integration for releases.

All this needs attention and strict release rules to keep everything  
in sync.


Inkscape (SVG editor):
1. No specific infrastructure.

2. Translation branches are not strict (translators should guess what  
and where

they are translating).

3. Manual extraction from time to time.

4. No specific integration or QA.

Medium attention paid from the Inkscape developers.

GnuPG (encryption framework):
1. No specific infrastructure.

2. Strict branches (1.4, 2.0 and 2.1) but no syncing (should be  
performed by

translators manually).

3. Manual extraction. No strict release schedule. You are lucky if you  
send

your translation in time.

4. Manual integration by Werner when he finds it necessary. ;)

Minimum attention paid from the developer.

I think that FreeIPA in the sense of translation handling should be  
something
in between. For not 

Re: [Freeipa-devel] Reviving FreeIPA translations

[Martin Basti]

On 16.05.2016 08:37, Martin Kosek wrote:

On 05/15/2016 09:34 PM, Yuri Chornoivan wrote:

написане Sun, 15 May 2016 21:51:45 +0300, Martin Kosek :


On 05/14/2016 01:29 PM, Yuri Chornoivan wrote:

написане Sat, 14 May 2016 12:57:13 +0300, Jérôme Fenal :


2016-05-13 13:32 GMT+02:00 Martin Kosek :


Hello,

As you may or may not know, Tomas Babej left the FreeIPA team as a Red Hat
employee, which of course does not mean he cannot contribute to the FreeIPA
project, but that he won't have as much time for contributions as
previously.

One of Tomas' responsibilities was taking care of FreeIPA translations
(Translation Maintainer role). As far as I know, there 2 main tasks
associated
with the Translation Maintainer role:

1) Periodically uploading new upstream strings to the FreeIPA translation
platform of choice, which is the Fedora Zanata instance:
https://fedora.zanata.org/project/view/freeipa
The upload should happen periodically, on the right occasions, so that the
translators (especially the French and Ukrainian translations which have
100%
translated) have sufficient time to translate strings for the next version
and
do not have to translate it all in couple days before release. (This was
one of
the feedback I heard recently).

2) Downloading translated strings, Tomas added a short HowTo to our
Release page:
http://www.freeipa.org/page/Release#Translations

We will need a new volunteer who would help doing 1) and 2) for the planned
releases and making sure this process runs. The first task would be
uploading
current strings in master as the next release is FreeIPA 4.4 planned for
June,
so it may be nice to already upload new strings we have in FreeIPA already
to
Zanata, so that they can be translated in sufficient time.

Volunteer(s)?

As part of the learning process, I think it would be useful to do more
documentation of the steps taken in every translation life-cycle, current
HowTo
in Release page is rather vague. I for example did not find information
how to
work with translation versions that I saw defined in Zanata (branching may
work
similarly as in current FreeIPA git).


​Thanks to the two volunteers​!

Looking forward to see this happen!

To reiterate on Martin K. message on uploads, I'd really like to see
regular strings uploads to the master branch in Zanata, say once a week or
every two weeks, so that translators can work on smaller strings batches.
"Release early, release oftem" :)

And strings that would be translated twice in a short time span wouldn't be
entirely lost because they may stay in the Zanata translation memory and
could help translators finalizing dot releases if the specific branches in
Zanata.

​And if ​we can see the upload to master soon, translators can start
working now before the branch for the 4.4 June release.

​Regards,

J.

Hi,

Similar thoughts here.

Thanks for feedback!


Just a note on branches, I think that it is worth to keep the translation just
for the current release because keeping several branches on Zanata (or any
other translation platform) is proved to be not effective from both sides,
developers and translators.

I see. My expectation would be that these branches would be only used if there
is a bug in the translation, not for active translation. The thing is, strings
in master may change or may be deleted, so they may no longer be applied to the
branched FreeIPA x.y.z releases. So practically, we would not be able to update
the translations for branched release once we branch.

Do you see that expected and acceptable?

Just two examples from the projects that I am involved (three polar examples).

KDE (Desktop environment):
1. Developed translation infrastructure (dedicated server,
specifically-tailored software (Lokalize)).

2. Four translation branches (stable and trunk for Qt4 and Qt5-based
applications).

3. Automatic message extraction every 24 hours.

4. Inbuild translation integration for releases.

All this needs attention and strict release rules to keep everything in sync.

Inkscape (SVG editor):
1. No specific infrastructure.

2. Translation branches are not strict (translators should guess what and where
they are translating).

3. Manual extraction from time to time.

4. No specific integration or QA.

Medium attention paid from the Inkscape developers.

GnuPG (encryption framework):
1. No specific infrastructure.

2. Strict branches (1.4, 2.0 and 2.1) but no syncing (should be performed by
translators manually).

3. Manual extraction. No strict release schedule. You are lucky if you send
your translation in time.

4. Manual integration by Werner when he finds it necessary. ;)

Minimum attention paid from the developer.

I think that FreeIPA in the sense of translation handling should be something
in between. For not wasting efforts of the developers (it is sensible, because
of too few more or less complete translations), I propose to have just one
branch (no switching, no 

Re: [Freeipa-devel] Provisioning throughput

[thierry bordaz]

On 05/26/2016 12:23 PM, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:
The limitation would be to run the provisioning on IPA master. 
During provisioning, membership attribute will be invalid (memberof 
not computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem 
if we

set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we 
are putting IPA master offline (is password update possible on 
replica ?).

Sure, update on replica would be queued in replication queue. Password
changes are local anyway, they result in updates of few password
attributes and that's all. These attributes replicated in the same way
as anything else.

Yes that is right.
I remember a discussion about the master key that was only available on 
IPA master and I thought that IPA master had a specific role around krb 
attributes. But if provisioning can be done on IPA master, it is then a 
good idea to use root/ldapi to avoid getting DM password.


thanks for all your feedback and help

thierry

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

[Alexander Bokovoy]

On Thu, 26 May 2016, thierry bordaz wrote:
The limitation would be to run the provisioning on IPA master. 
During provisioning, membership attribute will be invalid 
(memberof not computed). Is it acceptable that IPA master contains 
invalid membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we are 
putting IPA master offline (is password update possible on replica ?).

Sure, update on replica would be queued in replication queue. Password
changes are local anyway, they result in updates of few password
attributes and that's all. These attributes replicated in the same way
as anything else.
--
/ Alexander Bokovoy

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

[thierry bordaz]

On 05/26/2016 11:26 AM, Martin Basti wrote:



On 26.05.2016 11:24, thierry bordaz wrote:



On 05/26/2016 11:12 AM, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 




Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into 
temporary file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the 
provisioned

   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not 
match

   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the 
possibility of

looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new 
instance it

could use bulk load.
For set loop of ipa -add, I think they add many others 
direct

operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. 
Document how to turn off memberof, do the import whatever way is 
appropriate, then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a 
slew of hosts, sudorules, etc? I guess the potential is there but 
would it be on the same scale as users? If you focus only on 
users/groups does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor 
the provisioning.
If providing a set of scripts and document a procedure is enough I 
am fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the 
provisioning if it is using ldapi/DM slapi-nis will be offline by 
itself.


The limitation would be to run the provisioning on IPA master. 
During provisioning, membership attribute will be invalid (memberof 
not computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem 
if we

set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we 
are putting IPA master offline (is password update possible on 
replica ?).



thierry



How long it takes until memberof data are recreated using replication? 
(IIRC and memberof attributes are not replicated)


I have not precise data on what will be the replication latency.
IMHO provisioning will be "fast" on the server where we run the 
provisioning. Then the entries will be replicated to replicas where 
memberof is enabled. replication latency of 
users/usergroups/host/hostgroup should also be quite low, let say few 
minutes behind. Now the replication latency for entries like 
sudorules/hbacrules would be important, probably several hours.


thierry


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0484] remove unused code from automount plugin

[Martin Basti]

On 26.05.2016 08:13, Stanislav Laznicka wrote:

On 05/25/2016 06:21 PM, Martin Basti wrote:


On 25.05.2016 09:11, Stanislav Laznicka wrote:


LGTM, could you please just add the ticket to the commit message?


On 05/20/2016 04:28 PM, Martin Basti wrote:




On 20.05.2016 15:03, Martin Basti wrote:

The removed code is unused for long time.

Patch attached.




https://fedorahosted.org/freeipa/attachment/ticket/5899/





updated patch attached.

ACK

Pushed to master: 25eed1c6cb1808dfa05ce7be23aa7dea4ae0f466

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] 0001 Add missing CA options to the manpage for ipa-replica-install

[Stanislav Laznicka]

Hello,

Thank you for your first patch! It seems fine to me so ACK.

Standa

On 05/20/2016 12:52 PM, Florence Blanc-Renaud wrote:


Hi all,

this one will be my first patch submission, so apologies in advance if 
I make mistakes...


The man page for ipa-replica-install was missing some commands related 
to CA-less installation, as well as --allow-zone-overlap and 
--auto-reverse. I added them in the relevant sections (CERTIFICATE 
SYSTEM OPTIONS and DNS OPTIONS).


I also fixed a wrong short option for --realm (-r).

Fixes: https://fedorahosted.org/freeipa/ticket/5835




-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

[Martin Basti]

On 26.05.2016 11:24, thierry bordaz wrote:



On 05/26/2016 11:12 AM, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 




Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into 
temporary file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not 
match

   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the 
possibility of

looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new 
instance it

could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. 
Document how to turn off memberof, do the import whatever way is 
appropriate, then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would 
it be on the same scale as users? If you focus only on users/groups 
does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor 
the provisioning.
If providing a set of scripts and document a procedure is enough I 
am fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the 
provisioning if it is using ldapi/DM slapi-nis will be offline by 
itself.


The limitation would be to run the provisioning on IPA master. 
During provisioning, membership attribute will be invalid (memberof 
not computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we are 
putting IPA master offline (is password update possible on replica ?).



thierry



How long it takes until memberof data are recreated using replication? 
(IIRC and memberof attributes are not replicated)


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

[thierry bordaz]

On 05/26/2016 11:12 AM, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 




Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into temporary 
file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not 
match

   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the 
possibility of

looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new 
instance it

could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would 
it be on the same scale as users? If you focus only on users/groups 
does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor the 
provisioning.
If providing a set of scripts and document a procedure is enough I am 
fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the provisioning 
if it is using ldapi/DM slapi-nis will be offline by itself.


The limitation would be to run the provisioning on IPA master. During 
provisioning, membership attribute will be invalid (memberof not 
computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.


Yes I agree, provisioning mode is offline.
My concern is about side effects on the rest of the topology if we are 
putting IPA master offline (is password update possible on replica ?).



thierry

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

[Martin Basti]

On 26.05.2016 11:12, Alexander Bokovoy wrote:

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 




Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into temporary 
file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not 
match

   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the 
possibility of

looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new 
instance it

could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would 
it be on the same scale as users? If you focus only on users/groups 
does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor the 
provisioning.
If providing a set of scripts and document a procedure is enough I am 
fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the provisioning 
if it is using ldapi/DM slapi-nis will be offline by itself.


The limitation would be to run the provisioning on IPA master. During 
provisioning, membership attribute will be invalid (memberof not 
computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.

I agree

Martin^2

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [WIP] Thin client

[Martin Basti]

On 26.05.2016 07:15, Jan Cholasta wrote:

On 25.5.2016 18:17, Martin Basti wrote:



On 25.05.2016 16:07, Jan Cholasta wrote:

On 25.5.2016 15:03, David Kupka wrote:

On 18/05/16 08:01, Jan Cholasta wrote:

On 16.5.2016 16:35, Martin Basti wrote:



On 09.05.2016 14:07, Jan Cholasta wrote:

On 6.5.2016 14:32, Martin Basti wrote:



On 28.04.2016 14:45, Jan Cholasta wrote:

Hi,

I have pushed my thin client WIP branch to GitHub:
.

All commits up to "ipalib: use relative imports for cross-plugin
imports" should be good for review. The rest is subject to change
(WARNING: I will force push into this branch).

Honza


I did not test it yet, I just checked the code

* automount: do not inherit automountlocation_tofiles from
LDAPQuery *
LGTM

* dns: move all dnsrecord code called on client to a single 
class *

LGTM

* dns: do not rely on server data structures in code called on
client *
1)
you forgot to increment VERSION


This was deliberate, as it will no longer be necessary to bump
VERSION
for backward compatible changes after this whole patchset is 
merged.

But we're not there yet, so fixed.


How we should handle VERSION after your patches?


Otherwise LGTM

* otptoken: fix import of DN *
ACK

* otptoken_yubikey: fix otptoken_add_yubikey arguments *
1)
you forgot to increment VERSION


Fixed.



2)
Did you find out why this was issue?
-del answer['value']# Why does this cause an error if
omitted?
 -del answer['summary']  # Why does this cause an error if
omitted?


The command definition was not complete, it was missing has_output.



Otherwise LGTM

* vault: move client-side code to the module level *
LGTM

* vault: copy arguments of client commands from server
counterparts *
1)
you forgot to increment Version


Fixed.



* ipalib: use relative imports for cross-plugin imports *
1) Missing explanation for future generations why this change is
needed
in commit message


Fixed.



The other commits I will check later.
Martin^2



Okay. Thanks.



* frontend: remove the unused Command.soft_validate method
LGTM

* frontend: perform argument value validation only on server
LGTM

* frontend: do not forward argument defaults to server
I'm not a fan of returning  None in promt_param function, but I 
havent

found anything better to use.


It always returned None for unset params.


LGTM

* ipalib: optimize API.txt
this contains a lot of black magic, but because this is mainly 
copy of

current to proper places, LGTM


It's actually mostly cut-n-paste.



* makeaci: load additional plugins using API.add_module
Looks good, but I miss explanation why is this change needed


The next change would be impossible without this.



* plugable: replace API.import_plugins with new API.add_package
LGTM


* ipalib, ipaserver: migrate all plugins to Registry-based
registration
ACK

* ipalib, ipaserver: fix incorrect API.register calls in docstrings
LGTM


* plugable: remove the unused deprecated API.register method
LGTM


* plugable: switch API to Registry-based plugin discovery
LGTM

* frontend: merge baseldap.CallbackRegistry into Command
LGTM

*frontend: move the interactive_prompt callback type to Command
 LGTM

Martin^2







Hello,
first batch of 30 patches from Honza's trac-4739 branch
(https://github.com/jcholast/freeipa/commits/trac-4739) is ready to be
pushed into master.
All upto "frontend: allow commands to have an argument named 
`name`" got

over numerous test cycles in last week+ and is working as expected
now, ACK.


Thanks for the review.



Honzo, please rebase and push them, thanks!



Attaching the patches for reference.

Pushed to master: 2b50fc617024f18a81e97f30f75ed1b9221c4055



Guys, you forgot to test it with newer pylint.


I don't see any "BuildRequires: newer pylint" in the spec file.



Patch attached.


LGTM, although the commit message is wrong - this is not related to 
thin client at all, PublicError and PublicMessage had .kw since forever.



updated commit message



From a8a759a9f64ada4defda8cb3bc9021d93e3e64ce Mon Sep 17 00:00:00 2001
From: Martin Basti 
Date: Wed, 25 May 2016 18:12:57 +0200
Subject: [PATCH] fix pylint false positive errors

pylint 1.5 reports 'kw' as 'no-member' for PublicError and
PublicMessage. It is false positive in both cases.

https://fedorahosted.org/freeipa/ticket/4739
---
 pylint_plugins.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/pylint_plugins.py b/pylint_plugins.py
index 7460d9f4a4c45fd036a32da3e746a4836791a0c2..631d85ab1d9e79b7b90cd1c7742e64a5b74e4f61 100644
--- a/pylint_plugins.py
+++ b/pylint_plugins.py
@@ -114,6 +114,7 @@ ipa_class_members = {
 'ipalib.errors.PublicError': [
 'msg',
 'strerror',
+'kw',
 ],
 'ipalib.errors.SingleMatchExpected': [
 'found',
@@ -128,6 +129,7 @@ ipa_class_members = {
 'msg',
 'strerror',
 'type',
+'kw',
 ],
 'ipalib.parameters.Param': [
 

Re: [Freeipa-devel] Provisioning throughput

[Alexander Bokovoy]

On Thu, 26 May 2016, thierry bordaz wrote:



On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly 
and with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements)


Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done 
with memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into 
temporary file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not match
   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would 
it be on the same scale as users? If you focus only on users/groups 
does that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor the 
provisioning.
If providing a set of scripts and document a procedure is enough I am 
fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id 
is root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the provisioning 
if it is using ldapi/DM slapi-nis will be offline by itself.


The limitation would be to run the provisioning on IPA master. During 
provisioning, membership attribute will be invalid (memberof not 
computed). Is it acceptable that IPA master contains invalid 
membership for some time ?

Consider provisioning to be at the same level as running
ipa-server-upgrade -- access via 389/636 ports is not allowed, LDAPI is
the only interface enabled which implies there would be no problem if we
set expectations right: provisioning mode is offline.
--
/ Alexander Bokovoy

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

[thierry bordaz]

On 05/26/2016 09:32 AM, Alexander Bokovoy wrote:

On Wed, 25 May 2016, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly and 
with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 



Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into temporary 
file.

   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not match
   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew 
of hosts, sudorules, etc? I guess the potential is there but would it 
be on the same scale as users? If you focus only on users/groups does 
that change the use case at all?

I tend to agree with Rob on this. Maybe we should have a simple
script/update file that does preparatory work and another one that
returns configuration into the right state and document how to use them.


Ok.
rereading the thread I realize we are talking of 
user/usergroups/host/hostgroup.


Provisioning such entries is not that bad.
For example 5Kusers/hosts are provisioned in 5min without memberof and 
19min with memberof


The real problem is provisioning sudorules and hbacrules where the 
impact of memberof is very important.
For example 100 sudorules are provisioned in 30s without memberof and 2h 
with memberof.


Do you think provisioning should also considere sudorules/hbac or only 
user/usergroups/host/hostgroup ?


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

[thierry bordaz]

On 05/25/2016 09:31 PM, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly and 
with

additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 



Several improvements can be done, in particular in DS plugins 
(memberof,
retroCL), but for "easy" benefit provisioning will be done with 
memberof

disabled followed by fixup.

It remains some aspects that are not clear to me:

  * For best performance, DS tuning and provisioning/fixup would
preferably be done under 'directory manager'
That means prompting DM password and writing it into temporary 
file.

Is that a concern ?
  * Fixup requires that we know the filters matching the provisioned
entries. For example :
  o (objectClass=inetorgperson)
  o (objectClass=ipausergroup)
  o (objectClass=ipahost)
  o (objectClass=ipahostgroup)
  o (objectClass=ipasudorule)
  o (objectClass=ipahbacrule)

The set of objectclass could be hardcode or provided in the
provisioning CLI option
What to do if an entry in in the provision file does not match
any of those filter ? Should it stop without starting the
provisioning ?
  * The CLI doing the provisioning could be something like 'ipa
provision ' or should it be a separated command e.g.
ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew of 
hosts, sudorules, etc? I guess the potential is there but would it be 
on the same scale as users? If you focus only on users/groups does 
that change the use case at all?


In fact, I am using such small scripts to prepare and run/monitor the 
provisioning.
If providing a set of scripts and document a procedure is enough I am 
fine with this.



Would it be reasonable to require bulk import to be done on an IPA
master so we can leverage the ldapi socket?

Do you mean using ldapi to reduce network latency or automember or
something else ?


To avoid the DM password issues. ldapi autobinds to DM when the id is 
root.


Yes I said automember but was thinking to autobind.
That is nice idea to avoid prompting DM password.
In addition, slapi-nis participating to slowing down the provisioning if 
it is using ldapi/DM slapi-nis will be offline by itself.


The limitation would be to run the provisioning on IPA master. During 
provisioning, membership attribute will be invalid (memberof not 
computed). Is it acceptable that IPA master contains invalid membership 
for some time ?




rob



--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Provisioning throughput

[Alexander Bokovoy]

On Wed, 25 May 2016, Rob Crittenden wrote:

thierry bordaz wrote:



On 05/25/2016 08:49 PM, Rob Crittenden wrote:

thierry bordaz wrote:


Hello,

Thanks for all the feedbacks. I updated the design accordingly and with
additional tests results
(http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements)

Several improvements can be done, in particular in DS plugins (memberof,
retroCL), but for "easy" benefit provisioning will be done with memberof
disabled followed by fixup.

It remains some aspects that are not clear to me:

 * For best performance, DS tuning and provisioning/fixup would
   preferably be done under 'directory manager'
   That means prompting DM password and writing it into temporary file.
   Is that a concern ?
 * Fixup requires that we know the filters matching the provisioned
   entries. For example :
 o (objectClass=inetorgperson)
 o (objectClass=ipausergroup)
 o (objectClass=ipahost)
 o (objectClass=ipahostgroup)
 o (objectClass=ipasudorule)
 o (objectClass=ipahbacrule)

   The set of objectclass could be hardcode or provided in the
   provisioning CLI option
   What to do if an entry in in the provision file does not match
   any of those filter ? Should it stop without starting the
   provisioning ?
 * The CLI doing the provisioning could be something like 'ipa
   provision ' or should it be a separated command e.g.
   ipa-bulk-load ?


It depends. There is a migration command now, ipa migrate-ds, that
adds records and is impacted by this. There is also the possibility of
looping calls to ipa [user|group|etc]-add.


I agree that migration and bulk load can be linked. If migration
dump/update a set of entries before filling them into a new instance it
could use bulk load.
For set loop of ipa -add, I think they add many others direct
operations (mainly SRCH) before doing the ADD in order to check
coherency. bulk load looks more straightforward.


I just wonder if some (all) of this could be done manually. Document 
how to turn off memberof, do the import whatever way is appropriate, 
then run the fixup? I'm not sure what you had in mind.


I don't want to think small but do we expect to be importing a slew of 
hosts, sudorules, etc? I guess the potential is there but would it be 
on the same scale as users? If you focus only on users/groups does 
that change the use case at all?

I tend to agree with Rob on this. Maybe we should have a simple
script/update file that does preparatory work and another one that
returns configuration into the right state and document how to use them.

--
/ Alexander Bokovoy

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0484] remove unused code from automount plugin

[Stanislav Laznicka]

On 05/25/2016 06:21 PM, Martin Basti wrote:


On 25.05.2016 09:11, Stanislav Laznicka wrote:


LGTM, could you please just add the ticket to the commit message?


On 05/20/2016 04:28 PM, Martin Basti wrote:




On 20.05.2016 15:03, Martin Basti wrote:

The removed code is unused for long time.

Patch attached.




https://fedorahosted.org/freeipa/attachment/ticket/5899/





updated patch attached.

ACK
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code