Re: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates

2012-07-30 Thread Jan Cholasta 

Dne 27.7.2012 22:50, Rob Crittenden napsal(a):

Jan Cholasta wrote:

Dne 25.7.2012 22:58, Rob Crittenden napsal(a):

Jan Cholasta wrote:


All these scripts could use more exception handling, but I guess
potential bugs can be sorted out later.


Well, they all run in the background so even if they caught errors
nothing would see them unless we decide to syslog errors.


I decided to syslog the errors, there is no other way around this.



install/share/default-aci.ldif:

The ACIs are wrong (Kerberos principal instead of ldap URI in
userdn, in
40-delegation.update it is done right).


Nice catch, not sure how I missed that. Fixed.


You forgot to fix the allow(add) one, it still has userdn =
host/$FQDN@$REALM.



Fixed.


I did:

1. ipa-server-install on host1, using IPA from master
2. ipa-replica-install on host2, using IPA from master
3. update host1 to IPA with your patch applied
4. update host2 to IPA with your patch applied
5. ipa-ca-install on host2

After that, ipaCert is not tracked on host2 at all (I had to add it
manually using getcert start-tracking -d /etc/httpd/alias -n ipaCert -c
dogtag-ipa-retrieve-agent-submit -C
/usr/lib64/ipa/certmonger/restart_httpd -p /etc/httpd/alias/pwdfile.txt
-T ipaCert).


Fixed, it wasn't being tracked on upgrades.

I filed a ticket for the audit cert renewing for only 6 months. It is a
dogtag bug.


OK, thanks.



I've seen some oddness when testing by moving the date forward, CS
replication has stopped working. I kick it with ipa-csreplica-manage
force-sync --from=master and that fixes things. This is unrelated to
my patch.

rob


ACK.

Honza

--
Jan Cholasta

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket

2012-07-30 Thread Alexander Bokovoy 

On Fri, 27 Jul 2012, Rob Crittenden wrote:

Alexander Bokovoy wrote:

On Thu, 26 Jul 2012, Alexander Bokovoy wrote:

Hi,

When setting up AD trusts support, ipa-adtrust-install utility
needs to be run as:
 - root, for performing Samba configuration and using LDAPI/autobind
 - kinit-ed IPA admin user, to ensure proper ACIs are granted to
   fetch keytab

As result, we can get rid of Directory Manager credentials in
ipa-adtrust-install

https://fedorahosted.org/freeipa/ticket/2815

This ticket also simplifies a bit the way we handle admin connection in
Service class and particulary in Service._ldap_mod() by defaulting to
LDAPI/autobind in case of running as root and to GSSAPI otherwise.
Except few cases in remote replica management (not applicable in
_ldap_mod() case) we always run installation tools as root and can
benefit from using autobind feature. Unfortunately, it is not yet
possible to get away from using DM credentials for all cases as the same
class is used to perform initial directory server instance
configuration.

One side effect is explicit disconnect and reconnect in
Service.add_cert_to_service() due to way how SimpleLDAPObject class
handles stale connections (no handling at all). I've put some comments
in place so that others would not try to err out optimizing it in
future.

Finally, with next patch series which will introduce syncing ipaNTHash
attribute with RC4 key in existing kerberos credentials, we can remove
requirements to change passwords or re-kinit for majority of trust
cases. This should then conclude our trusts content for beta2 release.


Patch updated, fixed small typo (auth_parms was initialized as
auth_params which led to non-existing auth_parms in ipa-adtrust-install
case).


Nack, a couple of minor issues:

The exception handling is rather unusual in 
ensure_kerberos_admin_rights(api). I'm not sure if this is any more 
efficient than a series of excepts...

I've rewrote this code and put it directly in the main.


You don't need to pass in api, it's a global.

Fixed.


It may be safe to see if the user is in the group the way you are 
doing it, I wonder if it would be clearer to cast those into DN 
objects.

Not sure if checking DNs would be sustaining in long run. Ideally we
should check ACI here, not just hardcoded group name. I'd like to keep
it explicit with memberof for now because it shows what exactly we want
to check.

In the Service class what is the point of ldapi if it is going to be 
ignored in the case we know the realm? What if I really, really just 
want to use a password?

LDAPI bind in IPAAdmin.__local_init() requires that there is realm known.
No realm -- no LDAPI use because we otherwise cannot construct the
socket name. For 'just want to use a password' case you can simply set
self.dm_password.

However, I've changed the code in Service.ldap_connect() to do
following:

1. if DM password is provided, we'll try to use it
2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm
3. Otherwise (ldapi was False or realm not provided), we'll try to
   connect to fqdn:389 with GSSAPI

I think this covers all cases.

And later where it forces ldapi, it seems better to either commit all 
the way and drop the ldapi argument or convert it to a better name 
(like autobind).

ldapi requires realm but can be used with either GSSAPI or autobind.
Calling it autobind isn't really correct as autobind only available on
ldapi under root.

--
/ Alexander Bokovoy
From d7fea3f6f50218821560ea739a26030f3a76640f Mon Sep 17 00:00:00 2001
From: Alexander Bokovoy aboko...@redhat.com
Date: Fri, 13 Jul 2012 18:12:48 +0300
Subject: [PATCH 2/6] Ensure ipa-adtrust-install is run with Kerberos ticket
 for admin user

When setting up AD trusts support, ipa-adtrust-install utility
needs to be run as:
   - root, for performing Samba configuration and using LDAPI/autobind
   - kinit-ed IPA admin user, to ensure proper ACIs are granted to
 fetch keytab

As result, we can get rid of Directory Manager credentials in 
ipa-adtrust-install

https://fedorahosted.org/freeipa/ticket/2815
---
 install/tools/ipa-adtrust-install   | 37 +--
 install/tools/man/ipa-adtrust-install.1 |  3 --
 ipaserver/install/adtrustinstance.py| 21 ++---
 ipaserver/install/service.py| 53 -
 4 files changed, 71 insertions(+), 43 deletions(-)

diff --git a/install/tools/ipa-adtrust-install 
b/install/tools/ipa-adtrust-install
index 
6678018e6346d75d5042894cfb833d38079d3f21..f1cff30bd439ee07226034b3e49ae17cc59ce6b0
 100755
--- a/install/tools/ipa-adtrust-install
+++ b/install/tools/ipa-adtrust-install
@@ -37,8 +37,6 @@ log_file_name = /var/log/ipaserver-install.log
 
 def parse_options():
 parser = IPAOptionParser(version=version.VERSION)
-parser.add_option(-p, --ds-password, dest=dm_password,
-  sensitive=True, help=directory manager password)
 parser.add_option(-d, --debug, dest=debug,

Re: [Freeipa-devel] [PATCH] 1033 renew CA subsystem certificates

2012-07-30 Thread Martin Kosek 
On 07/30/2012 10:54 AM, Jan Cholasta wrote:
 Dne 27.7.2012 22:50, Rob Crittenden napsal(a):
 Jan Cholasta wrote:
 Dne 25.7.2012 22:58, Rob Crittenden napsal(a):
 Jan Cholasta wrote:

 All these scripts could use more exception handling, but I guess
 potential bugs can be sorted out later.

 Well, they all run in the background so even if they caught errors
 nothing would see them unless we decide to syslog errors.

 I decided to syslog the errors, there is no other way around this.


 install/share/default-aci.ldif:

 The ACIs are wrong (Kerberos principal instead of ldap URI in
 userdn, in
 40-delegation.update it is done right).

 Nice catch, not sure how I missed that. Fixed.

 You forgot to fix the allow(add) one, it still has userdn =
 host/$FQDN@$REALM.


 Fixed.

 I did:

 1. ipa-server-install on host1, using IPA from master
 2. ipa-replica-install on host2, using IPA from master
 3. update host1 to IPA with your patch applied
 4. update host2 to IPA with your patch applied
 5. ipa-ca-install on host2

 After that, ipaCert is not tracked on host2 at all (I had to add it
 manually using getcert start-tracking -d /etc/httpd/alias -n ipaCert -c
 dogtag-ipa-retrieve-agent-submit -C
 /usr/lib64/ipa/certmonger/restart_httpd -p /etc/httpd/alias/pwdfile.txt
 -T ipaCert).

 Fixed, it wasn't being tracked on upgrades.

 I filed a ticket for the audit cert renewing for only 6 months. It is a
 dogtag bug.
 
 OK, thanks.
 

 I've seen some oddness when testing by moving the date forward, CS
 replication has stopped working. I kick it with ipa-csreplica-manage
 force-sync --from=master and that fixes things. This is unrelated to
 my patch.

 rob
 
 ACK.
 
 Honza
 

Pushed to master.

Martin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket

2012-07-30 Thread Simo Sorce 
On Mon, 2012-07-30 at 14:34 +0300, Alexander Bokovoy wrote:
 On Fri, 27 Jul 2012, Rob Crittenden wrote:
 Alexander Bokovoy wrote:
 On Thu, 26 Jul 2012, Alexander Bokovoy wrote:
 Hi,
 
 When setting up AD trusts support, ipa-adtrust-install utility
 needs to be run as:
   - root, for performing Samba configuration and using LDAPI/autobind
   - kinit-ed IPA admin user, to ensure proper ACIs are granted to
 fetch keytab
 
 As result, we can get rid of Directory Manager credentials in
 ipa-adtrust-install
 
 https://fedorahosted.org/freeipa/ticket/2815
 
 This ticket also simplifies a bit the way we handle admin connection in
 Service class and particulary in Service._ldap_mod() by defaulting to
 LDAPI/autobind in case of running as root and to GSSAPI otherwise.
 Except few cases in remote replica management (not applicable in
 _ldap_mod() case) we always run installation tools as root and can
 benefit from using autobind feature. Unfortunately, it is not yet
 possible to get away from using DM credentials for all cases as the same
 class is used to perform initial directory server instance
 configuration.
 
 One side effect is explicit disconnect and reconnect in
 Service.add_cert_to_service() due to way how SimpleLDAPObject class
 handles stale connections (no handling at all). I've put some comments
 in place so that others would not try to err out optimizing it in
 future.
 
 Finally, with next patch series which will introduce syncing ipaNTHash
 attribute with RC4 key in existing kerberos credentials, we can remove
 requirements to change passwords or re-kinit for majority of trust
 cases. This should then conclude our trusts content for beta2 release.
 
 Patch updated, fixed small typo (auth_parms was initialized as
 auth_params which led to non-existing auth_parms in ipa-adtrust-install
 case).
 
 Nack, a couple of minor issues:
 
 The exception handling is rather unusual in 
 ensure_kerberos_admin_rights(api). I'm not sure if this is any more 
 efficient than a series of excepts...
 I've rewrote this code and put it directly in the main.
 
 You don't need to pass in api, it's a global.
 Fixed.
 
 
 It may be safe to see if the user is in the group the way you are 
 doing it, I wonder if it would be clearer to cast those into DN 
 objects.
 Not sure if checking DNs would be sustaining in long run. Ideally we
 should check ACI here, not just hardcoded group name. I'd like to keep
 it explicit with memberof for now because it shows what exactly we want
 to check.
 
 In the Service class what is the point of ldapi if it is going to be 
 ignored in the case we know the realm? What if I really, really just 
 want to use a password?
 LDAPI bind in IPAAdmin.__local_init() requires that there is realm known.
 No realm -- no LDAPI use because we otherwise cannot construct the
 socket name. For 'just want to use a password' case you can simply set
 self.dm_password.
 
 However, I've changed the code in Service.ldap_connect() to do
 following:
 
 1. if DM password is provided, we'll try to use it
 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and 
 realm
 3. Otherwise (ldapi was False or realm not provided), we'll try to
 connect to fqdn:389 with GSSAPI
 
 I think this covers all cases.
 
 And later where it forces ldapi, it seems better to either commit all 
 the way and drop the ldapi argument or convert it to a better name 
 (like autobind).
 ldapi requires realm but can be used with either GSSAPI or autobind.
 Calling it autobind isn't really correct as autobind only available on
 ldapi under root.

Alexander, the realm is always available ion api.* so it seem that the
case where the realm is missing can basically never happen, and it seem
like it make no sense to explicitly check that case.

This means you'll never fall back to GSSAPI unless you also check what
user are you running as. I think we should not try ldapi unless
explicitly requested if we are not root (uid 0).

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket

2012-07-30 Thread Martin Kosek 
On 07/30/2012 01:34 PM, Alexander Bokovoy wrote:
 On Fri, 27 Jul 2012, Rob Crittenden wrote:
 Alexander Bokovoy wrote:
 On Thu, 26 Jul 2012, Alexander Bokovoy wrote:
 Hi,

 When setting up AD trusts support, ipa-adtrust-install utility
 needs to be run as:
  - root, for performing Samba configuration and using LDAPI/autobind
  - kinit-ed IPA admin user, to ensure proper ACIs are granted to
fetch keytab

 As result, we can get rid of Directory Manager credentials in
 ipa-adtrust-install

 https://fedorahosted.org/freeipa/ticket/2815

 This ticket also simplifies a bit the way we handle admin connection in
 Service class and particulary in Service._ldap_mod() by defaulting to
 LDAPI/autobind in case of running as root and to GSSAPI otherwise.
 Except few cases in remote replica management (not applicable in
 _ldap_mod() case) we always run installation tools as root and can
 benefit from using autobind feature. Unfortunately, it is not yet
 possible to get away from using DM credentials for all cases as the same
 class is used to perform initial directory server instance
 configuration.

 One side effect is explicit disconnect and reconnect in
 Service.add_cert_to_service() due to way how SimpleLDAPObject class
 handles stale connections (no handling at all). I've put some comments
 in place so that others would not try to err out optimizing it in
 future.

 Finally, with next patch series which will introduce syncing ipaNTHash
 attribute with RC4 key in existing kerberos credentials, we can remove
 requirements to change passwords or re-kinit for majority of trust
 cases. This should then conclude our trusts content for beta2 release.

 Patch updated, fixed small typo (auth_parms was initialized as
 auth_params which led to non-existing auth_parms in ipa-adtrust-install
 case).

 Nack, a couple of minor issues:

 The exception handling is rather unusual in
 ensure_kerberos_admin_rights(api). I'm not sure if this is any more efficient
 than a series of excepts...
 I've rewrote this code and put it directly in the main.
 
 You don't need to pass in api, it's a global.
 Fixed.
 
 
 It may be safe to see if the user is in the group the way you are doing it, I
 wonder if it would be clearer to cast those into DN objects.
 Not sure if checking DNs would be sustaining in long run. Ideally we
 should check ACI here, not just hardcoded group name. I'd like to keep
 it explicit with memberof for now because it shows what exactly we want
 to check.
 
 In the Service class what is the point of ldapi if it is going to be ignored
 in the case we know the realm? What if I really, really just want to use a
 password?
 LDAPI bind in IPAAdmin.__local_init() requires that there is realm known.
 No realm -- no LDAPI use because we otherwise cannot construct the
 socket name. For 'just want to use a password' case you can simply set
 self.dm_password.
 
 However, I've changed the code in Service.ldap_connect() to do
 following:
 
 1. if DM password is provided, we'll try to use it
 2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and 
 realm
 3. Otherwise (ldapi was False or realm not provided), we'll try to
connect to fqdn:389 with GSSAPI
 
 I think this covers all cases.
 
 And later where it forces ldapi, it seems better to either commit all the way
 and drop the ldapi argument or convert it to a better name (like autobind).
 ldapi requires realm but can be used with either GSSAPI or autobind.
 Calling it autobind isn't really correct as autobind only available on
 ldapi under root.
 

Works fine, I also have just few minor-ish issues:

1) Uncatched exception

We may want to also catch for DatabaseException in this section:

+api.Backend.ldap2.connect(ccache.name)
+except errors.ACIError, e:
+sys.exit(Outdated Kerberos credentials. Use kdestroy and kinit to
update your ticket)

Otherwise ipa-adtrust-install throws unexpected exception when IPA is down:

# ipactl stop
# ipa-adtrust-install
...
NetBIOS domain name [IDM]:

Unexpected error - see /var/log/ipaserver-install.log for details:
DatabaseError: Can't contact LDAP server:


2) Wrong indentation:
...
+except errors.RequirementError, e:
+   sys.exit(Must have administrative privileges to setup AD trusts on
server)
+except Exception, e:
+   sys.exit(Unrecognized error during check of admin rights: %s %
(str(e)))

Martin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] [PATCH] 0065 Ensure ipa-adtrust-install is run with administrator privileges and Kerberos ticket

2012-07-30 Thread Alexander Bokovoy 

On Mon, 30 Jul 2012, Simo Sorce wrote:

On Mon, 2012-07-30 at 14:34 +0300, Alexander Bokovoy wrote:

On Fri, 27 Jul 2012, Rob Crittenden wrote:
Alexander Bokovoy wrote:
On Thu, 26 Jul 2012, Alexander Bokovoy wrote:
Hi,

When setting up AD trusts support, ipa-adtrust-install utility
needs to be run as:
  - root, for performing Samba configuration and using LDAPI/autobind
  - kinit-ed IPA admin user, to ensure proper ACIs are granted to
fetch keytab

As result, we can get rid of Directory Manager credentials in
ipa-adtrust-install

https://fedorahosted.org/freeipa/ticket/2815

This ticket also simplifies a bit the way we handle admin connection in
Service class and particulary in Service._ldap_mod() by defaulting to
LDAPI/autobind in case of running as root and to GSSAPI otherwise.
Except few cases in remote replica management (not applicable in
_ldap_mod() case) we always run installation tools as root and can
benefit from using autobind feature. Unfortunately, it is not yet
possible to get away from using DM credentials for all cases as the same
class is used to perform initial directory server instance
configuration.

One side effect is explicit disconnect and reconnect in
Service.add_cert_to_service() due to way how SimpleLDAPObject class
handles stale connections (no handling at all). I've put some comments
in place so that others would not try to err out optimizing it in
future.

Finally, with next patch series which will introduce syncing ipaNTHash
attribute with RC4 key in existing kerberos credentials, we can remove
requirements to change passwords or re-kinit for majority of trust
cases. This should then conclude our trusts content for beta2 release.

Patch updated, fixed small typo (auth_parms was initialized as
auth_params which led to non-existing auth_parms in ipa-adtrust-install
case).

Nack, a couple of minor issues:

The exception handling is rather unusual in
ensure_kerberos_admin_rights(api). I'm not sure if this is any more
efficient than a series of excepts...
I've rewrote this code and put it directly in the main.

You don't need to pass in api, it's a global.
Fixed.


It may be safe to see if the user is in the group the way you are
doing it, I wonder if it would be clearer to cast those into DN
objects.
Not sure if checking DNs would be sustaining in long run. Ideally we
should check ACI here, not just hardcoded group name. I'd like to keep
it explicit with memberof for now because it shows what exactly we want
to check.

In the Service class what is the point of ldapi if it is going to be
ignored in the case we know the realm? What if I really, really just
want to use a password?
LDAPI bind in IPAAdmin.__local_init() requires that there is realm known.
No realm -- no LDAPI use because we otherwise cannot construct the
socket name. For 'just want to use a password' case you can simply set
self.dm_password.

However, I've changed the code in Service.ldap_connect() to do
following:

1. if DM password is provided, we'll try to use it
2. Otherwise, if LDAPI is asked for and realm is set, we'll use LDAPI and realm
3. Otherwise (ldapi was False or realm not provided), we'll try to
connect to fqdn:389 with GSSAPI

I think this covers all cases.

And later where it forces ldapi, it seems better to either commit all
the way and drop the ldapi argument or convert it to a better name
(like autobind).
ldapi requires realm but can be used with either GSSAPI or autobind.
Calling it autobind isn't really correct as autobind only available on
ldapi under root.


Alexander, the realm is always available ion api.* so it seem that the
case where the realm is missing can basically never happen, and it seem
like it make no sense to explicitly check that case.

This is the code that has no access to api.* because it is run during
install when config files are not yet written.



This means you'll never fall back to GSSAPI unless you also check what
user are you running as. I think we should not try ldapi unless
explicitly requested if we are not root (uid 0).

There are checks already, look at Service.__get_conn() for real logic.

In Service class there are two stages:
1. Service.ldap_connect() which takes existing values from Service's
self and decides how to call __get_conn().
2. Service.__get_conn() which does connection and binds depending on its
arguments.

This split seems to be historical as nobody else is calling into
__get_conn() at all so all possible users of __get_conn() are in
Service.ldap_connect().



--
/ Alexander Bokovoy

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] [PATCHES][RFC] Implement special operation to revoer NT hash for a user

2012-07-30 Thread Simo Sorce 
On Fri, 2012-07-27 at 18:54 -0400, Simo Sorce wrote:
 On Fri, 2012-07-27 at 07:15 +0300, Alexander Bokovoy wrote:
  On Thu, 12 Jul 2012, Alexander Bokovoy wrote:
  On Thu, 12 Jul 2012, Simo Sorce wrote:
  On Thu, 2012-07-12 at 10:48 +0300, Alexander Bokovoy wrote:
  On Wed, 11 Jul 2012, Simo Sorce wrote:
  From 84ef09a1193ff42fc301fb71354055c5039f51a5 Mon Sep 17 00:00:00 2001
  From: Simo Sorce sso...@redhat.com
  Date: Fri, 6 Jul 2012 16:18:29 -0400
  Subject: [PATCH] Add special modify op to regen ipaNTHash
  
  The NT Hash is the same thing as the RC4-HMAC key, so we add a function 
  to
  extract it from krb5 keys if they are available to avoid forcing a 
  password
  change when configuring trust relationships.
  ---
   .../ipa-pwd-extop/ipapwd_prepost.c |  147 
   +++-
   1 file changed, 144 insertions(+), 3 deletions(-)
  
  diff --git a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c 
  b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c
  index 
  deae642f82edcc4674a1c9580661c3dae94b..24fa52eb9ac92004576ccdba4f576162c358770d
   100644
  --- a/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c
  +++ b/daemons/ipa-slapi-plugins/ipa-pwd-extop/ipapwd_prepost.c
  @@ -41,7 +41,12 @@
   #  include config.h
   #endif
  
  -#define _XOPEN_SOURCE /* strptime needs this */
  +/* strptime needs _XOPEN_SOURCE and endian.h needs __USE_BSD
  + * _GNU_SOURCE imply both, and we use it elsewhere, so use this */
  +#ifndef _GNU_SOURCE
  +#define _GNU_SOURCE 1
  +#endif
  +
   #include stdio.h
   #include string.h
   #include strings.h
  @@ -53,6 +58,7 @@
   #include dirsrv/slapi-plugin.h
   #include lber.h
   #include time.h
  +#include endian.h
  
   #include ipapwd.h
   #include util.h
  @@ -379,6 +385,12 @@ done:
   return 0;
   }
  
  +#define NTHASH_REGEN_VAL MagicRegen
  +#define NTHASH_REGEN_LEN sizeof(NTHASH_REGEN_VAL)
  +static int ipapwd_regen_nthash(Slapi_PBlock *pb, Slapi_Mods *smods,
  +   char *dn, struct slapi_entry *entry,
  +   struct ipapwd_krbcfg *krbcfg);
  +
   /* PRE MOD Operation:
* Gets the clean text password (fail the operation if the password 
   came
* pre-hashed, unless this is a replicated operation).
  @@ -407,6 +419,7 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb)
   int has_krb_keys = 0;
   int has_history = 0;
   int gen_krb_keys = 0;
  +int is_magic_regen = 0;
   int ret, rc;
  
   LOG_TRACE( =\n);
  @@ -447,6 +460,27 @@ static int ipapwd_pre_mod(Slapi_PBlock *pb)
   default:
   break;
   }
  +} else if (slapi_attr_types_equivalent(lmod-mod_type, 
  ipaNTHash)) {
  +/* check op filtering out LDAP_MOD_BVALUES */
  +switch (lmod-mod_op  0x0f) {
  +case LDAP_MOD_REPLACE:
  This is still LDAP_MOD_REPLACE, not LDAP_MOD_ADD.
  
  This is because I resent the old patch :(
  
  Hopefully the correct patch is now attached.
  Yes, now it is updated, thanks.
  
  I'm going to experiment a bit with these patches, adding ipasam
  responder to test them.
  Here is ipasam part.
 
 ACK the ipasam part.

Pushed all 4 patches to master.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] slow response

2012-07-30 Thread John Dennis 

On 07/30/2012 11:30 AM, Stephen Ingram wrote:

On Wed, Jul 25, 2012 at 11:18 PM, Stephen Ingram sbing...@gmail.com wrote:

On Wed, Jul 25, 2012 at 4:31 AM, John Dennis jden...@redhat.com wrote:

On 07/25/2012 02:59 AM, Stephen Ingram wrote:


Seeing python2.7, I'm guessing these patches were against Fedora.
Since I couldn't get them to apply against RH 6.3 I applied them by
hand. I couldn't get the WebUI to load after applying the patches. I'm
not sure of the code that caused the problem, but I did see mention of
global name datetime not being defined. You wouldn't happen to have
patches for RH 6.3?



Sorry, I thought you had a F17 install. It should be easy to fix the
datetime issue, just add this add the top of the file:

import time, datetime

If that simple fix doesn't work then let me know the version you've got and
I'll make up a new patch against that version.


Yes, please send a patch for 2.20 with RH 6.3. The import time,
datetime did not address all of the errors.


Not sure if you missed this one or not, but, yes, I do need a patch
for 2.20 on RH 6.3. I added datetime, but there were still other
errors that I couldn't resolve.


Sorry, I did see it but I was busy with other things. You can try the 
attached patch. However you need to have a clean set of files to patch 
against. The patch command I suggested included the -b option and would 
have created backup files of the original file with a .orig suffix. 
These are the files the patch touches.


ipalib/session.py
ipaserver/plugins/ldap2.py
ipaserver/rpcserver.py


Find their .orig version and cp it back to the original filename. Then 
you can try applying the patch again.


Caveat: I did not test this against 2.20, I just manually made the same 
set of changes and generated a patch, it's possible in my hurry I made a 
mistake you may need to tweak,


--
John Dennis jden...@redhat.com

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/
diff --git a/ipalib/session.py b/ipalib/session.py
index 4b783bb..8eb9105 100644
--- a/ipalib/session.py
+++ b/ipalib/session.py
@@ -23,6 +23,7 @@ import errors
 import os
 import re
 import time
+import datetime
 from text import _
 from ipapython.ipa_log_manager import *
 from ipalib import api, errors
@@ -628,9 +629,9 @@ mod_auth_kerb. Everything else remains the same.
 
 default_max_session_duration = 60*60 # number of seconds
 
-ISO8601_DATETIME_FMT = '%Y-%m-%dT%H:%M:%S' # FIXME jrd, this should be defined elsewhere
+ISO8601_DATETIME_FMT = '%Y-%m-%dT%H:%M:%S.%f' # FIXME jrd, this should be defined elsewhere
 def fmt_time(timestamp):
-return time.strftime(ISO8601_DATETIME_FMT, time.localtime(timestamp))
+return datetime.fromtimestamp(timestamp).strftime(ISO8601_DATETIME_FMT)
 
 #---
 
diff --git a/ipaserver/plugins/ldap2.py b/ipaserver/plugins/ldap2.py
index ddef8df..2d7447d 100644
--- a/ipaserver/plugins/ldap2.py
+++ b/ipaserver/plugins/ldap2.py
@@ -733,6 +733,7 @@ class ldap2(CrudBackend, Encoder):
 attrs_list = list(set(attrs_list))
 
 # pass arguments to python-ldap
+start_time = time.time()
 try:
 id = self.conn.search_ext(
 base_dn, scope, filter, attrs_list, timeout=time_limit,
@@ -751,6 +752,9 @@ class ldap2(CrudBackend, Encoder):
 except _ldap.LDAPError, e:
 _handle_errors(e)
 
+end_time = time.time()
+root_logger.debug('ldap2 find_entries: elapsed_time=%f', end_time-start_time)
+
 if not res and not truncated:
 raise errors.NotFound(reason='no such entry')
 
@@ -1381,4 +1385,3 @@ class ldap2(CrudBackend, Encoder):
 return (len(output), output)
 
 api.register(ldap2)
-
diff --git a/ipaserver/rpcserver.py b/ipaserver/rpcserver.py
index 83c5c2d..ffee3c6 100644
--- a/ipaserver/rpcserver.py
+++ b/ipaserver/rpcserver.py
@@ -293,6 +293,7 @@ class WSGIExecutioner(Executioner):
 args = ()
 options = {}
 
+start_time = time.time()
 if not 'HTTP_REFERER' in environ:
 return self.marshal(result, RefererError(referer='missing'), _id)
 if not environ['HTTP_REFERER'].startswith('https://%s/ipa' % self.api.env.host) and not self.env.in_tree:
@@ -337,10 +338,11 @@ class WSGIExecutioner(Executioner):
 # get at least some context of what is going on
 params = options
 principal = getattr(context, 'principal', 'UNKNOWN')
+end_time = time.time()
 if error:
 self.info('%s: %s(%s): %s', principal, name, ', '.join(self.Command[name]._repr_iter(**params)), e.__class__.__name__)
 else:
-self.info('%s: %s(%s): SUCCESS', principal, name, ', '.join(self.Command[name]._repr_iter(**params)))
+self.info('%s: %s(%s): SUCCESS elapsed_time=%f', principal, name, ', '.join(self.Command[name]._repr_iter(**params)), end_time-start_time)

Re: [Freeipa-devel] slow response

2012-07-30 Thread Stephen Ingram 
On Mon, Jul 30, 2012 at 9:15 AM, John Dennis jden...@redhat.com wrote:
 On 07/30/2012 11:30 AM, Stephen Ingram wrote:

 On Wed, Jul 25, 2012 at 11:18 PM, Stephen Ingram sbing...@gmail.com
 wrote:

 On Wed, Jul 25, 2012 at 4:31 AM, John Dennis jden...@redhat.com wrote:

 On 07/25/2012 02:59 AM, Stephen Ingram wrote:


 Seeing python2.7, I'm guessing these patches were against Fedora.
 Since I couldn't get them to apply against RH 6.3 I applied them by
 hand. I couldn't get the WebUI to load after applying the patches. I'm
 not sure of the code that caused the problem, but I did see mention of
 global name datetime not being defined. You wouldn't happen to have
 patches for RH 6.3?



 Sorry, I thought you had a F17 install. It should be easy to fix the
 datetime issue, just add this add the top of the file:

 import time, datetime

 If that simple fix doesn't work then let me know the version you've got
 and
 I'll make up a new patch against that version.


 Yes, please send a patch for 2.20 with RH 6.3. The import time,
 datetime did not address all of the errors.


 Not sure if you missed this one or not, but, yes, I do need a patch
 for 2.20 on RH 6.3. I added datetime, but there were still other
 errors that I couldn't resolve.


 Sorry, I did see it but I was busy with other things. You can try the
 attached patch. However you need to have a clean set of files to patch
 against. The patch command I suggested included the -b option and would have
 created backup files of the original file with a .orig suffix. These are the
 files the patch touches.

 ipalib/session.py
 ipaserver/plugins/ldap2.py
 ipaserver/rpcserver.py


 Find their .orig version and cp it back to the original filename. Then you
 can try applying the patch again.

 Caveat: I did not test this against 2.20, I just manually made the same set
 of changes and generated a patch, it's possible in my hurry I made a mistake
 you may need to tweak,

This is pretty much the same thing I applied previously by hand. I
noticed that you placed import datetime on its own line so I followed
suit. Here are the errors I'm receiving:

[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.] mod_wsgi
(pid=2634): Exception occurred processing WSGI script
'/usr/share/ipa/wsgi.py'.
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] Traceback (most
recent call last):
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]   File
/usr/share/ipa/wsgi.py, line 49, in application
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] return
api.Backend.wsgi_dispatch(environ, start_response)
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]   File
/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py, line 234,
in __call__
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] return
self.route(environ, start_response)
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]   File
/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py, line 246,
in route
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] return
app(environ, start_response)
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]   File
/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py, line 784,
in __call__
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] session_data =
session_mgr.load_session_data(environ.get('HTTP_COOKIE'))
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]   File
/usr/lib/python2.6/site-packages/ipalib/session.py, line 1004, in
load_session_data
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]
self.store_session_data(session_data)
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]   File
/usr/lib/python2.6/site-packages/ipalib/session.py, line 1047, in
store_session_data
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]
fmt_time(session_data['session_start_timestamp']),
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x]   File
/usr/lib/python2.6/site-packages/ipalib/session.py, line 634, in
fmt_time
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] return
datetime.fromtimestamp(timestamp).strftime(ISO8601_DATETIME_FMT)
[Mon Jul 30 10:09:09 2012] [error] [client x.x.x.x] AttributeError:
'module' object has no attribute 'fromtimestamp'

There seems to be a problem with the format of the log information
returned? Perhaps the fromtimestamp attribute was added after version
2.20?

Steve

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

[Freeipa-devel] [PATCH 0045] Fix zone transfers with non-FQDNs

2012-07-30 Thread Petr Spacek 

Hello,

this patch enables (finally!) full-fledged AXFR zone-transfers.

One-liner was enough, after two days of debugging ... :-)

I tried to find all consequences for this change, but please test it thoroughly.

Petr^2 Spacek
From c73219008ddd8e68ab6bfa7485efc25effc04d1b Mon Sep 17 00:00:00 2001
From: Petr Spacek pspa...@redhat.com
Date: Mon, 30 Jul 2012 19:39:14 +0200
Subject: [PATCH] Fix zone transfers with non-FQDNs.

https://fedorahosted.org/bind-dyndb-ldap/ticket/47

Signed-off-by: Petr Spacek pspa...@redhat.com
---
 src/ldap_driver.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/ldap_driver.c b/src/ldap_driver.c
index cae45d4f6cc1f201c40ca3413d1f626e03a0318e..d958d15bdebe5e89dc4948655ffba655073d53e0 100644
--- a/src/ldap_driver.c
+++ b/src/ldap_driver.c
@@ -689,7 +689,7 @@ createiterator(dns_db_t *db,
 
 	ldapdb_t *ldapdb = (ldapdb_t *) db;
 	result = ldapdb_nodelist_get(ldapdb-common.mctx, ldapdb-ldap_inst,
-	 ldapdb-common.origin, NULL,
+	 ldapdb-common.origin, ldapdb-common.origin,
 	 ldapiter-nodelist);
 
 	*iteratorp = (dns_dbiterator_t *) ldapiter;
-- 
1.7.11.2

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] slow response

2012-07-30 Thread John Dennis 

On 07/30/2012 01:17 PM, Stephen Ingram wrote:
 This is pretty much the same thing I applied previously by hand. I

noticed that you placed import datetime on its own line so I followed
suit. Here are the errors I'm receiving:

There seems to be a problem with the format of the log information
returned? Perhaps the fromtimestamp attribute was added after version
2.20?


sorry, that should be:

return 
datetime.datetime.fromtimestamp(timestamp).strftime(ISO8601_DATETIME_FMT)



In other words datetime has to be repeated twice. That's because 
fromtimestamp() is an attribute of the datetime class in the datetime 
module, thus the first datetime references the module and the second 
datetime references the datetime class in the datetime module.


I wish I had a dime for every time I typed just a single datetime ;-)

--
John Dennis jden...@redhat.com

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] slow response

2012-07-30 Thread John Dennis 
Stephen finally got his server patched with my timing instrumentation. 
He sent me his log file and I ran my script to parse it. The results are 
quite interesting.


Basically I timed two things

cmd_duration is the elapsed time to execute the IPA command.

json_duration is the elapsed time to execute the JSON RPC containing the 
IPA command. The difference (delta) between the two is the time we spend 
doing session bookkeeping. In all but the first RPC we were executing 
from the session cache. Results are below.


What is interesting is every command that executes from the session 
cache seems to have about a 1.5 second constant overhead. The other 
interesting bits are that most commands in the server execute quickly 
(from under a tenth of second to .3 second) and on average the server 
takes 1.5 seconds to respond to RPC (of which most of it appears to be 
in session code).


What is taking so long with session bookkeeping? I don't know yet. I 
would need more timing instrumentation. I will say when I looked at the 
python-krb5 code (which we use to populate the ccache from the session 
and read back to store in the session) seemed to be remarkably 
inefficient. We also elected to use file based ccache rather than 
in-memory ccache (that means there is a bit of file-IO occurring).


Has anyone used a Python profiler? Would that be better than adding 
instrumentation?


BTW, the actual commands were stripped to protect data anonymity.

cmd_duration: 0.107673 json_duration: 3.176758 delta: 3.069085
cmd_duration: 0.020068 json_duration: 1.543135 delta: 1.523067
cmd_duration: 0.387210 json_duration: 1.802954 delta: 1.415744
cmd_duration: 0.024086 json_duration: 1.405276 delta: 1.381190
cmd_duration: 0.342808 json_duration: 1.950365 delta: 1.607557
cmd_duration: 0.019286 json_duration: 1.398786 delta: 1.379500
cmd_duration: 0.200895 json_duration: 1.754358 delta: 1.553463
cmd_duration: 0.020293 json_duration: 1.410701 delta: 1.390408
cmd_duration: 0.383433 json_duration: 1.819478 delta: 1.436045
cmd_duration: 0.020350 json_duration: 1.406038 delta: 1.385688
cmd_duration: 0.346864 json_duration: 1.771281 delta: 1.424417
cmd_duration: 0.015998 json_duration: 1.707743 delta: 1.691745
cmd_duration: 0.314527 json_duration: 1.730894 delta: 1.416367
cmd_duration: 0.025323 json_duration: 1.582828 delta: 1.557505

--
John Dennis jden...@redhat.com

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel