Re: [Freeipa-devel] Unit tests failing on F18

2012-10-15 Thread Martin Kosek
On 10/12/2012 06:16 PM, John Dennis wrote:
 On 10/12/2012 11:20 AM, Martin Kosek wrote:
 Hello,

 I was investigating global unit test failure on Fedora 18 for most of today, 
 I
 would like to share results I found so far.

 Unit test and its related scripts on F18 now reports NSS BUSY exception, just
 like this one:

 # ./make-testcert
 Traceback (most recent call last):
File ./make-testcert, line 134, in module
  sys.exit(makecert(reqdir))
File ./make-testcert, line 111, in makecert
  add=True)
File ./make-testcert, line 68, in run
  result = self.execute(method, *args, **options)
File /root/freeipa-master2/ipalib/backend.py, line 146, in execute
  raise error #pylint: disable=E0702
 ipalib.errors.NetworkError: cannot connect to
 'http://vm-042.idm.lab.bos.redhat.com/ipa/session/xml': [Errno -8053]
 (SEC_ERROR_BUSY) NSS could not shutdown. Objects are still in use.

 Something In F18 must have changed, this worked before... But leaked
 NSSConnection objects without proper close() now ends with the exception 
 above.

 In case of make-testcert script, the exception is raised because the script
 does the following procedure:

 1) connect, do one command
 2) disconnect
 3) connect, do second command

 However, during disconnect, NSSConnection is leaked which makes NSS very
 uncomfortable during second connection atempt (and nss_shutdown()). I managed
 to fix this issue with attached patch. ./make-testcert or ./make-test
 tests/test_xmlrpc/test_group_plugin.py works fine now.

 But global ./make-test still fails, I think there is some remaining
 NSSConnection leak, I suspect there is something wrong with how we use our
 context (threading.local object). It looses a connection or some other thread
 invoked in ldap2 module may be kicking in, here is my debug output:

 CONTEXT[xmlclient] = ipalib.request.Connection object at 0x9a1f5ec

 Test a simple LDAP bind using ldap2 ... SKIP: No directory manager password 
 in
 /root/.ipa/.dmpw
 Test the `ipaserver.rpcserver.jsonserver.unmarshal` method. ... ok
 tests.test_ipaserver.test_rpcserver.test_session.test_mount ... CONTEXT
 150714476: GET languages

 CONTEXT[xmlclient] = None

 The connection is in the context, but then something happens and it is gone.
 Then, unit tests try to connect again and NSS fails.

 I would be really glad if somebody with a knowledge of NSS or how threads in
 Python/IPA work could give me some advice...
 
 O.K. I'll take a look at it. I seem to recall Rob looked into something 
 similar
 a couple of days ago. Rob, do you have any additional information to share?

Great, with your NSS+Python knowledge this should be walk in the park :-)
 Any luck with investigation of this issue?

Thanks,
Martin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] Unit tests failing on F18

2012-10-15 Thread Martin Kosek
On 10/12/2012 06:21 PM, Rob Crittenden wrote:
 John Dennis wrote:
 On 10/12/2012 11:20 AM, Martin Kosek wrote:
 Hello,

 I was investigating global unit test failure on Fedora 18 for most of
 today, I
 would like to share results I found so far.

 Unit test and its related scripts on F18 now reports NSS BUSY
 exception, just
 like this one:

 # ./make-testcert
 Traceback (most recent call last):
File ./make-testcert, line 134, in module
  sys.exit(makecert(reqdir))
File ./make-testcert, line 111, in makecert
  add=True)
File ./make-testcert, line 68, in run
  result = self.execute(method, *args, **options)
File /root/freeipa-master2/ipalib/backend.py, line 146, in execute
  raise error #pylint: disable=E0702
 ipalib.errors.NetworkError: cannot connect to
 'http://vm-042.idm.lab.bos.redhat.com/ipa/session/xml': [Errno -8053]
 (SEC_ERROR_BUSY) NSS could not shutdown. Objects are still in use.

 Something In F18 must have changed, this worked before... But leaked
 NSSConnection objects without proper close() now ends with the
 exception above.

 In case of make-testcert script, the exception is raised because the
 script
 does the following procedure:

 1) connect, do one command
 2) disconnect
 3) connect, do second command

 However, during disconnect, NSSConnection is leaked which makes NSS very
 uncomfortable during second connection atempt (and nss_shutdown()). I
 managed
 to fix this issue with attached patch. ./make-testcert or ./make-test
 tests/test_xmlrpc/test_group_plugin.py works fine now.

 But global ./make-test still fails, I think there is some remaining
 NSSConnection leak, I suspect there is something wrong with how we use
 our
 context (threading.local object). It looses a connection or some other
 thread
 invoked in ldap2 module may be kicking in, here is my debug output:

 CONTEXT[xmlclient] = ipalib.request.Connection object at 0x9a1f5ec

 Test a simple LDAP bind using ldap2 ... SKIP: No directory manager
 password in
 /root/.ipa/.dmpw
 Test the `ipaserver.rpcserver.jsonserver.unmarshal` method. ... ok
 tests.test_ipaserver.test_rpcserver.test_session.test_mount ... CONTEXT
 150714476: GET languages

 CONTEXT[xmlclient] = None

 The connection is in the context, but then something happens and it is
 gone.
 Then, unit tests try to connect again and NSS fails.

 I would be really glad if somebody with a knowledge of NSS or how
 threads in
 Python/IPA work could give me some advice...

 O.K. I'll take a look at it. I seem to recall Rob looked into something
 similar a couple of days ago. Rob, do you have any additional
 information to share?


 
 Nothing to add. I looked at this briefly yesterday but nothing jumped out at
 me. I think this may be unrelated to the previous problem I investigated. It
 very well could be we are not closing a connection somewhere.
 
 rob

Yes, we are not. I fixed it in one place (patch in my original e-mail). With
the patch, one could run at least some parts of the unit test. I assume we are
simply not treating initializing and shutting down of NSS correctly in some
places (also with regards to threads), but I have not found a fix yet.

Martin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


[Freeipa-devel] [PATCH 0079] Update NEWS file for 2.0 release

2012-10-15 Thread Petr Spacek

Hello,

Update NEWS file for 2.0 release

--
Petr^2 Spacek
From 304efbbd91db537f6b49cc380f8807c63a64beab Mon Sep 17 00:00:00 2001
From: Petr Spacek pspa...@redhat.com
Date: Mon, 15 Oct 2012 10:37:01 +0200
Subject: [PATCH] Update NEWS file for 2.0 release.

Signed-off-by: Petr Spacek pspa...@redhat.com
---
 NEWS | 38 ++
 1 file changed, 38 insertions(+)

diff --git a/NEWS b/NEWS
index 02e484506bdac32180f19b5e288204a08213..9e3818f4c53107786aaabfbafa57c5cb715a13da 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,41 @@
+2.0
+==
+New features:
+[1] SOA serial number can be incremented automatically after each change
+in LDAP database. (Configuration option serial_autoincrement.)
+
+Bug fixes:
+[1] It was possible to DoS named service via quiery which contained
+$ character. CVE-2012-3429 was fixed.
+
+[2] 18 crashes were fixed (crash on BIND reload, invalid zone handling,
+   persistent search and others).
+
+[3] DNS Dynamic Update returns codes NOTAUTH and REFUSED properly.
+
+[4] BIND will not refuse to start if initial connection times out.
+
+[5] Object renaming (LDAP moddn) in persistent is handled properly.
+
+[6] Internal record cache is flushed properly after reconnection
+to LDAP server (in configurations with persistent search).
+
+[7] Simple time-based deadlock detection code was added. Error message
+is printed after 10*(timeout) seconds.
+Some deadlocks in various situations with low connection count were fixed.
+
+[8] Libdns interface version = 90 is supported properly.
+
+[9] Zone transfers were fixed. Records with non-FQDNs are handled properly.
+
+[10] Logging in various situations was improved.
+
+[11] Memory leaks in dynamic update, persistent search, ldap_query
+ and configurations with multiple plugin instances were fixed.
+
+[12] Version numbering format changed to: [features].[bugfixes]
+
+
 1.1.0rc1
 ==
 
-- 
1.7.11.4

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] [PATCH] 1062 fix dogtag replication

2012-10-15 Thread Martin Kosek
On 10/12/2012 09:00 PM, Rob Crittenden wrote:
 This patch changes the replication protocol from SSL to TLS. This will fix
 installing a replica CA along with an updated version of dogtag that fixes
 other issues.
 
 rob
 

I tested 2.0 - 3.0 and 3.0 - 3.0 CA replicas and the recent dogtag + ipa with
this patch worked fine.

ACK. Pushed to master, ipa-3-0.

Martin

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH 0078] Use automatic connection management in LDAP modification code to prevent potential deadlock

2012-10-15 Thread Petr Spacek

On 10/09/2012 03:49 PM, Petr Spacek wrote:

On 10/09/2012 01:21 PM, Adam Tkac wrote:

On Mon, Oct 08, 2012 at 04:46:54PM +0200, Petr Spacek wrote:

Hello,

 Use automatic connection management in LDAP modification code to
 prevent potential deadlock.

 Without this patch the plugin will deadlock when modify_ldap_common()
 is called with PTR synchronization enabled and only single
 connection is available in the connection pool.


Nack

If I read the patch correctly, it leaves unused ldap_conn parameters in
ldap_modify_do() and modify_soa_record() functions.

Those params are always NULL so they can be safely removed. Please also remove
the autoconn variable from ldap_modify_do()


My intent was to keep the same connection management abilities as are in
ldap_query(): You can avoid repetitive ldap_pool_get/putconnection() calls by
passing connection via parameter.

I can remove it if it isn't worth. (Actually *_modify_*() functions do not use
this capability now.)


I forgot to send the patch after our discussion on IRC. Attached patch removes 
unused parameters.



Petr^2 Spacek


From 7924717aab7d1dc343f21f6f459c1b4f21b373e1 Mon Sep 17 00:00:00 2001
From: Petr Spacek pspa...@redhat.com
Date: Mon, 8 Oct 2012 16:41:40 +0200
Subject: [PATCH] Use automatic connection management in LDAP modification
 code to prevent potential deadlock.

Without this patch the plugin will deadlock when modify_ldap_common()
is called with PTR synchronization enabled and only single
connection is available in the connection pool.

Signed-off-by: Petr Spacek pspa...@redhat.com
---
 src/ldap_helper.c | 34 ++
 1 file changed, 14 insertions(+), 20 deletions(-)

diff --git a/src/ldap_helper.c b/src/ldap_helper.c
index f8df1b29871c28a1eeecfa93d5d91edd1aee3944..c91aea44aab845854818373b8b9dd91fb0e059c9 100644
--- a/src/ldap_helper.c
+++ b/src/ldap_helper.c
@@ -306,8 +306,7 @@ static void ldap_query_free(isc_boolean_t prepare_reuse, ldap_qresult_t **ldap_q
 
 /* Functions for writing to LDAP. */
 static isc_result_t ldap_modify_do(ldap_instance_t *ldap_inst,
-		ldap_connection_t *ldap_conn, const char *dn, LDAPMod **mods,
-		isc_boolean_t delete_node);
+		const char *dn, LDAPMod **mods,	isc_boolean_t delete_node);
 static isc_result_t ldap_rdttl_to_ldapmod(isc_mem_t *mctx,
 		dns_rdatalist_t *rdlist, LDAPMod **changep);
 static isc_result_t ldap_rdatalist_to_ldapmod(isc_mem_t *mctx,
@@ -2278,21 +2277,20 @@ reconnect:
 }
 
 static isc_result_t
-ldap_modify_do(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn,
-		const char *dn, LDAPMod **mods, isc_boolean_t delete_node)
+ldap_modify_do(ldap_instance_t *ldap_inst, const char *dn, LDAPMod **mods,
+	   isc_boolean_t delete_node)
 {
 	int ret;
 	int err_code;
 	const char *operation_str;
 	isc_result_t result;
-	isc_boolean_t autoconn = (ldap_conn == NULL);
+	ldap_connection_t *ldap_conn = NULL;
 
 	REQUIRE(dn != NULL);
 	REQUIRE(mods != NULL);
 	REQUIRE(ldap_inst != NULL);
 
-	if (autoconn)
-		CHECK(ldap_pool_getconnection(ldap_inst-pool, ldap_conn));
+	CHECK(ldap_pool_getconnection(ldap_inst-pool, ldap_conn));
 
 	if (ldap_conn-handle == NULL) {
 		/*
@@ -2375,8 +2373,7 @@ ldap_modify_do(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn,
 		result = ISC_R_FAILURE;
 	}
 cleanup:
-	if (autoconn)
-		ldap_pool_putconnection(ldap_inst-pool, ldap_conn);
+	ldap_pool_putconnection(ldap_inst-pool, ldap_conn);
 
 	return result;
 }
@@ -2544,8 +2541,8 @@ cleanup:
  * refresh, retry, expire and minimum attributes for each SOA record.
  */
 static isc_result_t
-modify_soa_record(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn,
-		const char *zone_dn, dns_rdata_t *rdata)
+modify_soa_record(ldap_instance_t *ldap_inst, const char *zone_dn,
+		  dns_rdata_t *rdata)
 {
 	isc_result_t result;
 	dns_rdata_soa_t soa;
@@ -2578,7 +2575,7 @@ modify_soa_record(ldap_instance_t *ldap_inst, ldap_connection_t *ldap_conn,
 
 	dns_rdata_freestruct((void *)soa);
 
-	result = ldap_modify_do(ldap_inst, ldap_conn, zone_dn, changep, ISC_FALSE);
+	result = ldap_modify_do(ldap_inst, zone_dn, changep, ISC_FALSE);
 
 cleanup:
 	return result;
@@ -2593,7 +2590,6 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst,
 {
 	isc_result_t result;
 	isc_mem_t *mctx = ldap_inst-mctx;
-	ldap_connection_t *ldap_conn = NULL;
 	ld_string_t *owner_dn = NULL;
 	LDAPMod *change[3] = { NULL };
 	LDAPMod *change_ptr = NULL;
@@ -2630,8 +2626,6 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst,
 
 	CHECK(dn_to_dnsname(mctx, zone_dn, zone_name, NULL));
 
-	CHECK(ldap_pool_getconnection(ldap_inst-pool, ldap_conn));
-
 	result = zr_get_zone_settings(ldap_inst-zone_register, zone_name,
   zone_settings);
 	if (result != ISC_R_SUCCESS) {
@@ -2655,7 +2649,7 @@ modify_ldap_common(dns_name_t *owner, ldap_instance_t *ldap_inst,
 	CHECK(discard_from_cache(cache, owner));
 
 	if (rdlist-type == dns_rdatatype_soa) {
-		result = 

[Freeipa-devel] [PATCH] 0091 ipautil.run: Log the command line before running the command

2012-10-15 Thread Petr Viktorin
As I was debugging code that calls long-running or failing commands, I 
got tired of the invocation being logged after the command is done.

This patch should improve the logging.

https://fedorahosted.org/freeipa/ticket/3174
---
PetrĀ³

From dd504f133857d310938ba3c43065485ee90b6073 Mon Sep 17 00:00:00 2001
From: Petr Viktorin pvikt...@redhat.com
Date: Tue, 25 Sep 2012 09:29:49 -0400
Subject: [PATCH] ipautil.run: Log the command line before running the command

When the user interrupts a long-running command, this ensures that
the command is logged. Also, when watching log files (or the -d
output), it's apparent what's being done.

https://fedorahosted.org/freeipa/ticket/3174
---
 ipapython/ipautil.py | 17 +++--
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/ipapython/ipautil.py b/ipapython/ipautil.py
index 0b519c2957f63770f9a28d7abe9083f724a9cf40..aa86f7557be0d90ac7ca7298aa0c5023bd776696 100644
--- a/ipapython/ipautil.py
+++ b/ipapython/ipautil.py
@@ -292,30 +292,35 @@ def run(args, stdin=None, raiseonerr=True,
 p_out = subprocess.PIPE
 p_err = subprocess.PIPE
 
+arg_string = nolog_replace(' '.join(args), nolog)
+root_logger.debug('Starting external process')
+root_logger.debug('args=%s' % arg_string)
+
 try:
 p = subprocess.Popen(args, stdin=p_in, stdout=p_out, stderr=p_err,
  close_fds=True, env=env, cwd=cwd)
 stdout,stderr = p.communicate(stdin)
 stdout,stderr = str(stdout), str(stderr)# Make pylint happy
 except KeyboardInterrupt:
+root_logger.debug('Process interrupted')
 p.wait()
 raise
+except:
+root_logger.debug('Process failed')
+raise
+
+root_logger.debug('Process successful')
 
 # The command and its output may include passwords that we don't want
 # to log. Replace those.
-args = ' '.join(args)
 if capture_output:
 stdout = nolog_replace(stdout, nolog)
 stderr = nolog_replace(stderr, nolog)
-args = nolog_replace(args, nolog)
-
-root_logger.debug('args=%s' % args)
-if capture_output:
 root_logger.debug('stdout=%s' % stdout)
 root_logger.debug('stderr=%s' % stderr)
 
 if p.returncode != 0 and raiseonerr:
-raise CalledProcessError(p.returncode, args)
+raise CalledProcessError(p.returncode, arg_string)
 
 return (stdout, stderr, p.returncode)
 
-- 
1.7.11.7

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] [PATCH 0079] Update NEWS file for 2.0 release

2012-10-15 Thread Adam Tkac
On Mon, Oct 15, 2012 at 10:38:41AM +0200, Petr Spacek wrote:
 Hello,
 
   Update NEWS file for 2.0 release

Hi Peter,

are you OK with this version of NEWS? (patch attached)

A

-- 
Adam Tkac, Red Hat, Inc.
From bcd017c75978e2f78976bb8a2b6d47af26df429a Mon Sep 17 00:00:00 2001
From: Petr Spacek pspa...@redhat.com
Date: Mon, 15 Oct 2012 10:37:01 +0200
Subject: [PATCH] Update NEWS file for 2.0 release.

Signed-off-by: Petr Spacek pspa...@redhat.com
Signed-off-by: Adam Tkac at...@redhat.com
---
 NEWS | 34 ++
 1 file changed, 34 insertions(+)

diff --git a/NEWS b/NEWS
index 02e4845..4312741 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,37 @@
+2.0
+==
+[1] SOA serial number can be incremented automatically after each change
+in LDAP database. (Configuration option serial_autoincrement.)
+
+[2] It was possible to DoS named service via quiery which contained
+$ character. CVE-2012-3429 was fixed.
+
+[3] DNS Dynamic Update returns codes NOTAUTH and REFUSED properly.
+
+[4] BIND doesn't refuse to start if initial connection times out.
+
+[5] Object renaming (LDAP moddn) in persistent mode is handled properly.
+
+[6] Internal record cache is flushed properly after reconnection
+to the LDAP server (in configurations with persistent search).
+
+[7] Simple time-based deadlock detection code was added. Error message
+is printed after 10*(timeout) seconds.
+Some deadlocks in various situations with low connection count were fixed.
+
+[8] Libdns interface version = 90 is supported properly.
+
+[9] Zone transfers were fixed. Records with non-FQDNs are handled properly.
+
+[10] Logging was improved.
+
+[11] Memory leaks in dynamic update, persistent search, ldap_query
+ and configurations with multiple plugin instances were fixed.
+
+[12] Version numbering format changed to: [features].[bugfixes]
+
+[13] Many other bugfixes
+
 1.1.0rc1
 ==
 
-- 
1.7.11.7

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] Unit tests failing on F18

2012-10-15 Thread Rob Crittenden

Martin Kosek wrote:

On 10/12/2012 06:21 PM, Rob Crittenden wrote:

John Dennis wrote:

On 10/12/2012 11:20 AM, Martin Kosek wrote:

Hello,

I was investigating global unit test failure on Fedora 18 for most of
today, I
would like to share results I found so far.

Unit test and its related scripts on F18 now reports NSS BUSY
exception, just
like this one:

# ./make-testcert
Traceback (most recent call last):
File ./make-testcert, line 134, in module
  sys.exit(makecert(reqdir))
File ./make-testcert, line 111, in makecert
  add=True)
File ./make-testcert, line 68, in run
  result = self.execute(method, *args, **options)
File /root/freeipa-master2/ipalib/backend.py, line 146, in execute
  raise error #pylint: disable=E0702
ipalib.errors.NetworkError: cannot connect to
'http://vm-042.idm.lab.bos.redhat.com/ipa/session/xml': [Errno -8053]
(SEC_ERROR_BUSY) NSS could not shutdown. Objects are still in use.

Something In F18 must have changed, this worked before... But leaked
NSSConnection objects without proper close() now ends with the
exception above.

In case of make-testcert script, the exception is raised because the
script
does the following procedure:

1) connect, do one command
2) disconnect
3) connect, do second command

However, during disconnect, NSSConnection is leaked which makes NSS very
uncomfortable during second connection atempt (and nss_shutdown()). I
managed
to fix this issue with attached patch. ./make-testcert or ./make-test
tests/test_xmlrpc/test_group_plugin.py works fine now.

But global ./make-test still fails, I think there is some remaining
NSSConnection leak, I suspect there is something wrong with how we use
our
context (threading.local object). It looses a connection or some other
thread
invoked in ldap2 module may be kicking in, here is my debug output:

CONTEXT[xmlclient] = ipalib.request.Connection object at 0x9a1f5ec

Test a simple LDAP bind using ldap2 ... SKIP: No directory manager
password in
/root/.ipa/.dmpw
Test the `ipaserver.rpcserver.jsonserver.unmarshal` method. ... ok
tests.test_ipaserver.test_rpcserver.test_session.test_mount ... CONTEXT
150714476: GET languages

CONTEXT[xmlclient] = None

The connection is in the context, but then something happens and it is
gone.
Then, unit tests try to connect again and NSS fails.

I would be really glad if somebody with a knowledge of NSS or how
threads in
Python/IPA work could give me some advice...


O.K. I'll take a look at it. I seem to recall Rob looked into something
similar a couple of days ago. Rob, do you have any additional
information to share?




Nothing to add. I looked at this briefly yesterday but nothing jumped out at
me. I think this may be unrelated to the previous problem I investigated. It
very well could be we are not closing a connection somewhere.

rob


Yes, we are not. I fixed it in one place (patch in my original e-mail). With
the patch, one could run at least some parts of the unit test. I assume we are
simply not treating initializing and shutting down of NSS correctly in some
places (also with regards to threads), but I have not found a fix yet.


If you modify tests/test_xmlrpc/xmlrpc_test.py and remove 
request.destroy_context() from tearDown then the tests will pass. But 
this shouldn't be necessary, we really want to clean up the context 
between requests.


I'm guessing that we're being affected by an external change. It could 
be in a number of places: httplib, xmlrpclib, NSS, etc.


rob

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] Unit tests failing on F18

2012-10-15 Thread John Dennis

On 10/15/2012 02:27 AM, Martin Kosek wrote:

On 10/12/2012 06:16 PM, John Dennis wrote:

O.K. I'll take a look at it. I seem to recall Rob looked into something similar
a couple of days ago. Rob, do you have any additional information to share?


Great, with your NSS+Python knowledge this should be walk in the park :-)
  Any luck with investigation of this issue?


Not yet, it's next on my to-do list.


--
John Dennis jden...@redhat.com

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] Proposal: Use argparse for option parsing

2012-10-15 Thread Petr Viktorin

On 10/12/2012 05:45 PM, Dmitri Pal wrote:

On 10/12/2012 11:04 AM, Petr Viktorin wrote:

Hello,
I read through https://fedorahosted.org/freeipa/ticket/3060 and
noticed that the requirements are similar to what an existing library
offers.

The optparse module we currently use for CLI option parsing is
deprecated[1]. The replacement is called argparse, which is part of
the Python 2.7 standard library. Out of the box, it provides several
features we either lack or had to handle manually, namely:
* supporting sub-commands [2]
* producing more informative usage messages [2]
* handling both optional and positional arguments [3]
* as well as including a number of other more minor improvements on
the optparse API. [3]

The package has also been backported to Python 2.6, the only catch
there is that it's not in the standard library.


Not much of our code calls optparse directly, so switching shouldn't
be a major effort (within the scope of reorganizing the help system).
In exchange we get better CLI help messages, forward compatibility,
and reuse of proven code instead of our own solutions.


[1] http://docs.python.org/library/optparse.html
[2] http://code.google.com/p/argparse/
[3] yum info python-argparse (EPEL-6)


The ticket is IMO nice to have. Does it provide any value for other main
commands?


Yes, it is nice to have. I thought that only affects priority, not what 
technical solution is correct.

I don't think there's value for other commands than help.


If so can you reference other tickets that it would help with.
I am not sure I want the switch for just help.


The help improvement ticket describes describes sane behavior for a 
subcommand-based CLI, which is what argparse was written for. We have 
our own implementation of it, which is subtly wrong, so we can either 
continue maintaining it or switch to reusing existing, tested code.
I think using the library is better in the long run. (And as a 
nice-to-have ticket, this is for the long run, right?)


--
PetrĀ³

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH 0079] Update NEWS file for 2.0 release

2012-10-15 Thread Petr Spacek

On 10/15/2012 02:57 PM, Adam Tkac wrote:

On Mon, Oct 15, 2012 at 10:38:41AM +0200, Petr Spacek wrote:

Hello,

Update NEWS file for 2.0 release


Hi Peter,

are you OK with this version of NEWS? (patch attached)


Sure, pushed to master: b0ceb4aaaba7452f92f91acb46a90eb9509ee72e

Petr^2 Spacek

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


[Freeipa-devel] [PATCH] 0092 Remove bogus check for smbpasswd

2012-10-15 Thread Alexander Bokovoy

Hi!

We don't use smbpasswd in adtrustinstance anymore so the check is
bogus.

One-liner.

--
/ Alexander Bokovoy
From 687f448a4b7d12ddb356f8e2a35a93fe9611b7cb Mon Sep 17 00:00:00 2001
From: Alexander Bokovoy aboko...@redhat.com
Date: Mon, 15 Oct 2012 16:01:26 +0300
Subject: [PATCH] Remove bogus check for smbpasswd

We don't use smbpasswd when configuring IPA for AD trusts anymore
because we switched to use Kerberos authentication in IPA passdb
backend based on CIFS service keytab.
---
 ipaserver/install/adtrustinstance.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ipaserver/install/adtrustinstance.py 
b/ipaserver/install/adtrustinstance.py
index 
b74f4b685b1cb56bdcdbae15ab6a8f38960bfc66..fcf2a73920567bde26c0df3b4797d727d241b36c
 100644
--- a/ipaserver/install/adtrustinstance.py
+++ b/ipaserver/install/adtrustinstance.py
@@ -60,7 +60,7 @@ and re-run ipa-adtrust-instal again afterwards.
 
 
 def check_inst():
-for smbfile in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']:
+for smbfile in ['/usr/sbin/smbd', '/usr/bin/net']:
 if not os.path.exists(smbfile):
 print %s was not found on this system % file
 print Please install the 'samba' packages and  \
-- 
1.7.12

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Re: [Freeipa-devel] Proposal: Use argparse for option parsing

2012-10-15 Thread Dmitri Pal
On 10/15/2012 09:05 AM, Petr Viktorin wrote:
 On 10/12/2012 05:45 PM, Dmitri Pal wrote:
 On 10/12/2012 11:04 AM, Petr Viktorin wrote:
 Hello,
 I read through https://fedorahosted.org/freeipa/ticket/3060 and
 noticed that the requirements are similar to what an existing library
 offers.

 The optparse module we currently use for CLI option parsing is
 deprecated[1]. The replacement is called argparse, which is part of
 the Python 2.7 standard library. Out of the box, it provides several
 features we either lack or had to handle manually, namely:
 * supporting sub-commands [2]
 * producing more informative usage messages [2]
 * handling both optional and positional arguments [3]
 * as well as including a number of other more minor improvements on
 the optparse API. [3]

 The package has also been backported to Python 2.6, the only catch
 there is that it's not in the standard library.


 Not much of our code calls optparse directly, so switching shouldn't
 be a major effort (within the scope of reorganizing the help system).
 In exchange we get better CLI help messages, forward compatibility,
 and reuse of proven code instead of our own solutions.


 [1] http://docs.python.org/library/optparse.html
 [2] http://code.google.com/p/argparse/
 [3] yum info python-argparse (EPEL-6)

 The ticket is IMO nice to have. Does it provide any value for other main
 commands?

 Yes, it is nice to have. I thought that only affects priority, not
 what technical solution is correct.
 I don't think there's value for other commands than help.

 If so can you reference other tickets that it would help with.
 I am not sure I want the switch for just help.

 The help improvement ticket describes describes sane behavior for a
 subcommand-based CLI, which is what argparse was written for. We have
 our own implementation of it, which is subtly wrong, so we can either
 continue maintaining it or switch to reusing existing, tested code.
 I think using the library is better in the long run. (And as a
 nice-to-have ticket, this is for the long run, right?)

Since there is not value for other commands I do not see an urgency to
switch now. The homegrown solution might be ugly but it works so
replacing it would not be a high priority. 3.2 is already pretty full
and I do not see this work getting in.

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager for IdM portfolio
Red Hat Inc.


---
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/



___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 0092 Remove bogus check for smbpasswd

2012-10-15 Thread Sumit Bose
On Mon, Oct 15, 2012 at 04:10:45PM +0300, Alexander Bokovoy wrote:
 Hi!
 
 We don't use smbpasswd in adtrustinstance anymore so the check is
 bogus.
 
 One-liner.
 
 -- 
 / Alexander Bokovoy

ACK

bye,
Sumit

 From 687f448a4b7d12ddb356f8e2a35a93fe9611b7cb Mon Sep 17 00:00:00 2001
 From: Alexander Bokovoy aboko...@redhat.com
 Date: Mon, 15 Oct 2012 16:01:26 +0300
 Subject: [PATCH] Remove bogus check for smbpasswd
 
 We don't use smbpasswd when configuring IPA for AD trusts anymore
 because we switched to use Kerberos authentication in IPA passdb
 backend based on CIFS service keytab.
 ---
  ipaserver/install/adtrustinstance.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/ipaserver/install/adtrustinstance.py 
 b/ipaserver/install/adtrustinstance.py
 index 
 b74f4b685b1cb56bdcdbae15ab6a8f38960bfc66..fcf2a73920567bde26c0df3b4797d727d241b36c
  100644
 --- a/ipaserver/install/adtrustinstance.py
 +++ b/ipaserver/install/adtrustinstance.py
 @@ -60,7 +60,7 @@ and re-run ipa-adtrust-instal again afterwards.
  
  
  def check_inst():
 -for smbfile in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']:
 +for smbfile in ['/usr/sbin/smbd', '/usr/bin/net']:
  if not os.path.exists(smbfile):
  print %s was not found on this system % file
  print Please install the 'samba' packages and  \
 -- 
 1.7.12
 

 ___
 Freeipa-devel mailing list
 Freeipa-devel@redhat.com
 https://www.redhat.com/mailman/listinfo/freeipa-devel

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 0092 Remove bogus check for smbpasswd

2012-10-15 Thread Rob Crittenden

Sumit Bose wrote:

On Mon, Oct 15, 2012 at 04:10:45PM +0300, Alexander Bokovoy wrote:

Hi!

We don't use smbpasswd in adtrustinstance anymore so the check is
bogus.

One-liner.

--
/ Alexander Bokovoy


ACK


NACK. Please fix the error message too. It should be using the variable 
smbfile and not file.


rob



bye,
Sumit


From 687f448a4b7d12ddb356f8e2a35a93fe9611b7cb Mon Sep 17 00:00:00 2001
From: Alexander Bokovoy aboko...@redhat.com
Date: Mon, 15 Oct 2012 16:01:26 +0300
Subject: [PATCH] Remove bogus check for smbpasswd

We don't use smbpasswd when configuring IPA for AD trusts anymore
because we switched to use Kerberos authentication in IPA passdb
backend based on CIFS service keytab.
---
  ipaserver/install/adtrustinstance.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ipaserver/install/adtrustinstance.py 
b/ipaserver/install/adtrustinstance.py
index 
b74f4b685b1cb56bdcdbae15ab6a8f38960bfc66..fcf2a73920567bde26c0df3b4797d727d241b36c
 100644
--- a/ipaserver/install/adtrustinstance.py
+++ b/ipaserver/install/adtrustinstance.py
@@ -60,7 +60,7 @@ and re-run ipa-adtrust-instal again afterwards.
  

  def check_inst():
-for smbfile in ['/usr/sbin/smbd', '/usr/bin/net', '/usr/bin/smbpasswd']:
+for smbfile in ['/usr/sbin/smbd', '/usr/bin/net']:
  if not os.path.exists(smbfile):
  print %s was not found on this system % file
  print Please install the 'samba' packages and  \
--
1.7.12




___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel



___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel