Re: [Freeipa-devel] [PATCHES] 0772-0774 Python3 fixes in for client installation

2016-03-03 Thread Jan Cholasta 

On 2.3.2016 14:58, Petr Viktorin wrote:

On 03/01/2016 08:57 AM, Jan Cholasta wrote:

On 29.2.2016 17:14, Petr Viktorin wrote:

On 02/29/2016 08:12 AM, Jan Cholasta wrote:

Hi,

On 26.2.2016 17:14, Petr Viktorin wrote:

On 02/22/2016 12:37 PM, Petr Viktorin wrote:

Hello,
These fixes are needed for the "happy path" of ipa-client-install
--server on Python 3.


Hello,
Could someone please look at these patches?


Patch 0772:


It seems this bit:

+[cn] = lres.raw['cn']
+lrealms.append(cn.decode('utf-8'))

breaks ipa-client-install:

Traceback (most recent call last):
   File "/sbin/ipa-client-install", line 3099, in 
 sys.exit(main())
   File "/sbin/ipa-client-install", line 3080, in main
 rval = install(options, env, fstore, statestore)
   File "/sbin/ipa-client-install", line 2769, in install
 delete_persistent_client_session_data(host_principal)
   File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 145, in
delete_persistent_client_session_data
 kernel_keyring.del_key(keyname)
   File "/usr/lib/python2.7/site-packages/ipapython/kernel_keyring.py",
line 130, in del_key
 assert isinstance(key, str)
AssertionError

because realm is returned as unicode from ipadiscovery.


Ah, thhanks for the catch. This should fix it.





1) Instead of decoding when the type is bytes in get_ipa_basedn(), read
attribute values from 'entry.raw' rather than 'entry' and decode always.


Done.


2) get_ipa_basedn() is used only in ipadiscovery, perhaps we should move
it there?


I did that in an extra patch, mainly to make the diff more meaningful.


OK. The patch needs a rebase for ipa-4-3, there is a stray
get_ipa_basedn import in install/migration/migration.py.


Attaching patch 0775 for that, please apply it before the others.


Patch 0773: ACK, but the patch does not apply on ipa-4-3.


Could you try `git cherry-pick` from the master branch after applying to
master?


Patch 0774: ACK.


Thanks, ACK.

Pushed to:
master: 34db5759fa8733d875377f4b487d1bcfbf768b0e
ipa-4-3: 6b747431a8289d2fbe931eefbe0a56b36c595b7b

--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCHES] 0772-0774 Python3 fixes in for client installation

2016-02-29 Thread Jan Cholasta 

On 29.2.2016 17:14, Petr Viktorin wrote:

On 02/29/2016 08:12 AM, Jan Cholasta wrote:

Hi,

On 26.2.2016 17:14, Petr Viktorin wrote:

On 02/22/2016 12:37 PM, Petr Viktorin wrote:

Hello,
These fixes are needed for the "happy path" of ipa-client-install
--server on Python 3.


Hello,
Could someone please look at these patches?


Patch 0772:


It seems this bit:

+[cn] = lres.raw['cn']
+lrealms.append(cn.decode('utf-8'))

breaks ipa-client-install:

Traceback (most recent call last):
  File "/sbin/ipa-client-install", line 3099, in 
sys.exit(main())
  File "/sbin/ipa-client-install", line 3080, in main
rval = install(options, env, fstore, statestore)
  File "/sbin/ipa-client-install", line 2769, in install
delete_persistent_client_session_data(host_principal)
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 145, in 
delete_persistent_client_session_data

kernel_keyring.del_key(keyname)
  File "/usr/lib/python2.7/site-packages/ipapython/kernel_keyring.py", 
line 130, in del_key

assert isinstance(key, str)
AssertionError

because realm is returned as unicode from ipadiscovery.



1) Instead of decoding when the type is bytes in get_ipa_basedn(), read
attribute values from 'entry.raw' rather than 'entry' and decode always.


Done.


2) get_ipa_basedn() is used only in ipadiscovery, perhaps we should move
it there?


I did that in an extra patch, mainly to make the diff more meaningful.


OK. The patch needs a rebase for ipa-4-3, there is a stray 
get_ipa_basedn import in install/migration/migration.py.





Patch 0773: ACK, but the patch does not apply on ipa-4-3.


Patch 0774: ACK.



--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCHES] 0772-0774 Python3 fixes in for client installation

2016-02-28 Thread Jan Cholasta 

Hi,

On 26.2.2016 17:14, Petr Viktorin wrote:

On 02/22/2016 12:37 PM, Petr Viktorin wrote:

Hello,
These fixes are needed for the "happy path" of ipa-client-install
--server on Python 3.


Hello,
Could someone please look at these patches?


Patch 0772:

1) Instead of decoding when the type is bytes in get_ipa_basedn(), read 
attribute values from 'entry.raw' rather than 'entry' and decode always.



2) get_ipa_basedn() is used only in ipadiscovery, perhaps we should move 
it there?



Patch 0773: ACK, but the patch does not apply on ipa-4-3.


Patch 0774: ACK.


Honza

--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCHES] 0772-0774 Python3 fixes in for client installation

2016-02-26 Thread Petr Viktorin 
On 02/22/2016 12:37 PM, Petr Viktorin wrote:
> Hello,
> These fixes are needed for the "happy path" of ipa-client-install
> --server on Python 3.

Hello,
Could someone please look at these patches?



-- 
Petr Viktorin

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCHES] 0772-0774 Python3 fixes in for client installation

2016-02-22 Thread Petr Viktorin 
Hello,
These fixes are needed for the "happy path" of ipa-client-install
--server on Python 3.

-- 
Petr Viktorin
From d51ff822165f434ed7c87cde44f64b8f05e33797 Mon Sep 17 00:00:00 2001
From: Petr Viktorin 
Date: Fri, 19 Feb 2016 17:26:57 +0100
Subject: [PATCH] ipadiscovery: Decode to unicode in ipacheckldap()

ipacheckldap uses a schema-less connection with decode_attrs=False,
so bytes need to be decoded manually.
This was not a problem in Python2 where bytes and unicode could
be mixed freely.

Part of the work for https://fedorahosted.org/freeipa/ticket/5638
---
 ipaclient/ipadiscovery.py | 2 +-
 ipapython/ipautil.py  | 8 
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/ipaclient/ipadiscovery.py b/ipaclient/ipadiscovery.py
index 45a71e190e56d33d51d37f16ae61a7b4c28df521..e1a565bd4bf82c07bf2e52ddab1669209919249d 100644
--- a/ipaclient/ipadiscovery.py
+++ b/ipaclient/ipadiscovery.py
@@ -381,7 +381,7 @@ class IPADiscovery(object):
 
 for lres in lret:
 root_logger.debug("Found: %s", lres.dn)
-lrealms.append(lres.single_value['cn'])
+lrealms.append(lres.single_value['cn'].decode('utf-8'))
 
 if trealm:
 for r in lrealms:
diff --git a/ipapython/ipautil.py b/ipapython/ipautil.py
index 6d07e07a08260995fb78e2ac23c825f734a4cd09..1d2a213cd28ea18712df19770f89edc288de5ac8 100644
--- a/ipapython/ipautil.py
+++ b/ipapython/ipautil.py
@@ -1141,6 +1141,8 @@ def get_ipa_basedn(conn):
 None is returned if the suffix is not found
 
 :param conn: Bound LDAPClient that will be used for searching
+
+May be called with a connection with decode_attrs set to False
 """
 entry = conn.get_entry(
 DN(), attrs_list=['defaultnamingcontext', 'namingcontexts'])
@@ -1156,6 +1158,9 @@ def get_ipa_basedn(conn):
 contexts.remove(default)
 contexts.insert(0, default)
 for context in contexts:
+if isinstance(context, bytes):
+# connection's decode_attrs is turned off, decode manually
+context = context.decode('utf-8')
 root_logger.debug("Check if naming context '%s' is for IPA" % context)
 try:
 [entry] = conn.get_entries(
@@ -1165,6 +1170,9 @@ def get_ipa_basedn(conn):
   "check for IPA version")
 continue
 info = entry.single_value['info'].lower()
+if isinstance(info, bytes):
+# connection's decode_attrs is turned off, decode manually
+info = info.decode('utf-8')
 if info != IPA_BASEDN_INFO:
 root_logger.debug("Detected IPA server version (%s) did not match the client (%s)" \
 % (info, IPA_BASEDN_INFO))
-- 
2.5.0

From b066a348d859d244428f6a9c618450a44ff705ae Mon Sep 17 00:00:00 2001
From: Petr Viktorin 
Date: Fri, 19 Feb 2016 17:38:30 +0100
Subject: [PATCH] ipapython.sysrestore: Use str methods instead of functions
 from the string module

For historical reasons, the string module contained some functions
that mirror methods of the str type. These are eremoved in Python 3.

Use str methods instead.

Part of the work for https://fedorahosted.org/freeipa/ticket/5638
---
 ipapython/sysrestore.py | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ipapython/sysrestore.py b/ipapython/sysrestore.py
index c43948db536574453923bb69d49d6c9ed00f57dc..c1c2bce8764d6b343a9c14736dd969be2f975c17 100644
--- a/ipapython/sysrestore.py
+++ b/ipapython/sysrestore.py
@@ -28,7 +28,6 @@ import os.path
 import shutil
 from ipapython.ipa_log_manager import root_logger
 import random
-import string
 
 import six
 from six.moves.configparser import SafeConfigParser
@@ -133,7 +132,8 @@ class FileStore:
 
 stat = os.stat(path)
 
-self.files[filename] = string.join([str(stat.st_mode),str(stat.st_uid),str(stat.st_gid),path], ',')
+template = '{stat.st_mode},{stat.st_uid},{stat.st_gid},{path}'
+self.files[filename] = template.format(stat=stat, path=path)
 self.save()
 
 def has_file(self, path):
@@ -143,7 +143,7 @@ class FileStore:
 """
 result = False
 for (key, value) in self.files.items():
-(mode,uid,gid,filepath) = string.split(value, ',', 3)
+(mode,uid,gid,filepath) = value.split(',', 3)
 if (filepath == path):
 result = True
 break
@@ -176,7 +176,7 @@ class FileStore:
 filename = None
 
 for (key, value) in self.files.items():
-(mode,uid,gid,filepath) = string.split(value, ',', 3)
+(mode,uid,gid,filepath) = value.split(',', 3)
 if (filepath == path):
 filename = key
 break
@@ -218,7 +218,7 @@ class FileStore:
 
 for (filename, value) in self.files.items():
 
-(mode,uid,gid,path) = string.split(value, ',', 3)
+