Re: [Freeipa-devel] [PATCHES] 0789-0796 Python 3 fixes for the server part

2016-05-30 Thread Martin Basti 



On 20.05.2016 17:04, Petr Viktorin wrote:

Hello,
Here are some more Python3 patches. Most are pretty routine, but pay
special attention to the first and last patch.


With these patches, running the in-tree test suite gives me the same
errors in Python 2 and Python 3, except:
- test_install – failures in the updater that I haven't investigated yet
- test_ipaserver – test bug (relying on order of values in an LDAP
attribute) and a text/bytes issue in certificate parsing


In the next few months, I'll need to focus less on IPA and more on
Samba, which is a prerequisite for porting the IPA server. So I'll
quickly summarize the current state of the porting effort:

All of FreeIPA's dependencies except Samba are ported to Python 3 (and
packaged in Fedora).
A recent change switched the IPA client to running on Python 3. With the
patches I'm sending now, most of the "single machine" tests are passing.
The install scripts will still need some work, as will the server parts
that aren't shared with the client.


I'd like to ask the IPA team to sometimes take a look at the Python 3
tests, and try to avoid too many regressions.





ACK, pushed

master:
* 36094b2a542a9472506034dc6c86a573e95c71de ipaldap: Keep attribute names 
as text, not bytes
* 75d0a73bbc426e9b6cabc38f2e932d701b25a762 ipapython.secrets.kem: Use 
ConfigParser from six.moves
* 9477cddfeb9adc03e039155ce3b8a0b560f71098 test_topology_plugin: Don't 
rely on order of an attribute's values
* 9ca450ac436344afd3a46d9852d8329a2e9a48d2 test_rpcserver: Expect 
updated error message under Python 3
* 743828b0f47ca4934125cfb8dc57f79283b95a4d ipaplatform.redhat: Use 
bytestrings when calling rpm.so for version comparison
* 25560f0e1db0c432de35a2496d9c1f8e63dc3dfd test_ipaserver.test_ldap: Use 
bytestrings for raw LDAP values
* c192c1ae3e080597995b906d93cc7607ffc605c0 ipaldap: Convert dict items 
to list before iterating
* 037eae26d0cd8467d3a559bb4cc585c61b626734 test_ipaserver.test_ldap: 
Adjust tests to Python 3's KeyView

ipa-4-3:
* f01b5e506c720181f56e54aa89925a4720642d83 ipaldap: Keep attribute names 
as text, not bytes
* 546b1d0fe6cefd2cbaa3d56f0b1d2939fbdff291 ipapython.secrets.kem: Use 
ConfigParser from six.moves
* 937ebf43745dff7c9894954262bab2d71d9afc71 test_topology_plugin: Don't 
rely on order of an attribute's values
* 74e3fd1d4ae4067caf0d4bda9e962e7c5ceb821c test_rpcserver: Expect 
updated error message under Python 3
* 12e73c95ccd26b5d8156abfddd59ddaca0689a6b ipaplatform.redhat: Use 
bytestrings when calling rpm.so for version comparison
* 3c610bee162f6b420ae91592d90303b662249b7e test_ipaserver.test_ldap: Use 
bytestrings for raw LDAP values
* a78c350589733585b78d6cd7ece63276c2b57634 ipaldap: Convert dict items 
to list before iterating
* 1933e604fb0822bc08caa4aec499be949f731d3b test_ipaserver.test_ldap: 
Adjust tests to Python 3's KeyView


-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCHES] 0789-0796 Python 3 fixes for the server part

2016-05-20 Thread Petr Viktorin 
Hello,
Here are some more Python3 patches. Most are pretty routine, but pay
special attention to the first and last patch.


With these patches, running the in-tree test suite gives me the same
errors in Python 2 and Python 3, except:
- test_install – failures in the updater that I haven't investigated yet
- test_ipaserver – test bug (relying on order of values in an LDAP
attribute) and a text/bytes issue in certificate parsing


In the next few months, I'll need to focus less on IPA and more on
Samba, which is a prerequisite for porting the IPA server. So I'll
quickly summarize the current state of the porting effort:

All of FreeIPA's dependencies except Samba are ported to Python 3 (and
packaged in Fedora).
A recent change switched the IPA client to running on Python 3. With the
patches I'm sending now, most of the "single machine" tests are passing.
The install scripts will still need some work, as will the server parts
that aren't shared with the client.


I'd like to ask the IPA team to sometimes take a look at the Python 3
tests, and try to avoid too many regressions.


-- 
Petr Viktorin
From 3a645b39f2ab80e60dd2dbcd5f6f4e39946d4450 Mon Sep 17 00:00:00 2001
From: Petr Viktorin 
Date: Fri, 29 Apr 2016 17:13:08 +0200
Subject: [PATCH] ipaldap: Keep attribute names as text, not bytes

Part of the work for: https://fedorahosted.org/freeipa/ticket/4985
---
 ipapython/ipaldap.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ipapython/ipaldap.py b/ipapython/ipaldap.py
index 7e920e1003d84b729547d75d8e1324d2f73c120c..74add512adb6af092e9220c0ca2164c95caae262 100644
--- a/ipapython/ipaldap.py
+++ b/ipapython/ipaldap.py
@@ -813,7 +813,7 @@ class LDAPClient(object):
 If there is a problem loading the schema or the attribute is
 not in the schema return None
 """
-if isinstance(name_or_oid, unicode):
+if six.PY2 and isinstance(name_or_oid, unicode):
 name_or_oid = name_or_oid.encode('utf-8')
 
 if name_or_oid in self._SINGLE_VALUE_OVERRIDE:
@@ -1516,7 +1516,7 @@ class LDAPClient(object):
 
 # pass arguments to python-ldap
 with self.error_handler():
-modlist = [(a, self.encode(b), self.encode(c))
+modlist = [(a, str(b), self.encode(c))
for a, b, c in modlist]
 self.conn.modify_s(str(entry.dn), modlist)
 
-- 
2.5.5

From 85a4c73ca57e1f3ba64bdb76c386b6a9c32f9d36 Mon Sep 17 00:00:00 2001
From: Petr Viktorin 
Date: Fri, 6 May 2016 17:08:18 +0200
Subject: [PATCH] ipapython.secrets.kem: Use ConfigParser from six.moves

In Python 3, the module name changed from 'ConfigParser' to
'configparser'. Use the appropriate location from six.

Part of the work for: https://fedorahosted.org/freeipa/ticket/4985
---
 ipapython/secrets/kem.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ipapython/secrets/kem.py b/ipapython/secrets/kem.py
index 1025ed7980f055c82c602634e8845fa490cf0514..0abf28ae4403a7b6225404df361d12cb07ccc70b 100644
--- a/ipapython/secrets/kem.py
+++ b/ipapython/secrets/kem.py
@@ -2,7 +2,7 @@
 
 from __future__ import print_function
 from ipaplatform.paths import paths
-import ConfigParser
+from six.moves.configparser import ConfigParser
 from cryptography.hazmat.backends import default_backend
 from cryptography.hazmat.primitives import serialization
 from cryptography.hazmat.primitives.asymmetric import rsa, ec
@@ -154,7 +154,7 @@ class IPAKEMKeys(KEMKeysStore):
 
 def __init__(self, config=None, ipaconf=paths.IPA_DEFAULT_CONF):
 super(IPAKEMKeys, self).__init__(config)
-conf = ConfigParser.ConfigParser()
+conf = ConfigParser()
 conf.read(ipaconf)
 self.host = conf.get('global', 'host')
 self.realm = conf.get('global', 'realm')
-- 
2.5.5

From f5d3da6a63a41b626bc9c1b4ac7f6603e220d6ff Mon Sep 17 00:00:00 2001
From: Petr Viktorin 
Date: Fri, 6 May 2016 17:35:26 +0200
Subject: [PATCH] test_topology_plugin: Don't rely on order of an attribute's
 values

Order of Python dicts/sets was always unreliable, but in Python 3
it's usually different every time. This affects the order in which
values of a LDAP attribute appear.
LDAP values are also specified to be unordered.

Part of the work for: https://fedorahosted.org/freeipa/ticket/4985
---
 ipatests/test_ipaserver/test_topology_plugin.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ipatests/test_ipaserver/test_topology_plugin.py b/ipatests/test_ipaserver/test_topology_plugin.py
index 6407f3e6f4ef4ff0fe12de721749dd41a667885c..eaa7b01cb977e885a731e83d0a31597cf7a05168 100644
--- a/ipatests/test_ipaserver/test_topology_plugin.py
+++ b/ipatests/test_ipaserver/test_topology_plugin.py
@@ -72,4 +72,4 @@ class TestTopologyPlugin(object):
 entry = self.conn.get_entry(topoplugindn)
 assert(set(entry.keys()) == set(pluginattrs.keys()))
 for