Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
On 03/12/2013 03:34 PM, Petr Viktorin wrote: > On 03/12/2013 01:37 PM, Martin Kosek wrote: >> On 03/12/2013 10:10 AM, Petr Viktorin wrote: >>> On 03/11/2013 02:56 PM, Martin Kosek wrote: On 03/11/2013 01:48 PM, Jan Cholasta wrote: > On 11.3.2013 13:43, Petr Viktorin wrote: >> On 03/11/2013 01:13 PM, Jan Cholasta wrote: >>> On 8.3.2013 14:14, Petr Viktorin wrote: On 03/07/2013 05:42 PM, Jan Cholasta wrote: > Patch 191: > > The patch is missing the ipapython/ipaldap.py file. >>> >>> On 7.3.2013 18:29, Petr Viktorin wrote: >>> > It's there, it's just copied from ipaserver/ipaldap.py with a small >>> > change at the bottom. >>> >>> There is no sign of the file, except in the patch header and the patch >>> cannot be applied with git am nor with git apply. But perhaps I'm doing >>> something wrong. >> >> Attaching a re-formatted version of the patch. >> >> [...] >>> ACK. >>> >>> Honza >>> >> >> > > ACK for real. > > Honza > I would not want to rush this, I still see errors: 1) ipa-ldap-updater is broken: # ipa-ldap-updater --upgrade Upgrading IPA: [1/8]: stopping directory server [2/8]: saving configuration [3/8]: disabling listeners [4/8]: starting directory server [5/8]: upgrading server Upgrade failed with 'NameSpace' object has no attribute 'ldap2' [6/8]: stopping directory server [7/8]: restoring configuration [8/8]: starting directory server Done. IPA upgrade failed. >>> >>> Thanks for the catch! >>> >>> This is a symptom of the fact the plugins attach themselves to the default >>> API >>> object as soon as they're imported. >>> Before, ipaldap imported ldap2, so the ldap2 server plugin was magically >>> available whenever ipaldap was imported before. >>> Now, ldap2 needs to be imported explicitly if api.Backend.ldap2 needs to be >>> available. >>> 2) What's the purpose of this new error? +class DatabaseTimeout(DatabaseError): +""" +**4211** Raised when an LDAP call times out + +For example: + +>>> raise DatabaseTimeout() +Traceback (most recent call last): + ... +DatabaseTimeout: LDAP timeout +""" + +errno = 4211 +format = _('LDAP timeout') >>> >>> Thanks for this catch too, I mis-squashed the code to raise it. >>> It is not raised anywhere (as far as I can see). BTW I assume it is not related to errors.LimitsExceeded in any way, right? >>> >>> No, it's timeout in the client↔server communication rather than the LDAP >>> operation. It wraps ldap.TIMEOUT rather than ldap.TIMELIMIT_EXCEEDED. >>> 3) Client installation no longer works if the server has disabled anonymous authentication: # ipa-client-install Error checking LDAP: Inappropriate authentication: Anonymous access is not allowed. DNS discovery failed to determine your DNS domain Provide the domain name of your IPA server (ex: example.com): ^C >>> >>> I couldn't reproduce this. But I did find some misleading log messages in >>> this >>> case. It work well now. >>> 4) I suddenly cannot run some tests, looks like import loop: # ./make-test tests/test_xmlrpc/test_host_plugin.py /usr/bin/nosetests -v --with-doctest --doctest-tests --exclude=plugins tests/test_xmlrpc/test_host_plugin.py Failure: ImportError (cannot import name ipautil) ... ERROR == ERROR: Failure: ImportError (cannot import name ipautil) -- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/loader.py", line 390, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/site-packages/nose/importer.py", line 39, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/root/freeipa-master/tests/test_xmlrpc/test_host_plugin.py", line 27, in from ipapython import ipautil File "/root/freeipa-master/ipapython/ipautil.py", line 52, in from ipalib import errors File "/root/freeipa-master/ipalib/__init__.py", line 930, in api.finalize() File "/root/freeipa-master/ipalib/plugable.py", line 674, in finalize self.__do_if_not_done('load_plugins') File "/root/freeipa-master/ipalib/plugable.py", line 454, in __do_if_not_done getattr(self, name)() File "/root/freeipa-master/ipalib/plugable.py", line 613,
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
On 03/12/2013 10:10 AM, Petr Viktorin wrote: > On 03/11/2013 02:56 PM, Martin Kosek wrote: >> On 03/11/2013 01:48 PM, Jan Cholasta wrote: >>> On 11.3.2013 13:43, Petr Viktorin wrote: On 03/11/2013 01:13 PM, Jan Cholasta wrote: > On 8.3.2013 14:14, Petr Viktorin wrote: >> On 03/07/2013 05:42 PM, Jan Cholasta wrote: >>> Patch 191: >>> >>> The patch is missing the ipapython/ipaldap.py file. > > On 7.3.2013 18:29, Petr Viktorin wrote: > > It's there, it's just copied from ipaserver/ipaldap.py with a small > > change at the bottom. > > There is no sign of the file, except in the patch header and the patch > cannot be applied with git am nor with git apply. But perhaps I'm doing > something wrong. Attaching a re-formatted version of the patch. [...] > ACK. > > Honza > >>> >>> ACK for real. >>> >>> Honza >>> >> >> I would not want to rush this, I still see errors: >> >> 1) ipa-ldap-updater is broken: >> >> # ipa-ldap-updater --upgrade >> Upgrading IPA: >>[1/8]: stopping directory server >>[2/8]: saving configuration >>[3/8]: disabling listeners >>[4/8]: starting directory server >>[5/8]: upgrading server >> Upgrade failed with 'NameSpace' object has no attribute 'ldap2' >>[6/8]: stopping directory server >>[7/8]: restoring configuration >>[8/8]: starting directory server >> Done. >> IPA upgrade failed. > > Thanks for the catch! > > This is a symptom of the fact the plugins attach themselves to the default API > object as soon as they're imported. > Before, ipaldap imported ldap2, so the ldap2 server plugin was magically > available whenever ipaldap was imported before. > Now, ldap2 needs to be imported explicitly if api.Backend.ldap2 needs to be > available. > >> 2) What's the purpose of this new error? >> >> +class DatabaseTimeout(DatabaseError): >> +""" >> +**4211** Raised when an LDAP call times out >> + >> +For example: >> + >> +>>> raise DatabaseTimeout() >> +Traceback (most recent call last): >> + ... >> +DatabaseTimeout: LDAP timeout >> +""" >> + >> +errno = 4211 >> +format = _('LDAP timeout') > > Thanks for this catch too, I mis-squashed the code to raise it. > >> It is not raised anywhere (as far as I can see). BTW I assume it is not >> related to errors.LimitsExceeded in any way, right? > > No, it's timeout in the client↔server communication rather than the LDAP > operation. It wraps ldap.TIMEOUT rather than ldap.TIMELIMIT_EXCEEDED. > >> 3) Client installation no longer works if the server has disabled >> anonymous authentication: >> >> # ipa-client-install >> Error checking LDAP: Inappropriate authentication: Anonymous access is >> not allowed. >> DNS discovery failed to determine your DNS domain >> Provide the domain name of your IPA server (ex: example.com): ^C > > I couldn't reproduce this. But I did find some misleading log messages in this > case. It work well now. > >> 4) I suddenly cannot run some tests, looks like import loop: >> >> # ./make-test tests/test_xmlrpc/test_host_plugin.py >> /usr/bin/nosetests -v --with-doctest --doctest-tests --exclude=plugins >> tests/test_xmlrpc/test_host_plugin.py >> Failure: ImportError (cannot import name ipautil) ... ERROR >> >> == >> ERROR: Failure: ImportError (cannot import name ipautil) >> -- >> Traceback (most recent call last): >>File "/usr/lib/python2.7/site-packages/nose/loader.py", line 390, in >> loadTestsFromName >> addr.filename, addr.module) >>File "/usr/lib/python2.7/site-packages/nose/importer.py", line 39, in >> importFromPath >> return self.importFromDir(dir_path, fqname) >>File "/usr/lib/python2.7/site-packages/nose/importer.py", line 86, in >> importFromDir >> mod = load_module(part_fqname, fh, filename, desc) >>File "/root/freeipa-master/tests/test_xmlrpc/test_host_plugin.py", >> line 27, in >> from ipapython import ipautil >>File "/root/freeipa-master/ipapython/ipautil.py", line 52, in >> from ipalib import errors >>File "/root/freeipa-master/ipalib/__init__.py", line 930, in >> api.finalize() >>File "/root/freeipa-master/ipalib/plugable.py", line 674, in finalize >> self.__do_if_not_done('load_plugins') >>File "/root/freeipa-master/ipalib/plugable.py", line 454, in >> __do_if_not_done >> getattr(self, name)() >>File "/root/freeipa-master/ipalib/plugable.py", line 613, in >> load_plugins >> self.import_plugins('ipalib') >>File "/root/freeipa-master/ipalib/plugable.py", line 655, in >> import_plugins >> __import__(fullname) >>File "/root/freeipa-master/ipalib/plugins/cert.py", line 30, in >> from ipalib import pkcs10 >>File "/root/freeipa-master/ipalib/pkcs10.py", line 24, in >> from ipapython import ipautil >>
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
On 03/11/2013 01:48 PM, Jan Cholasta wrote: On 11.3.2013 13:43, Petr Viktorin wrote: On 03/11/2013 01:13 PM, Jan Cholasta wrote: On 8.3.2013 14:14, Petr Viktorin wrote: On 03/07/2013 05:42 PM, Jan Cholasta wrote: Patch 191: The patch is missing the ipapython/ipaldap.py file. On 7.3.2013 18:29, Petr Viktorin wrote: > It's there, it's just copied from ipaserver/ipaldap.py with a small > change at the bottom. There is no sign of the file, except in the patch header and the patch cannot be applied with git am nor with git apply. But perhaps I'm doing something wrong. Attaching a re-formatted version of the patch. [...] ACK. Honza ACK for real. Honza I would not want to rush this, I still see errors: 1) ipa-ldap-updater is broken: # ipa-ldap-updater --upgrade Upgrading IPA: [1/8]: stopping directory server [2/8]: saving configuration [3/8]: disabling listeners [4/8]: starting directory server [5/8]: upgrading server Upgrade failed with 'NameSpace' object has no attribute 'ldap2' [6/8]: stopping directory server [7/8]: restoring configuration [8/8]: starting directory server Done. IPA upgrade failed. 2) What's the purpose of this new error? +class DatabaseTimeout(DatabaseError): +""" +**4211** Raised when an LDAP call times out + +For example: + +>>> raise DatabaseTimeout() +Traceback (most recent call last): + ... +DatabaseTimeout: LDAP timeout +""" + +errno = 4211 +format = _('LDAP timeout') It is not raised anywhere (as far as I can see). BTW I assume it is not related to errors.LimitsExceeded in any way, right? 3) Client installation no longer works if the server has disabled anonymous authentication: # ipa-client-install Error checking LDAP: Inappropriate authentication: Anonymous access is not allowed. DNS discovery failed to determine your DNS domain Provide the domain name of your IPA server (ex: example.com): ^C 4) I suddenly cannot run some tests, looks like import loop: # ./make-test tests/test_xmlrpc/test_host_plugin.py /usr/bin/nosetests -v --with-doctest --doctest-tests --exclude=plugins tests/test_xmlrpc/test_host_plugin.py Failure: ImportError (cannot import name ipautil) ... ERROR == ERROR: Failure: ImportError (cannot import name ipautil) -- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/loader.py", line 390, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/site-packages/nose/importer.py", line 39, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/root/freeipa-master/tests/test_xmlrpc/test_host_plugin.py", line 27, in from ipapython import ipautil File "/root/freeipa-master/ipapython/ipautil.py", line 52, in from ipalib import errors File "/root/freeipa-master/ipalib/__init__.py", line 930, in api.finalize() File "/root/freeipa-master/ipalib/plugable.py", line 674, in finalize self.__do_if_not_done('load_plugins') File "/root/freeipa-master/ipalib/plugable.py", line 454, in __do_if_not_done getattr(self, name)() File "/root/freeipa-master/ipalib/plugable.py", line 613, in load_plugins self.import_plugins('ipalib') File "/root/freeipa-master/ipalib/plugable.py", line 655, in import_plugins __import__(fullname) File "/root/freeipa-master/ipalib/plugins/cert.py", line 30, in from ipalib import pkcs10 File "/root/freeipa-master/ipalib/pkcs10.py", line 24, in from ipapython import ipautil ImportError: cannot import name ipautil -- Ran 1 test in 0.002s FAILED (errors=1) == FAILED under '/usr/bin/python2.7' ** FAIL ** Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
On 11.3.2013 13:43, Petr Viktorin wrote: On 03/11/2013 01:13 PM, Jan Cholasta wrote: On 8.3.2013 14:14, Petr Viktorin wrote: On 03/07/2013 05:42 PM, Jan Cholasta wrote: Patch 191: The patch is missing the ipapython/ipaldap.py file. On 7.3.2013 18:29, Petr Viktorin wrote: > It's there, it's just copied from ipaserver/ipaldap.py with a small > change at the bottom. There is no sign of the file, except in the patch header and the patch cannot be applied with git am nor with git apply. But perhaps I'm doing something wrong. Attaching a re-formatted version of the patch. [...] ACK. Honza ACK for real. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
On 8.3.2013 14:14, Petr Viktorin wrote: On 03/07/2013 05:42 PM, Jan Cholasta wrote: Patch 191: The patch is missing the ipapython/ipaldap.py file. On 7.3.2013 18:29, Petr Viktorin wrote: > It's there, it's just copied from ipaserver/ipaldap.py with a small > change at the bottom. There is no sign of the file, except in the patch header and the patch cannot be applied with git am nor with git apply. But perhaps I'm doing something wrong. I think it should go into ipalib instead of ipapython. It doesn't make sense to keep ipapython and ipalib separate if they depend on each other. We should either merge them or clean up the mess by removing ipalib imports from ipapython. I'm not saying we should do it now, just please don't add new modules to ipapython which import from ipalib. This is a bigger problem. Conceptually ipaldap should be in ipapython, it just needs the problematic errors and text modules. I think we should move those rather than ipaldap. Moving test to ipapython makes sense. I think we should have a separate set of errors for ipaldap and translate them to framework errors in ipalib. Also I am not very fond of the "ipa" prefix in "ipaldap". The module lives in the namespace of our own package, so there's no need for it to have such a prefix, is there? It's nice to have a unique name for talking about it. Since "ldap" is already a package we use, having another "ldap" would be confusing, even if it is properly namespaced. Since ipaldap should be the sole user of python-ldap after the refactoring is done, I don't think there would be any confusion. But whatever, I'm fine with both. Patch 193: +scope=conn.SCOPE_BASE, +filter='objectclass=pkiCA', +attrs_list=[ca_cert_attr], Can we use a proper filter here please? +:param conn: Bound LDAPConnection that will be used for searching Fixed, thanks LDAPClient Patch 194: -ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, True) and -lh.set_option(ldap.OPT_X_TLS_DEMAND, True) Is removing these options safe? I re-added them. I also removed the forgotten debugging `raise`s Martin found. Adding patch 0196, which disables downloading the schema for discovery. Updated patches attached. They now depend on Honza's patches 116-119 ACK. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
Petr Viktorin wrote: On 03/07/2013 06:01 PM, Rob Crittenden wrote: Jan Cholasta wrote: On 6.3.2013 16:29, Petr Viktorin wrote: Hello, These patches move ipaldap to ipapython, and make the client installer use it. Also password migration web-app is made to use ipaldap; they both called a shared a utility function that is converted to use ipaldap. This should fix https://fedorahosted.org/freeipa/ticket/3446 (freeipa-client-install KeyError in 'namingcontexts') and similar errors. https://fedorahosted.org/freeipa/ticket/3487 Patch 191: The patch is missing the ipapython/ipaldap.py file. It's there, it's just copied from ipaserver/ipaldap.py with a small change at the bottom. I'll respond to the other comments tomorrow. I think it should go into ipalib instead of ipapython. It doesn't make sense to keep ipapython and ipalib separate if they depend on each other. We should either merge them or clean up the mess by removing ipalib imports from ipapython. I'm not saying we should do it now, just please don't add new modules to ipapython which import from ipalib. There have been platforms in the past that we do not ship the framework on (e.g. RHEL 5). ipapython is supposed to be platform agnostic, though it clearly isn't, to allow a bridge for those systems that can't or won't have the framework. So I think removing the imports from ipapython is the way to go. Then we need to move errors and text to ipapython, those need to be imported pretty much everywhere. Yup, that too is a bit of a violation, at least for errors as we really want to publish those as part of the framework. This is why I didn't raise a fuss when we starting doing minor mixing of things (and I really felt the pain when trying to do certain things in RHEL 5). The client is now using more and more from the framework (SSH uploads, for example). So maybe it is time to revisit how we package things. If we decide to go whole hog and merge ipapython and ipalib I think there'd be no going back. We'd have to carefully consider all the consequences. I tend to be rather conservative in this regard, but perhaps I'm just living in the past. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
On 03/07/2013 06:01 PM, Rob Crittenden wrote: Jan Cholasta wrote: On 6.3.2013 16:29, Petr Viktorin wrote: Hello, These patches move ipaldap to ipapython, and make the client installer use it. Also password migration web-app is made to use ipaldap; they both called a shared a utility function that is converted to use ipaldap. This should fix https://fedorahosted.org/freeipa/ticket/3446 (freeipa-client-install KeyError in 'namingcontexts') and similar errors. https://fedorahosted.org/freeipa/ticket/3487 Patch 191: The patch is missing the ipapython/ipaldap.py file. It's there, it's just copied from ipaserver/ipaldap.py with a small change at the bottom. I'll respond to the other comments tomorrow. I think it should go into ipalib instead of ipapython. It doesn't make sense to keep ipapython and ipalib separate if they depend on each other. We should either merge them or clean up the mess by removing ipalib imports from ipapython. I'm not saying we should do it now, just please don't add new modules to ipapython which import from ipalib. There have been platforms in the past that we do not ship the framework on (e.g. RHEL 5). ipapython is supposed to be platform agnostic, though it clearly isn't, to allow a bridge for those systems that can't or won't have the framework. So I think removing the imports from ipapython is the way to go. Then we need to move errors and text to ipapython, those need to be imported pretty much everywhere. -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
Jan Cholasta wrote: On 6.3.2013 16:29, Petr Viktorin wrote: Hello, These patches move ipaldap to ipapython, and make the client installer use it. Also password migration web-app is made to use ipaldap; they both called a shared a utility function that is converted to use ipaldap. This should fix https://fedorahosted.org/freeipa/ticket/3446 (freeipa-client-install KeyError in 'namingcontexts') and similar errors. https://fedorahosted.org/freeipa/ticket/3487 Patch 191: The patch is missing the ipapython/ipaldap.py file. I think it should go into ipalib instead of ipapython. It doesn't make sense to keep ipapython and ipalib separate if they depend on each other. We should either merge them or clean up the mess by removing ipalib imports from ipapython. I'm not saying we should do it now, just please don't add new modules to ipapython which import from ipalib. There have been platforms in the past that we do not ship the framework on (e.g. RHEL 5). ipapython is supposed to be platform agnostic, though it clearly isn't, to allow a bridge for those systems that can't or won't have the framework. So I think removing the imports from ipapython is the way to go. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
On 6.3.2013 16:29, Petr Viktorin wrote: Hello, These patches move ipaldap to ipapython, and make the client installer use it. Also password migration web-app is made to use ipaldap; they both called a shared a utility function that is converted to use ipaldap. This should fix https://fedorahosted.org/freeipa/ticket/3446 (freeipa-client-install KeyError in 'namingcontexts') and similar errors. https://fedorahosted.org/freeipa/ticket/3487 Patch 191: The patch is missing the ipapython/ipaldap.py file. I think it should go into ipalib instead of ipapython. It doesn't make sense to keep ipapython and ipalib separate if they depend on each other. We should either merge them or clean up the mess by removing ipalib imports from ipapython. I'm not saying we should do it now, just please don't add new modules to ipapython which import from ipalib. Also I am not very fond of the "ipa" prefix in "ipaldap". The module lives in the namespace of our own package, so there's no need for it to have such a prefix, is there? Patch 193: +scope=conn.SCOPE_BASE, +filter='objectclass=pkiCA', +attrs_list=[ca_cert_attr], Can we use a proper filter here please? +:param conn: Bound LDAPConnection that will be used for searching LDAPClient Patch 194: -ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, True) and -lh.set_option(ldap.OPT_X_TLS_DEMAND, True) Is removing these options safe? Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0191-0195 Use ipaldap in the client installer & password migration
On 03/06/2013 04:29 PM, Petr Viktorin wrote: > Hello, > These patches move ipaldap to ipapython, and make the client installer use it. > Also password migration web-app is made to use ipaldap; they both called a > shared a utility function that is converted to use ipaldap. > > This should fix https://fedorahosted.org/freeipa/ticket/3446 > (freeipa-client-install KeyError in 'namingcontexts') and similar errors. > > https://fedorahosted.org/freeipa/ticket/3487 > As we discussed, a prerequisite for these patches is that IPAdmin should be able to avoid fetching remote LDAP schema. We do not want to get schema of every LDAP server we try to discover on. I did not test the patches, I just saw some strange changes when reading the patches. 0193: @@ -1645,6 +1640,7 @@ def get_ca_cert(fstore, options, server, basedn): os.unlink(ca_file) raise except Exception, e: +raise root_logger.debug(str(e)) raise errors.NoCertificateError(entry=url) @@ -2018,6 +2014,7 @@ def install(options, env, fstore, statestore): del os.environ['KRB5_CONFIG'] except Exception, e: root_logger.error("Cannot obtain CA certificate\n%s", e) +raise return CLIENT_INSTALL_ERROR # Now join the domain What is it good for? Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel