On 05/06/2015 04:25 PM, Milan Kubik wrote:
On 02/19/2015 03:51 PM, Petr Vobornik wrote:
This patch is a prerequisite for patch 801 which will follow. It was
developed to enable to use ipalib RPC client in Web UI tests. Plus it
will enable to significantly speed up Web UI tests suite (if
preparation of data is transformed to use this method).

Partly related https://fedorahosted.org/freeipa/ticket/4772 and
https://fedorahosted.org/freeipa/ticket/4307


Leverage session support to enable forms-based authenticate in rpc
client.

In order to do that session support in KerbTransport was moved to new
SessionTransport. RPCClient.create_connection is then modified to
force forms-based auth if new optional options - user and password are
specified. For this case SessionTransport is used and user is
authenticated by calling
'https://ipa.server/ipa/session/login_password'. Session cookie is
stored and used in subsequent calls.

This feature is usable for use cases where one wants to call the API
without being on ipa client. Non-being on ipa client also means that
IPA's NSS database and configuration is not available. Therefore one
has to define "~/.ipa/default.conf" in a similar way as ipa client
does and prepare a NSS database with IPA CA cert.

Usage:

    api.Backend.rpcclient.connect(
        nss_dir=my_nss_dir_path,
        user=user,
        password=password
    )

It's possible to switch users with:

    api.Backend.rpcclient.disconnect()

    api.Backend.rpcclient.connect(
        nss_dir=my_nss_dir_path,
        user=other_user,
        password=other_password
    )

Or check connection with:

    api.Backend.rpcclient.isconnected()

Example: download a CA cert and add it to a new temporary NSS database:
    from urllib2 import urlparse
    from ipaplatform.paths import paths
    from ipapython import certdb, ipautil
    from ipapython.ipautil import run
    from ipalib import x509

    # create new NSSDatabase
    tmp_db = certdb.NSSDatabase()
    pwd_file = ipautil.write_tmp_file(ipautil.ipa_generate_password())
    tmp_db.create_db(pwd_file.name)

    # download and add cert
    url = urlparse.urlunparse(('http', ipautil.format_netloc(ipa_server),
                               '/ipa/config/ca.crt', '', '', ''))
    stdout, stderr, rc = run([paths.BIN_WGET, "-O", "-", url])
    certs = x509.load_certificate_list(stdout, tmp_db.secdir)
    ca_certs = [cert.der_data for cert in certs]
    for i, cert in enumerate(ca_certs):
        tmp_db.add_cert(cert, 'CA certificate %d' % (i + 1), 'C,,')

    my_nss_dir_path = tmp_db.secdir



thanks for the patch. Please, fix the pep8 complaints.

I've fixed existing E128 error in imports. But the remaining:

$ git diff HEAD~1 -U0 | pep8 --diff
./ipalib/rpc.py:518:80: E501 line too long (86 > 79 characters)
./ipalib/rpc.py:524:80: E501 line too long (84 > 79 characters)
./ipalib/rpc.py:609:80: E501 line too long (80 > 79 characters)
./ipalib/rpc.py:634:80: E501 line too long (82 > 79 characters)
./ipalib/rpc.py:641:80: E501 line too long (94 > 79 characters)
./ipalib/rpc.py:796:80: E501 line too long (80 > 79 characters)
./ipalib/rpc.py:800:80: E501 line too long (82 > 79 characters)
./ipalib/rpc.py:913:80: E501 line too long (84 > 79 characters)

I won't fix. Since it's just E501 in moved code.


Can someone else look at the code as well, please?

Thanks,
Milan

--
Petr Vobornik
From 39ba81ccbe184c50d9f3a55a076f50396a6e66cf Mon Sep 17 00:00:00 2001
From: Petr Vobornik <pvobo...@redhat.com>
Date: Wed, 10 Dec 2014 18:57:51 +0100
Subject: [PATCH] rpc-client: add forms based auth support

Leverage session support to enable forms-based authenticate in rpc client.

In order to do that session support in KerbTransport was moved to new
SessionTransport. RPCClient.create_connection is then modified to
force forms-based auth if new optional options - user and password are
specified. For this case SessionTransport is used and user is
authenticated by calling
'https://ipa.server/ipa/session/login_password'. Session cookie is
stored and used in subsequent calls.

This feature is usable for use cases where one wants to call the API
without being on ipa client. Non-being on ipa client also means that
IPA's NSS database and configuration is not available. Therefore one
has to define "~/.ipa/default.conf" in a similar way as ipa client
does and prepare a NSS database with IPA CA cert.

Usage:

    api.Backend.rpcclient.connect(
        nss_dir=my_nss_dir_path,
        user=user,
        password=password
    )

It's possible to switch users with:

    api.Backend.rpcclient.disconnect()

    api.Backend.rpcclient.connect(
        nss_dir=my_nss_dir_path,
        user=other_user,
        password=other_password
    )

Or check connection with:

    api.Backend.rpcclient.isconnected()

Example: download a CA cert and add it to a new temporary NSS database:
    from urllib2 import urlparse
    from ipaplatform.paths import paths
    from ipapython import certdb, ipautil
    from ipapython.ipautil import run
    from ipalib import x509

    # create new NSSDatabase
    tmp_db = certdb.NSSDatabase()
    pwd_file = ipautil.write_tmp_file(ipautil.ipa_generate_password())
    tmp_db.create_db(pwd_file.name)

    # download and add cert
    url = urlparse.urlunparse(('http', ipautil.format_netloc(ipa_server),
                               '/ipa/config/ca.crt', '', '', ''))
    stdout, stderr, rc = run([paths.BIN_WGET, "-O", "-", url])
    certs = x509.load_certificate_list(stdout, tmp_db.secdir)
    ca_certs = [cert.der_data for cert in certs]
    for i, cert in enumerate(ca_certs):
        tmp_db.add_cert(cert, 'CA certificate %d' % (i + 1), 'C,,')

    my_nss_dir_path = tmp_db.secdir
---
 ipalib/rpc.py | 315 ++++++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 205 insertions(+), 110 deletions(-)

diff --git a/ipalib/rpc.py b/ipalib/rpc.py
index 6a1c983c0419cf314e96c697a556825d679961c9..7fd3276c8fe0db8c8b5a5c2648fb05ac6eb0b2c3 100644
--- a/ipalib/rpc.py
+++ b/ipalib/rpc.py
@@ -38,9 +38,11 @@ import os
 import locale
 import base64
 import urllib
+import urllib2
 import json
 import socket
-from urllib2 import urlparse
+from urllib2 import urlparse, HTTPError
+import cookielib
 
 from xmlrpclib import (Binary, Fault, DateTime, dumps, loads, ServerProxy,
         Transport, ProtocolError, MININT, MAXINT)
@@ -51,8 +53,9 @@ from nss.error import NSPRError
 
 from ipalib.backend import Connectible
 from ipalib.constants import LDAP_GENERALIZED_TIME_FORMAT
-from ipalib.errors import (public_errors, UnknownError, NetworkError,
-    KerberosError, XMLRPCMarshallError, JSONError, ConversionError)
+from ipalib.errors import (
+    public_errors, UnknownError, NetworkError, KerberosError,
+    XMLRPCMarshallError, JSONError, ConversionError, InvalidSessionPassword)
 from ipalib import errors, capabilities
 from ipalib.request import context, Connection
 from ipalib.util import get_current_principal
@@ -506,7 +509,76 @@ class SSLTransport(LanguageAwareTransport):
             return self._connection[1]
 
 
-class KerbTransport(SSLTransport):
+class SessionTransport(SSLTransport):
+
+    def get_host_info(self, host):
+        """
+        Adds session cookie
+        """
+        (host, extra_headers, x509) = LanguageAwareTransport.get_host_info(self, host)
+
+        if not isinstance(extra_headers, list):
+            extra_headers = []
+
+        session_cookie = getattr(context, 'session_cookie', None)
+        root_logger.debug("SessionTransport, using session: " + str(session_cookie))
+        if session_cookie:
+            extra_headers.append(('Cookie', session_cookie))
+        return (host, extra_headers, x509)
+
+    def store_session_cookie(self, cookie_header):
+        '''
+        Given the contents of a Set-Cookie header scan the header and
+        extract each cookie contained within until the session cookie
+        is located. Examine the session cookie if the domain and path
+        are specified, if not update the cookie with those values from
+        the request URL. Then write the session cookie into the key
+        store for the principal. If the cookie header is None or the
+        session cookie is not present in the header no action is
+        taken.
+
+        Context Dependencies:
+
+        The per thread context is expected to contain:
+            principal
+                The current pricipal the HTTP request was issued for.
+            request_url
+                The URL of the HTTP request.
+
+        '''
+
+        if cookie_header is None:
+            return
+
+        principal = getattr(context, 'principal', None)
+        request_url = getattr(context, 'request_url', None)
+        root_logger.debug("received Set-Cookie '%s'", cookie_header)
+
+        # Search for the session cookie
+        try:
+            session_cookie = Cookie.get_named_cookie_from_string(cookie_header,
+                                                                 COOKIE_NAME, request_url)
+        except Exception, e:
+            root_logger.error("unable to parse cookie header '%s': %s", cookie_header, e)
+            return
+
+        if session_cookie is None:
+            return
+
+        cookie_string = str(session_cookie)
+        root_logger.debug("storing cookie '%s' for principal %s", cookie_string, principal)
+        try:
+            update_persistent_client_session_data(principal, cookie_string)
+        except Exception, e:
+            # Not fatal, we just can't use the session cookie we were sent.
+            pass
+
+    def parse_response(self, response):
+        self.store_session_cookie(response.getheader('Set-Cookie'))
+        return SSLTransport.parse_response(self, response)
+
+
+class KerbTransport(SessionTransport):
     """
     Handles Kerberos Negotiation authentication to an XML-RPC server.
     """
@@ -534,14 +606,10 @@ class KerbTransport(SSLTransport):
         Two things can happen here. If we have a session we will add
         a cookie for that. If not we will set an Authorization header.
         """
-        (host, extra_headers, x509) = SSLTransport.get_host_info(self, host)
-
-        if not isinstance(extra_headers, list):
-            extra_headers = []
+        (host, extra_headers, x509) = SessionTransport.get_host_info(self, host)
 
         session_cookie = getattr(context, 'session_cookie', None)
         if session_cookie:
-            extra_headers.append(('Cookie', session_cookie))
             return (host, extra_headers, x509)
 
         # Set the remote host principal
@@ -570,61 +638,10 @@ class KerbTransport(SSLTransport):
 
     def single_request(self, host, handler, request_body, verbose=0):
         try:
-            return SSLTransport.single_request(self, host, handler, request_body, verbose)
+            return SessionTransport.single_request(self, host, handler, request_body, verbose)
         finally:
             self.close()
 
-    def store_session_cookie(self, cookie_header):
-        '''
-        Given the contents of a Set-Cookie header scan the header and
-        extract each cookie contained within until the session cookie
-        is located. Examine the session cookie if the domain and path
-        are specified, if not update the cookie with those values from
-        the request URL. Then write the session cookie into the key
-        store for the principal. If the cookie header is None or the
-        session cookie is not present in the header no action is
-        taken.
-
-        Context Dependencies:
-
-        The per thread context is expected to contain:
-            principal
-                The current pricipal the HTTP request was issued for.
-            request_url
-                The URL of the HTTP request.
-
-        '''
-
-        if cookie_header is None:
-            return
-
-        principal = getattr(context, 'principal', None)
-        request_url = getattr(context, 'request_url', None)
-        root_logger.debug("received Set-Cookie '%s'", cookie_header)
-
-        # Search for the session cookie
-        try:
-            session_cookie = Cookie.get_named_cookie_from_string(cookie_header,
-                                                                 COOKIE_NAME, request_url)
-        except Exception, e:
-            root_logger.error("unable to parse cookie header '%s': %s", cookie_header, e)
-            return
-
-        if session_cookie is None:
-            return
-
-        cookie_string = str(session_cookie)
-        root_logger.debug("storing cookie '%s' for principal %s", cookie_string, principal)
-        try:
-            update_persistent_client_session_data(principal, cookie_string)
-        except Exception, e:
-            # Not fatal, we just can't use the session cookie we were sent.
-            pass
-
-    def parse_response(self, response):
-        self.store_session_cookie(response.getheader('Set-Cookie'))
-        return SSLTransport.parse_response(self, response)
-
 
 class DelegatedKerbTransport(KerbTransport):
     """
@@ -762,24 +779,46 @@ class RPCClient(Connectible):
         setattr(context, 'session_cookie', session_cookie.http_cookie())
 
         # Form the session URL by substituting the session path into the original URL
-        scheme, netloc, path, params, query, fragment = urlparse.urlparse(original_url)
+        return self.get_session_url(original_url)
+
+    def get_session_url(self, url):
+        scheme, netloc, path, params, query, fragment = urlparse.urlparse(url)
         path = self.session_path
         session_url = urlparse.urlunparse((scheme, netloc, path, params, query, fragment))
-
         return session_url
 
+    def destroy_session(self, principal):
+        if hasattr(context, 'session_cookie'):
+            delattr(context, 'session_cookie')
+            try:
+                delete_persistent_client_session_data(principal)
+            except Exception, e:
+                # This shouldn't happen if we have a session but it isn't fatal.
+                pass
+
     def create_connection(self, ccache=None, verbose=0, fallback=True,
-                          delegate=False, nss_dir=None):
-        try:
-            rpc_uri = self.env[self.env_rpc_uri_key]
+                          delegate=False, nss_dir=None, user=None, password=None):
+        """
+        Create connection
+        """
+
+        setattr(context, 'user', user)
+        setattr(context, 'password', password)
+        use_krb = not user or not password
+
+        rpc_uri = self.env[self.env_rpc_uri_key]
+        if use_krb:
             principal = get_current_principal()
-            setattr(context, 'principal', principal)
+        else:
+            principal = user
+        setattr(context, 'principal', principal)
+        try:
             # We have a session cookie, try using the session URI to see if it
             # is still valid
             if not delegate:
                 rpc_uri = self.apply_session_cookie(rpc_uri)
         except ValueError:
-            # No session key, do full Kerberos auth
+            # No session key, do full auth
             pass
         # This might be dangerous. Use at your own risk!
         if nss_dir:
@@ -790,7 +829,9 @@ class RPCClient(Connectible):
             kw = dict(allow_none=True, encoding='UTF-8')
             kw['verbose'] = verbose
             if url.startswith('https://'):
-                if delegate:
+                if not use_krb:
+                    transport_class = SessionTransport
+                elif delegate:
                     transport_class = DelegatedKerbTransport
                 else:
                     transport_class = KerbTransport
@@ -798,46 +839,41 @@ class RPCClient(Connectible):
                 transport_class = LanguageAwareTransport
             kw['transport'] = transport_class(protocol=self.protocol)
             self.log.info('trying %s' % url)
-            setattr(context, 'request_url', url)
-            serverproxy = self.server_proxy_class(url, **kw)
-            if len(urls) == 1:
-                # if we have only 1 server and then let the
-                # main requester handle any errors. This also means it
-                # must handle a 401 but we save a ping.
-                return serverproxy
             try:
-                command = getattr(serverproxy, 'ping')
-                try:
-                    response = command([], {})
-                except Fault, e:
-                    e = decode_fault(e)
-                    if e.faultCode in errors_by_code:
-                        error = errors_by_code[e.faultCode]
-                        raise error(message=e.faultString)
-                    else:
-                        raise UnknownError(
-                            code=e.faultCode,
-                            error=e.faultString,
-                            server=url,
-                        )
-                # We don't care about the response, just that we got one
+                if use_krb:
+                    setattr(context, 'request_url', url)
+                    serverproxy = self.server_proxy_class(url, **kw)
+                    if len(urls) == 1:
+                        # if we have only 1 server and then let the
+                        # main requester handle any errors. This also means it
+                        # must handle a 401 but we save a ping.
+                        return serverproxy
+                    self.krb_auth(serverproxy, url)
+                else:
+                    self.forms_auth(url, principal, user, password)
+                    # further forms-based communication requires session url
+                    if self.session_path not in url:
+                        url = self.get_session_url(url)
+                    setattr(context, 'request_url', url)
+                    serverproxy = self.server_proxy_class(url, **kw)
                 break
-            except KerberosError, krberr:
-                # kerberos error on one server is likely on all
-                raise errors.KerberosError(major=str(krberr), minor='')
+
             except ProtocolError, e:
                 if hasattr(context, 'session_cookie') and e.errcode == 401:
-                    # Unauthorized. Remove the session and try again.
-                    delattr(context, 'session_cookie')
-                    try:
-                        delete_persistent_client_session_data(principal)
-                    except Exception, e:
-                        # This shouldn't happen if we have a session but it isn't fatal.
-                        pass
-                    return self.create_connection(ccache, verbose, fallback, delegate)
+                    self.destroy_session(principal)
+                    return self.create_connection(
+                        ccache, verbose, fallback, delegate, nss_dir,
+                        user, password)
                 if not fallback:
                     raise
                 serverproxy = None
+            except HTTPError, e:
+                if e.code == 401:
+                    self.destroy_session(principal)
+                if not fallback:
+                    raise
+                self.log.info('Connection to %s failed with %s', url, e)
+                serverproxy = None
             except Exception, e:
                 if not fallback:
                     raise
@@ -850,6 +886,63 @@ class RPCClient(Connectible):
                 error=', '.join(urls))
         return serverproxy
 
+    def krb_auth(self, serverproxy, url):
+        try:
+            command = getattr(serverproxy, 'ping')
+            try:
+                response = command([], {})
+            except Fault, e:
+                e = decode_fault(e)
+                if e.faultCode in errors_by_code:
+                    error = errors_by_code[e.faultCode]
+                    raise error(message=e.faultString)
+                else:
+                    raise UnknownError(
+                        code=e.faultCode,
+                        error=e.faultString,
+                        server=url,
+                    )
+        except KerberosError, krberr:
+            # kerberos error on one server is likely on all
+            raise errors.KerberosError(major=str(krberr), minor='')
+
+    def forms_auth(self, rpc_url, principal, user, password):
+        """
+        Try forms-based authentication.
+        """
+        (scheme, netloc, path, params, query, fragment) = urlparse.urlparse(rpc_url)
+        login_url = urlparse.urlunparse(
+            (scheme, netloc, 'ipa/session/login_password', '', '', ''))
+        self.log.debug('Forms-based authentication for: %s' % rpc_url)
+        self.log.debug('User: %s' % user)
+
+        headers = {
+            'Content-type': 'application/x-www-form-urlencoded; charset=UTF-8',
+            'Referer': 'https://%s/ipa/xml' % str(netloc),
+        }
+        data = {
+            'user': user,
+            'password': password,
+        }
+        data = urllib.urlencode(data)
+        handlers = [urllib2.HTTPSHandler()]
+        opener = urllib2.build_opener(*handlers)
+        req = urllib2.Request(login_url, data, headers)
+        response = urllib2.urlopen(req)
+        if response.getcode() == 200:
+            self.log.debug("Forms based auth successfull")
+            session_cookie = Cookie.get_named_cookie_from_string(
+                response.info().getheader('Set-Cookie'),
+                COOKIE_NAME,
+                rpc_url
+            )
+            cookie_string = str(session_cookie)
+            update_persistent_client_session_data(principal, cookie_string)
+            ipa_cookie = session_cookie.http_cookie()
+            setattr(context, 'session_cookie', ipa_cookie)
+            self.log.debug("forms_auth session:" + ipa_cookie)
+
+
     def destroy_connection(self):
         if sys.version_info >= (2, 7):
             conn = getattr(context, self.id, None)
@@ -906,19 +999,21 @@ class RPCClient(Connectible):
             session_cookie = getattr(context, 'session_cookie', None)
             if session_cookie and e.errcode == 401:
                 # Unauthorized. Remove the session and try again.
-                delattr(context, 'session_cookie')
-                try:
-                    principal = getattr(context, 'principal', None)
-                    delete_persistent_client_session_data(principal)
-                except Exception, e:
-                    # This shouldn't happen if we have a session but it isn't fatal.
-                    pass
+                self.destroy_session(getattr(context, 'principal', None))
 
                 # Create a new serverproxy with the non-session URI. If there
                 # is an existing connection we need to save the NSS dbdir so
                 # we can skip an unnecessary NSS_Initialize() and avoid
                 # NSS_Shutdown issues.
-                serverproxy = self.create_connection(os.environ.get('KRB5CCNAME'), self.env.verbose, self.env.fallback, self.env.delegate)
+                serverproxy = self.create_connection(
+                    os.environ.get('KRB5CCNAME'),
+                    self.env.verbose,
+                    self.env.fallback,
+                    self.env.delegate,
+                    getattr(context, 'nss_dir', None),
+                    getattr(context, 'user', None),
+                    getattr(context, 'password', None)
+                )
 
                 dbdir = None
                 current_conn = getattr(context, self.id, None)
-- 
2.1.0

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Reply via email to