On 03/30/2016 04:52 PM, Martin Basti wrote:
On 24.03.2016 19:10, Stanislav Laznicka wrote:
On 03/23/2016 08:13 PM, Martin Basti wrote:
[...]
Can you please update design
http://www.freeipa.org/page/V4/Manage_replication_topology_4_4 (mainly
the --suffix option)? Also there are missing clean-ruv and list-ruv
commands in design, and fix usage at the bottom.

1)
I don't understand this expression
+            if dirman_passwd is None or (
+               not dirman_passwd and args[0] in cs_enabled_commands):

You already tested if subcommand belongs to cs_enabled_commands few
lines above, IMO the 'dirman_password is None' expression is enough.
If I understand it well, when empty password is entered, the program continues and uses Kerberos credentials for some operations. E.g. for list-ruv, if empty password is entered, the command would only display RUVs for domain tree but not for the CA tree no matter if CA is set up or not (in the current state of the patch, after get_ruv refactoring). This here is one possible way around this, although the check for non-empty password might probably just as well be in get_ruv_both_suffixes.

ok
2)
+# tuple of commands that work with ca tree and need Directory Manager
password
+cs_enabled_commands = ("list-ruv", "clean-ruv", "abort-clean-ruv")

this variable is used only toi detect if dirman passwd is needed, I
suggest to rename it to commands_req_dirman_passwd, or something better.

3)
Q: Do we need is_cs_set() function?
A: Yes!

I wanted to give you ultimate NACK, but then I checked how get_ruv code
works and I changed my mind.

Please write a comment where is_cs_set function is used, why we need
extra function instead of catching an exception, possibly you can open a
refactoring ticket.
After the refactoring changes, is_cs_set should not be needed anymore, removed it.

Shame:
1)
+        if not test_connection(realm, host, options.nolookup) or\
Please use parentheses instead of backslash

2)
+           args[0] in cs_enabled_commands:

+               not dirman_passwd and args[0] in cs_enabled_commands):

Indentation is not multiplication of 4
Shame on me indeed, fixed it.

Nitpicks (I don't insist on fixing these):
1)
+    if servers.get('ca', None):

None is default

2)
+        for (netloc, rid) in servers['ca']:
parentheses are not needed

3)
+ print("\t%s: %s" % (netloc, rid))
Would be nice to use .format() instead of %

Martin^2



I changed my mind, ultimate NACK.
Please fix get_ruv function, is_cs_set will not help. In case there are no RUVs but CA is installed, sys.exit there prevents us from removing RUVs (or any else operation) on domain suffix, and vice versa. I propose to move ticket to 4.4 milestone because I would like to avoid breaking something in 4.3, as this will hit many places in ipa-replica-manage.

Please provide the refactoring of get_ruv as separate patch a put these patches on top of it.

Martin2
Did the refactoring. There also seemed to be duplicit code in abort_clean_ruv for some reason, removed it (I hope it does not break anything, but it seemed rather useless). Also had to change the numbers of the patches so that they would apply.

NACK

* ipa-replica-manage refactoring *

1)
Instead of:
-        print("Failed to connect to server %s: %s" % (host, e))
-        sys.exit(1)
+        root_logger.debug("Failed to connect to server {host}: {err}"
+                          .format(host=host, err=e))
+        raise RuntimeError(e)

I expected

-        print("Failed to connect to server %s: %s" % (host, e))
-        sys.exit(1)
+        root_logger.debug(traceback.format_exc())
+        raise RuntimeError("Failed to connect to server {host}: {err}"
+                                      .format(host=host, err=e)))
Should be fixed now.

2)
-        print("No RUV records found.")
-        sys.exit(0)

Here is exit state 0, so we should not raise error.

I think that we should create new nonfatal exception.
Made a new nonfatal error exception, then. This step was a bit controversial when it comes to get_ruv_both_suffixes because it needs to catch both this new exception and a RuntimeError exception for both trees. As the main idea here was not to stop if either tree is missing (resulting in RuntimeError in get_ruv) or contains no records (NonFatalError), it is only printed that something bad may have happened (or it's debug-logged in case of nonfatal errors). In the end, only NonFatalError exception is raised if no records were found for whatever reason resulting in the script always returning 0.

3)
-                print("unable to decode: %s" % ruv)
+                root_logger.debug("unable to decode: %s" % ruv)
This may break tests, because the logger logs to stderr, leave it as print for now

4)
-    servers = get_ruv(realm, host, dirman_passwd, nolookup)
+    try:
+        servers = get_ruv(realm, host, dirman_passwd, nolookup)
+    except RuntimeError as e:
+        print(e)
+        sys.exit(1)

again we have to print it to stdout for now.
3), 4) done, although it might be better to log to stderr from patch number 27 and on since the default behavior is changed anyway.

* abort-clean/list/clean-ruv now work for both suffixes *

-            if dirman_passwd is None:
+            if dirman_passwd is None or (
+ not dirman_passwd and args[0] in dirman_passwd_req_commands):
                 sys.exit("Directory Manager password required")
Done.

Please fix other patch accordingly.
Martin^2

From 508d5429829cf5eedca47420c1ce3eaaa3a8bd0c Mon Sep 17 00:00:00 2001
From: Stanislav Laznicka <slazn...@redhat.com>
Date: Thu, 24 Mar 2016 15:59:31 +0100
Subject: [PATCH 1/3] ipa-replica-manage refactoring

get_ruv does not call sys.exit anymore, instead it raises RuntimeError
for better error handling

Also removed duplicit code from abort_clean_ruv
---
 install/tools/ipa-replica-manage | 72 ++++++++++++++++++++++++----------------
 1 file changed, 43 insertions(+), 29 deletions(-)

diff --git a/install/tools/ipa-replica-manage b/install/tools/ipa-replica-manage
index 6d1fc93c4c8499dcb4cd0924ff837a0216395821..705dbd1e6d0052f6e25efec55548b51c059518c1 100755
--- a/install/tools/ipa-replica-manage
+++ b/install/tools/ipa-replica-manage
@@ -68,6 +68,13 @@ commands = {
     "dnanextrange-set":(2, 2, "<master fqdn> <range>", "must provide a master and ID range"),
 }
 
+# tuple of commands that need proper Directory Manager password
+dirman_passwd_req_commands = ("list-ruv", "clean-ruv", "abort-clean-ruv")
+
+
+class NonFatalError(Exception):
+    pass
+
 
 def parse_options():
     parser = IPAOptionParser(version=version.VERSION)
@@ -361,8 +368,9 @@ def get_ruv(realm, host, dirman_passwd, nolookup=False, ca=False):
         else:
             thisrepl = replication.ReplicationManager(realm, host, dirman_passwd)
     except Exception as e:
-        print("Failed to connect to server %s: %s" % (host, e))
-        sys.exit(1)
+        root_logger.debug(traceback.format_exc())
+        raise RuntimeError("Failed to connect to server {host}: {err}"
+                           .format(host=host, err=e))
 
     search_filter = '(&(nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff)(objectclass=nstombstone))'
     try:
@@ -370,8 +378,8 @@ def get_ruv(realm, host, dirman_passwd, nolookup=False, ca=False):
             thisrepl.db_suffix, thisrepl.conn.SCOPE_SUBTREE, search_filter,
             ['nsds50ruv'])
     except errors.NotFound:
-        print("No RUV records found.")
-        sys.exit(0)
+        root_logger.debug(traceback.format_exc())
+        raise NonFatalError("No RUV records found.")
 
     servers = []
     for e in entries:
@@ -393,8 +401,11 @@ def list_ruv(realm, host, dirman_passwd, verbose, nolookup=False):
     List the Replica Update Vectors on this host to get the available
     replica IDs.
     """
-
-    servers = get_ruv(realm, host, dirman_passwd, nolookup)
+    try:
+        servers = get_ruv(realm, host, dirman_passwd, nolookup)
+    except (NonFatalError, RuntimeError) as e:
+        print(e)
+        sys.exit(0 if isinstance(e, NonFatalError) else 1)
     for (netloc, rid) in servers:
         print("%s: %s" % (netloc, rid))
 
@@ -402,7 +413,11 @@ def get_rid_by_host(realm, sourcehost, host, dirman_passwd, nolookup=False):
     """
     Try to determine the RID by host name.
     """
-    servers = get_ruv(realm, sourcehost, dirman_passwd, nolookup)
+    try:
+        servers = get_ruv(realm, sourcehost, dirman_passwd, nolookup)
+    except (NonFatalError, RuntimeError) as e:
+        print(e)
+        sys.exit(0 if isinstance(e, NonFatalError) else 1)
     for (netloc, rid) in servers:
         if '%s:389' % host == netloc:
             return int(rid)
@@ -416,14 +431,19 @@ def clean_ruv(realm, ruv, options, ca=False):
     except ValueError:
         sys.exit("Replica ID must be an integer: %s" % ruv)
 
-    servers = get_ruv(realm, options.host, options.dirman_passwd,
-                      options.nolookup, ca=ca)
+    try:
+        servers = get_ruv(realm, options.host, options.dirman_passwd,
+                          options.nolookup, ca=ca)
+    except (NonFatalError, RuntimeError) as e:
+        print(e)
+        sys.exit(0 if isinstance(e, NonFatalError) else 1)
+
     found = False
     for (netloc, rid) in servers:
         if ruv == int(rid):
-           found = True
-           hostname = netloc
-           break
+            found = True
+            hostname = netloc
+            break
 
     if not found:
         sys.exit("Replica ID %s not found" % ruv)
@@ -461,26 +481,19 @@ def abort_clean_ruv(realm, ruv, options):
     except ValueError:
         sys.exit("Replica ID must be an integer: %s" % ruv)
 
-    servers = get_ruv(realm, options.host, options.dirman_passwd,
-                      options.nolookup)
-    found = False
-    for (netloc, rid) in servers:
-        if ruv == int(rid):
-           found = True
-           hostname = netloc
-           break
-
-    if not found:
-        sys.exit("Replica ID %s not found" % ruv)
+    try:
+        servers = get_ruv(realm, options.host, options.dirman_passwd,
+                          options.nolookup)
+    except (NonFatalError, RuntimeError) as e:
+        print(e)
+        sys.exit(0 if isinstance(e, NonFatalError) else 1)
 
-    servers = get_ruv(realm, options.host, options.dirman_passwd,
-                      options.nolookup)
     found = False
     for (netloc, rid) in servers:
         if ruv == int(rid):
-           found = True
-           hostname = netloc
-           break
+            found = True
+            hostname = netloc
+            break
 
     if not found:
         sys.exit("Replica ID %s not found" % ruv)
@@ -1564,7 +1577,8 @@ def main(options, args):
         if not test_connection(realm, host, options.nolookup):
             dirman_passwd = installutils.read_password("Directory Manager",
                 confirm=False, validate=False, retry=False)
-            if dirman_passwd is None:
+            if dirman_passwd is None or (
+               not dirman_passwd and args[0] in dirman_passwd_req_commands):
                 sys.exit("Directory Manager password required")
 
     options.dirman_passwd = dirman_passwd
-- 
2.5.5

From f4688cc93af7d53f6438ae25bd98c2b1f868cd48 Mon Sep 17 00:00:00 2001
From: Stanislav Laznicka <slazn...@redhat.com>
Date: Fri, 11 Mar 2016 10:15:02 +0100
Subject: [PATCH 2/3] abort-clean/list/clean-ruv now work for both suffixes

The rid passed to abort-clean-ruv and clean-ruv is now searched
for in both ipaca and domain trees as well as list-ruv now
displays both RUVs and CS-RUVs

https://fedorahosted.org/freeipa/ticket/4987
---
 install/tools/ipa-replica-manage       | 121 ++++++++++++++++++++++++---------
 install/tools/man/ipa-replica-manage.1 |   8 ++-
 2 files changed, 94 insertions(+), 35 deletions(-)

diff --git a/install/tools/ipa-replica-manage b/install/tools/ipa-replica-manage
index 705dbd1e6d0052f6e25efec55548b51c059518c1..7c946017651a3281ceb09b39adc6805183b63ce2 100755
--- a/install/tools/ipa-replica-manage
+++ b/install/tools/ipa-replica-manage
@@ -68,7 +68,7 @@ commands = {
     "dnanextrange-set":(2, 2, "<master fqdn> <range>", "must provide a master and ID range"),
 }
 
-# tuple of commands that need proper Directory Manager password
+# tuple of commands that work with ca tree and need Directory Manager password
 dirman_passwd_req_commands = ("list-ruv", "clean-ruv", "abort-clean-ruv")
 
 
@@ -396,18 +396,56 @@ def get_ruv(realm, host, dirman_passwd, nolookup=False, ca=False):
 
     return servers
 
+
+def get_ruv_both_suffixes(realm, host, dirman_passwd, nolookup=False):
+    """
+    Get RUVs for both domain and ipaca suffixes
+    """
+    ruvs = {}
+
+    try:
+        ruvs['ca'] = get_ruv(realm, host, dirman_passwd, nolookup, True)
+    except (NonFatalError, RuntimeError) as e:
+        err = "Failed to get CS-RUVs from {host}: {err}".format(host=host,
+                                                                err=e)
+        if isinstance(e, RuntimeError):
+            print(err)
+        else:
+            root_logger.debug(err)
+    try:
+        ruvs['domain'] = get_ruv(realm, host, dirman_passwd, nolookup)
+    except (NonFatalError, RuntimeError) as e:
+        err = "Failed to get RUVs from {host}: {err}".format(host=host, err=e)
+        if isinstance(e, RuntimeError):
+            print(err)
+        else:
+            root_logger.debug(err)
+
+    if not ruvs.keys():
+        raise NonFatalError("No RUV records found.")
+
+    return ruvs
+
+
 def list_ruv(realm, host, dirman_passwd, verbose, nolookup=False):
     """
     List the Replica Update Vectors on this host to get the available
     replica IDs.
     """
     try:
-        servers = get_ruv(realm, host, dirman_passwd, nolookup)
-    except (NonFatalError, RuntimeError) as e:
+        servers = get_ruv_both_suffixes(realm, host, dirman_passwd, nolookup)
+    except NonFatalError as e:
         print(e)
-        sys.exit(0 if isinstance(e, NonFatalError) else 1)
-    for (netloc, rid) in servers:
-        print("%s: %s" % (netloc, rid))
+        sys.exit(0)
+    if servers.get('domain'):
+        print('Replica Update Vectors:')
+        for netloc, rid in servers['domain']:
+            print("\t{name}: {id}".format(name=netloc, id=rid))
+    if servers.get('ca'):
+        print('Certificate Server Replica Update Vectors:')
+        for netloc, rid in servers['ca']:
+            print("\t{name}: {id}".format(name=netloc, id=rid))
+
 
 def get_rid_by_host(realm, sourcehost, host, dirman_passwd, nolookup=False):
     """
@@ -422,7 +460,8 @@ def get_rid_by_host(realm, sourcehost, host, dirman_passwd, nolookup=False):
         if '%s:389' % host == netloc:
             return int(rid)
 
-def clean_ruv(realm, ruv, options, ca=False):
+
+def clean_ruv(realm, ruv, options):
     """
     Given an RID create a CLEANALLRUV task to clean it up.
     """
@@ -432,23 +471,27 @@ def clean_ruv(realm, ruv, options, ca=False):
         sys.exit("Replica ID must be an integer: %s" % ruv)
 
     try:
-        servers = get_ruv(realm, options.host, options.dirman_passwd,
-                          options.nolookup, ca=ca)
-    except (NonFatalError, RuntimeError) as e:
+        servers = get_ruv_both_suffixes(realm, options.host,
+                                        options.dirman_passwd,
+                                        options.nolookup)
+    except NonFatalError as e:
         print(e)
-        sys.exit(0 if isinstance(e, NonFatalError) else 1)
+        sys.exit(0)
 
-    found = False
-    for (netloc, rid) in servers:
-        if ruv == int(rid):
-            found = True
-            hostname = netloc
+    tree_found = None
+    for tree, ruvs in servers.items():
+        for netloc, rid in ruvs:
+            if ruv == int(rid):
+                tree_found = tree
+                hostname = netloc
+                break
+        if tree_found:
             break
 
-    if not found:
+    if not tree_found:
         sys.exit("Replica ID %s not found" % ruv)
 
-    if ca:
+    if tree_found == 'ca':
         print("Clean the Certificate Server Replication Update Vector for %s"
               % hostname)
     else:
@@ -463,7 +506,7 @@ def clean_ruv(realm, ruv, options, ca=False):
         if not ipautil.user_input("Continue to clean?", False):
             sys.exit("Aborted")
 
-    if ca:
+    if tree_found == 'ca':
         thisrepl = replication.get_cs_replication_manager(realm, options.host,
                                                           options.dirman_passwd)
     else:
@@ -472,6 +515,7 @@ def clean_ruv(realm, ruv, options, ca=False):
     thisrepl.cleanallruv(ruv)
     print("Cleanup task created")
 
+
 def abort_clean_ruv(realm, ruv, options):
     """
     Given an RID abort a CLEANALLRUV task.
@@ -482,30 +526,39 @@ def abort_clean_ruv(realm, ruv, options):
         sys.exit("Replica ID must be an integer: %s" % ruv)
 
     try:
-        servers = get_ruv(realm, options.host, options.dirman_passwd,
-                          options.nolookup)
-    except (NonFatalError, RuntimeError) as e:
+        servers = get_ruv_both_suffixes(realm, options.host,
+                                        options.dirman_passwd,
+                                        options.nolookup)
+    except NonFatalError as e:
         print(e)
-        sys.exit(0 if isinstance(e, NonFatalError) else 1)
+        sys.exit(0)
 
-    found = False
-    for (netloc, rid) in servers:
-        if ruv == int(rid):
-            found = True
-            hostname = netloc
+    tree_found = None
+    for tree, ruvs in servers.items():
+        for netloc, rid in ruvs:
+            if ruv == int(rid):
+                tree_found = tree
+                hostname = netloc
+                break
+        if tree_found:
             break
 
-    if not found:
+    if not tree_found:
         sys.exit("Replica ID %s not found" % ruv)
 
     print("Aborting the clean Replication Update Vector task for %s" % hostname)
     print()
-    thisrepl = replication.ReplicationManager(realm, options.host,
-                                              options.dirman_passwd)
+    if tree_found == 'ca':
+        thisrepl = replication.get_cs_replication_manager(realm, options.host,
+                                                          options.dirman_passwd)
+    else:
+        thisrepl = replication.ReplicationManager(realm, options.host,
+                                                  options.dirman_passwd)
     thisrepl.abortcleanallruv(ruv, options.force)
 
     print("Cleanup task stopped")
 
+
 def list_clean_ruv(realm, host, dirman_passwd, verbose, nolookup=False):
     """
     List all clean RUV tasks.
@@ -701,7 +754,7 @@ def clean_dangling_ruvs(realm, host, options):
         for csruv in master_info['clean_csruv']:
             if csruv[1] not in cleaned:
                 cleaned.add(csruv[1])
-                clean_ruv(realm, csruv[1], options, ca=True)
+                clean_ruv(realm, csruv[1], options)
 
 
 def check_last_link(delrepl, realm, dirman_passwd, force):
@@ -1574,7 +1627,8 @@ def main(options, args):
     if options.dirman_passwd:
         dirman_passwd = options.dirman_passwd
     else:
-        if not test_connection(realm, host, options.nolookup):
+        if (not test_connection(realm, host, options.nolookup) or
+           args[0] in dirman_passwd_req_commands):
             dirman_passwd = installutils.read_password("Directory Manager",
                 confirm=False, validate=False, retry=False)
             if dirman_passwd is None or (
@@ -1596,6 +1650,7 @@ def main(options, args):
             replica = args[1]
         list_replicas(realm, host, replica, dirman_passwd, options.verbose,
                       options.nolookup)
+
     elif args[0] == "list-ruv":
         list_ruv(realm, host, dirman_passwd, options.verbose, options.nolookup)
     elif args[0] == "del":
diff --git a/install/tools/man/ipa-replica-manage.1 b/install/tools/man/ipa-replica-manage.1
index ae109c4c5ff4720eb70b06c6f14a791088696d47..68be0232fae9309b108e69f9144501be3277f503 100644
--- a/install/tools/man/ipa-replica-manage.1
+++ b/install/tools/man/ipa-replica-manage.1
@@ -135,6 +135,7 @@ Password for the IPA system user used by the Windows PassSync plugin to synchron
 .TP
 \fB\-\-from\fR=\fISERVER\fR
 The server to pull the data from, used by the re\-initialize and force\-sync commands.
+.TP
 .SH "RANGES"
 IPA uses the 389\-ds Distributed Numeric Assignment (DNA) Plugin to allocate POSIX ids for users and groups. A range is created when IPA is installed and half the range is assigned to the first IPA master for the purposes of allocation.
 .TP
@@ -190,8 +191,11 @@ Using connect/disconnect you can manage the replication topology.
 .TP
 List the replication IDs in use:
  # ipa\-replica\-manage list\-ruv
- srv1.example.com:389: 7
- srv2.example.com:389: 4
+ Replica Update Vectors:
+     srv1.example.com:389: 7
+     srv2.example.com:389: 4
+ Certificate Server Replica Update Vectors:
+     srv1.example.com:389: 9
 .TP
 Remove references to an orphaned and deleted master:
  # ipa\-replica\-manage del \-\-force \-\-cleanup master.example.com
-- 
2.5.5

From 051c930b6d049178f758566137b849adb9eed5a4 Mon Sep 17 00:00:00 2001
From: Stanislav Laznicka <slazn...@redhat.com>
Date: Wed, 16 Mar 2016 12:28:36 +0100
Subject: [PATCH 3/3] Moved password check from clean_dangling_ruv

The proper password check is now done elsewhere

https://fedorahosted.org/freeipa/ticket/4987
---
 install/tools/ipa-replica-manage | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/install/tools/ipa-replica-manage b/install/tools/ipa-replica-manage
index 7c946017651a3281ceb09b39adc6805183b63ce2..9e99eb544ee9878f0cbc1f12b8a1235b9fd305c7 100755
--- a/install/tools/ipa-replica-manage
+++ b/install/tools/ipa-replica-manage
@@ -69,7 +69,8 @@ commands = {
 }
 
 # tuple of commands that work with ca tree and need Directory Manager password
-dirman_passwd_req_commands = ("list-ruv", "clean-ruv", "abort-clean-ruv")
+dirman_passwd_req_commands = ("list-ruv", "clean-ruv", "abort-clean-ruv",
+                              "clean-dangling-ruv")
 
 
 class NonFatalError(Exception):
@@ -606,15 +607,6 @@ def clean_dangling_ruvs(realm, host, options):
     Cleans all RUVs and CS-RUVs that are left in the system from
     uninstalled replicas
     """
-    # get the Directory Manager password
-    if not options.dirman_passwd:
-        options.dirman_passwd = installutils.read_password('Directory Manager',
-                                                           confirm=False,
-                                                           validate=False,
-                                                           retry=False)
-        if options.dirman_passwd is None:
-            sys.exit('Directory Manager password is required')
-
     conn = ipaldap.IPAdmin(host, 636, cacert=CACERT)
     try:
         conn.do_simple_bind(bindpw=options.dirman_passwd)
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Reply via email to