Re: [Freeipa-devel] [PATCH 016, 024, 025] First part of the replica promotion tests + testplan

[Martin Basti]

On 22.03.2016 14:30, Oleg Fayans wrote:

On 03/22/2016 02:06 PM, Martin Basti wrote:

On 21.03.2016 15:54, Oleg Fayans wrote:

Hi Lukas, Martin,

Looks I've implemented the approach proposed by Martin. The issue seems
to have gone (see the external_ca_out for external_ca test output).
Would like you to take a look and tell me what'd you think.


On 03/17/2016 08:37 PM, Lukas Slebodnik wrote:

On (17/03/16 16:00), Oleg Fayans wrote:

Hi Lukas,

On 03/17/2016 11:28 AM, Lukas Slebodnik wrote:

On (10/03/16 23:09), Oleg Fayans wrote:

Hi Martin,



On 03/08/2016 08:18 PM, Martin Basti wrote:

On 08.03.2016 18:24, Martin Basti wrote:

On 08.03.2016 12:38, Oleg Fayans wrote:

The patches were rebased against the current master

On 03/04/2016 05:33 PM, Martin Basti wrote:

* old messages have been removed *

1)
this method is unused please remove it

        def test_kra_install_master(self):

Well, in fact it is used twice: in both domain levels, so I'd
better
keep it:

-bash-4.3$ ipa-run-tests
test_integration/test_replica_promotion.py
--collect-only
====================================================================================



test session starts
=====================================================================================



platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.3
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile:
pytest.ini
plugins: sourceorder, multihost
collected 8 items
<Module 'test_integration/test_replica_promotion.py'>
      <Class 'TestReplicaPromotionLevel0'>
        <Instance '()'>
          <Function 'test_kra_install_master'>
          <Function 'test_promotion_disabled'>
          <Function 'test_backup_restore'>
      <Class 'TestKRAInstall'>
        <Instance '()'>
          <Function 'test_kra_install_without_replica_file'>
      <Class 'TestCAInstall'>
        <Instance '()'>
          <Function 'test_ca_install_without_replica_file'>
      <Class 'TestReplicaPromotionLevel1'>
        <Instance '()'>
          <Function 'test_kra_install_master'>
          <Function 'test_replica_prepare_disabled'>
      <Class 'TestReplicaManageCommands'>
        <Instance '()'>
          <Function 'test_replica_manage_commands'>

aah my bad, I forgot that pytest executes it when it begins
with test_*
even in parent class

2)
Why are these there? I do not see any usage

from env_config import get_global_config
config = get_global_config()

Removed

3) nitpick
+    num_clients = 0
this is set by default

Removed

otherwise LGTM

Results of testing tomorrow.

Martin^2

I applied all patches including workarounds, but test failed.

ipatests.test_integration.test_replica_promotion.TestReplicaPromotionLevel0





[ipa.ipatests.test_integration.host.Host.replica1.cmd51] RUN
['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
'Secret123',
'--setup-ca', '--ip-address', '192.168.144.102',
'/root/ipatests/replica-info.gpg']
[ipa.ipatests.test_integration.host.Host.replica1.cmd51]
The host
replica1.ipa.test already exists on the master server.
[ipa.ipatests.test_integration.host.Host.replica1.cmd51]
You should
remove it before proceeding:
[ipa.ipatests.test_integration.host.Host.replica1.cmd51] % ipa
host-del replica1.ipa.test
[ipa.ipatests.test_integration.host.Host.replica1.cmd51]
ipa.ipapython.install.cli.install_tool(Replica): ERROR    The
ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
[ipa.ipatests.test_integration.host.Host.replica1.cmd51] Exit
code: 3
FAILED

this is exactly the error that happens when a workaround for
5627
is not
applied. I have re-run the tests with all the patches and
everything
passed. Could you please double-check, whether patch 0027 was
applied
correctly?

bash-4.3$ ipa-run-tests
test_integration/test_replica_promotion.py
--pdb
====================================================================================



test session starts
=====================================================================================



platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.3
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile:
pytest.ini
plugins: sourceorder, multihost
collected 8 items

test_integration/test_replica_promotion.py ........

================================================================================



8 passed in 7561.93 seconds
=================================================================================

I will

And it needs ticket, otherwise it will not be in 4-3 branch.

https://fedorahosted.org/freeipa/ticket/5723

NACK

1)
ipatests.test_integration.test_replica_promotion.TestReplicaPromotionLevel0



[ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]

RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
'Secret123', '--setup-ca', '--ip-address', '192.168.200.103', '-r',
'IPA.TEST']
[ipa.ipatests.test_integration.host.Host.replica2.cmd65] RUN
['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
'--setup-ca', '--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
[ipa.ipatests.test_integration.host.Host.replica2.cmd65] IPA
client is
already configured on this system, ignoring the --domain, --server,
--realm, --hostname, --password and --keytab options.
[ipa.ipatests.test_integration.host.Host.replica2.cmd65] Your
system
may be partly configured.
[ipa.ipatests.test_integration.host.Host.replica2.cmd65] Run
/usr/sbin/ipa-server-install --uninstall to clean up.
[ipa.ipatests.test_integration.host.Host.replica2.cmd65]
[ipa.ipatests.test_integration.host.Host.replica2.cmd65]
ipa.ipapython.install.cli.install_tool(Replica): ERROR    You must
provide a file generated by ipa-replica-prepare to create a replica
when the domain is at level 0.
[ipa.ipatests.test_integration.host.Host.replica2.cmd65]
ipa.ipapython.install.cli.install_tool(Replica): ERROR    The
ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
[ipa.ipatests.test_integration.host.Host.replica2.cmd65] Exit
code: 1
FAILED

2)
ipatests.test_integration.test_replica_promotion.TestKRAInstall

[ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]

RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
'Secret123', '--setup-ca', '--ip-address', '192.168.200.103', '-r',
'IPA.TEST']
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] RUN
['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
'--setup-ca', '--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] IPA
client is
already configured on this system, ignoring the --domain, --server,
--realm, --hostname, --password and --keytab options.
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] Your
system
may be partly configured.
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] Run
/usr/sbin/ipa-server-install --uninstall to clean up.
[ipa.ipatests.test_integration.host.Host.replica2.cmd22]
[ipa.ipatests.test_integration.host.Host.replica2.cmd22]
ipa.ipapython.install.cli.install_tool(Replica): ERROR    You must
provide a file generated by ipa-replica-prepare to create a replica
when the domain is at level 0.
[ipa.ipatests.test_integration.host.Host.replica2.cmd22]
ipa.ipapython.install.cli.install_tool(Replica): ERROR    The
ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] Exit
code: 1
FAILED


3)
ipatests.test_integration.test_replica_promotion.TestCAInstall

[ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]

RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
'Secret123', '--setup-dns', '--forwarder', '10.34.78.1',
'--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] RUN
['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
'--setup-dns', '--forwarder', '10.34.78.1', '--ip-address',
'192.168.200.103', '-r', 'IPA.TEST']
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] IPA
client is
already configured on this system, ignoring the --domain, --server,
--realm, --hostname, --password and --keytab options.
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] Your
system
may be partly configured.
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] Run
/usr/sbin/ipa-server-install --uninstall to clean up.
[ipa.ipatests.test_integration.host.Host.replica2.cmd22]
[ipa.ipatests.test_integration.host.Host.replica2.cmd22]
ipa.ipapython.install.cli.install_tool(Replica): ERROR    You must
provide a file generated by ipa-replica-prepare to create a replica
when the domain is at level 0.
[ipa.ipatests.test_integration.host.Host.replica2.cmd22]
ipa.ipapython.install.cli.install_tool(Replica): ERROR    The
ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
[ipa.ipatests.test_integration.host.Host.replica2.cmd22] Exit
code: 1
FAILED


Reason:

   def install_replica(master, replica, setup_ca=True,
setup_dns=False,
-                    setup_kra=False, extra_args=()):
+                    setup_kra=False, extra_args=(),
domain_level=None):
+    if domain_level is None:
+        domain_level = domainlevel(master)

Oops. Sorry, must have been a result of inaccurate rebase. Fixed
The successful run is attached

-    if domainlevel(master) == DOMAIN_LEVEL_0:
+    if domainlevel == DOMAIN_LEVEL_0:

domain_level variable is actually unused

Also with your patch that removes host_prepare, I see many following
errors in test debug output

[ipa.ipatests.test_integration.host.Host.replica1.cmd8] -bash:
line 1:
cd: /root/ipatests: No such file or directory
[ipa.ipatests.test_integration.host.Host.replica1.cmd8] -bash:
line 2:
/root/ipatests/env.sh: No such file or directory


I do not see these errors in current tests, so the patch does not
seem
right to me.

http://jenkins.idm.lab.eng.brq.redhat.com:8080/job/freeipa-integration-f23master-customized_ds_config_install-domlevel-1/23/consoleFull


I've been experimenting a lot to find get rid of it but was
unsuccessful. Seemingly it does not affect the tests.

Oleg it looks like you broke something
in the test_integration/test_external_ca.py

_______________________ TestExternalCA.test_external_ca
________________________

self = <ipatests.test_integration.test_external_ca.TestExternalCA
object at 0x7ff560a51750>

      def test_external_ca(self):
          # Step 1 of ipa-server-install
          self.master.run_command([
              'ipa-server-install', '-U',
              '-a', self.master.config.admin_password,
              '-p', self.master.config.dirman_password,
              '--setup-dns', '--no-forwarders',
              '-n', self.master.domain.name,
              '-r', self.master.domain.realm,
              '--domain-level=%i' % self.master.config.domain_level,
              '--external-ca'
          ])
               nss_db = os.path.join(self.master.config.test_dir,
'testdb')
          external_cert_file = os.path.join(nss_db, 'ipa.crt')
          external_ca_file = os.path.join(nss_db, 'ca.crt')
          noisefile = os.path.join(self.master.config.test_dir,
'noise.txt')
          pwdfile = os.path.join(self.master.config.test_dir,
'pwdfile.txt')
               # Create noise and password files for NSS database

        self.master.run_command('date | sha256sum > %s' % noisefile)

test_integration/test_external_ca.py:49:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _
../pytest_multihost/host.py:246: in run_command
      command.wait(raiseonerr=raiseonerr)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _

self = <pytest_multihost.transport.SSHCommand object at
0x7ff5609eb610>
raiseonerr = True

      def wait(self, raiseonerr=True):
          """Wait for the remote process to exit
                   Raises an excption if the exit code is not 0,
unless raiseonerr is
              true.
              """
          if self._done:
              return self.returncode
               self._end_process()
               self._done = True
               if raiseonerr and self.returncode:
              self.log.error('Exit code: %s', self.returncode)

            raise subprocess.CalledProcessError(self.returncode,
self.argv)

E           CalledProcessError: Command 'date | sha256sum >
/root/ipatests/noise.txt' returned non-zero exit status 1

../pytest_multihost/transport.py:159: CalledProcessError
---------------------------- Captured stdout setup
-----------------------------
<ipatests.test_integration.config.Config object at 0x7ff560a51cd0>
========================== 1 failed in 32.25 seconds
===========================



The command "date | sha256sum > /root/ipatests/noise.txt"

The only explanation is that the directory /root/ipatests/ does not
exist.

Obviously this is caused by my patch N 0025
freeipa-ofayans-0025-Removed-a-constantly-failing-call-to-prepare_host.patch


Without it replica_promotion tests would fail. The only compromise I
see
is to remove this line in unapply_fixes function in
ipatests/test_integration/tasks.py:

host.run_command(['rm', '-rvf', host.config.test_dir])

Now the biggest question is: why do we keep removing the config folder
anyway? All the files (backups of /etc/hosts, etc) that ever get there
get overwritten each time anyway so it's quite safe to just keep the
folder throughout the whole test execution.

I do not know why it was removed. IMHO it shoudl be enough
to remove just content of this directory.

But it will bee godd to fix it ASAP
and unblock test test_integration/test_external_ca.py
I hope we want to have green test in 4.3 branch.
Please also open a ticket so it can be backported to
stable branch (if needed)

LS

NACK

0)
it is not refactoring, it is bugfixing because it is broken

1)
Originally prepare hosts were called for every installation where tasks
install_server, install_replica, install_client were used. Now hosts are
prepared only in base install class, so all test cases where install
classmethod is overriden will have no test directory prepared. Does have
py.test any method that is called for each test class before install?

Well, we can revert my changes from patch-0025 that actually removed
prepare_host call from a mh fixture in
ipatests/pytest_plugins/integration.py
Then we probably do not need most of the changes from this patch except
the try-except section in prepare_host method itself.

OK

2)
I dont like new option in unapply fixes, it breaks symetry, if
prepare_host is adding directory, then reverse operation should remove
directory, IMO unapply_fixes is reverse operation for prepare_host and
it should not have any option that changes behavior.

Agreed. Will remove it.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code