Re: [Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

2016-09-14 Thread Oleg Fayans 

Again ping for review, please it completely blocks the whole job.

On 09/07/2016 03:27 PM, Oleg Fayans wrote:

ping for review

On 08/24/2016 01:58 PM, Oleg Fayans wrote:

And here is how the run looks like:

$ ipa-run-tests test_integration/test_topology.py
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
denied: 'yacctab.py'


test session starts
=


platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ..x

===

2 passed, 1 xfailed in 1558.66 seconds
===




On 08/12/2016 04:05 PM, Martin Basti wrote:



On 12.08.2016 15:48, Oleg Fayans wrote:

Hi Martin,



On 08/11/2016 10:05 AM, Martin Basti wrote:



On 10.08.2016 20:32, Oleg Fayans wrote:





Hello,

before we jump into fixing tests, my question is: Was this planned
change and not reflected by test, or switched values are unwanted side
effect and thus bug for us?


That's a marvelous question! The test used to pass, which means that
at some point the convention of naming the segments must have changed.
Is it a bug? I do not think so: the feature still works as expected.


Ludwig, do you know details about this change, why positions of server
names are different than used to be in topology name?





Ticket contains almost no info, except a traceback and it says
nothing.
Commit message says at least something.

I'm not sure if this patch fixes that ticket, because traceback in
test
shows error message that "removal of segment will disconnect
topology",
but this patch only swap order of replica names in segment name. I
would
expect that you should get different error, something like segment
does
not exist.

Which I do get in jenkins job N 37: "segment not found"

In fact, the error in the issue is unrelated to the fix, you are right.



To tell the truth, I just put a random error from one of the jenkins
topology testruns into the issue.

This is very good way how to report tickets:
* nobody knows what happened
* nobody can search in current tickets,  what is wrong without proper
description
* developers cannot investigate issue, because there is even no name of
exact test in ticket, no steps to reproduce, nothing
* without proper tickets it is hard to backport patches correctly, if
patch fixes different issue than is reported

I'm closing ticket as invalid, please follow
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html and file a new
proper ticket.


This particular error message was caused by a previous replica
installation failure, which resulted in existing only one segment
instead of three:
master <-> replica1
instead of:
master <-> replica1,
master <-> replica2
replica1 <-> replica2

In fact the patch supplied fixes 2 tests at once:
The first test tries to remove the unexisting segment master <->
replica2 and fails, the second test expects the line topology
master <-> replica1 <-> replica2.
It removes the connection between replica1 and replica2, expects the
operation to fail but it does not because the connection between
master and replica2 exists

the output from the testrun with the patch applied:


-bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
denied: 'yacctab.py'



test session starts
=



platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ...




3 passed in 2156.82 seconds
=






I don't care about test output until there is no valid description of
problem, fixing test may just cover real issue.
Martin^2


Martin^2












--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA:

Re: [Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

2016-09-07 Thread Oleg Fayans 

ping for review

On 08/24/2016 01:58 PM, Oleg Fayans wrote:

And here is how the run looks like:

$ ipa-run-tests test_integration/test_topology.py
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
denied: 'yacctab.py'

test session starts
=

platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ..x

===
2 passed, 1 xfailed in 1558.66 seconds
===



On 08/12/2016 04:05 PM, Martin Basti wrote:



On 12.08.2016 15:48, Oleg Fayans wrote:

Hi Martin,



On 08/11/2016 10:05 AM, Martin Basti wrote:



On 10.08.2016 20:32, Oleg Fayans wrote:





Hello,

before we jump into fixing tests, my question is: Was this planned
change and not reflected by test, or switched values are unwanted side
effect and thus bug for us?


That's a marvelous question! The test used to pass, which means that
at some point the convention of naming the segments must have changed.
Is it a bug? I do not think so: the feature still works as expected.


Ludwig, do you know details about this change, why positions of server
names are different than used to be in topology name?





Ticket contains almost no info, except a traceback and it says nothing.
Commit message says at least something.

I'm not sure if this patch fixes that ticket, because traceback in test
shows error message that "removal of segment will disconnect topology",
but this patch only swap order of replica names in segment name. I
would
expect that you should get different error, something like segment does
not exist.

Which I do get in jenkins job N 37: "segment not found"

In fact, the error in the issue is unrelated to the fix, you are right.



To tell the truth, I just put a random error from one of the jenkins
topology testruns into the issue.

This is very good way how to report tickets:
* nobody knows what happened
* nobody can search in current tickets,  what is wrong without proper
description
* developers cannot investigate issue, because there is even no name of
exact test in ticket, no steps to reproduce, nothing
* without proper tickets it is hard to backport patches correctly, if
patch fixes different issue than is reported

I'm closing ticket as invalid, please follow
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html and file a new
proper ticket.


This particular error message was caused by a previous replica
installation failure, which resulted in existing only one segment
instead of three:
master <-> replica1
instead of:
master <-> replica1,
master <-> replica2
replica1 <-> replica2

In fact the patch supplied fixes 2 tests at once:
The first test tries to remove the unexisting segment master <->
replica2 and fails, the second test expects the line topology
master <-> replica1 <-> replica2.
It removes the connection between replica1 and replica2, expects the
operation to fail but it does not because the connection between
master and replica2 exists

the output from the testrun with the patch applied:


-bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
denied: 'yacctab.py'


test session starts
=


platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ...



3 passed in 2156.82 seconds
=





I don't care about test output until there is no valid description of
problem, fixing test may just cover real issue.
Martin^2


Martin^2










--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

2016-08-24 Thread Oleg Fayans 

Hi Martin,

Updated the test according to our discussion.
There are 2 patches: the one related to the dynamic segment naming and 
the one that xfails one of the tests which fails due to trac ticket 6250.


Please, disregard my previous patch

On 08/12/2016 04:05 PM, Martin Basti wrote:



On 12.08.2016 15:48, Oleg Fayans wrote:

Hi Martin,



On 08/11/2016 10:05 AM, Martin Basti wrote:



On 10.08.2016 20:32, Oleg Fayans wrote:





Hello,

before we jump into fixing tests, my question is: Was this planned
change and not reflected by test, or switched values are unwanted side
effect and thus bug for us?


That's a marvelous question! The test used to pass, which means that
at some point the convention of naming the segments must have changed.
Is it a bug? I do not think so: the feature still works as expected.


Ludwig, do you know details about this change, why positions of server
names are different than used to be in topology name?





Ticket contains almost no info, except a traceback and it says nothing.
Commit message says at least something.

I'm not sure if this patch fixes that ticket, because traceback in test
shows error message that "removal of segment will disconnect topology",
but this patch only swap order of replica names in segment name. I would
expect that you should get different error, something like segment does
not exist.

Which I do get in jenkins job N 37: "segment not found"

In fact, the error in the issue is unrelated to the fix, you are right.



To tell the truth, I just put a random error from one of the jenkins
topology testruns into the issue.

This is very good way how to report tickets:
* nobody knows what happened
* nobody can search in current tickets,  what is wrong without proper
description
* developers cannot investigate issue, because there is even no name of
exact test in ticket, no steps to reproduce, nothing
* without proper tickets it is hard to backport patches correctly, if
patch fixes different issue than is reported

I'm closing ticket as invalid, please follow
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html and file a new
proper ticket.


This particular error message was caused by a previous replica
installation failure, which resulted in existing only one segment
instead of three:
master <-> replica1
instead of:
master <-> replica1,
master <-> replica2
replica1 <-> replica2

In fact the patch supplied fixes 2 tests at once:
The first test tries to remove the unexisting segment master <->
replica2 and fails, the second test expects the line topology
master <-> replica1 <-> replica2.
It removes the connection between replica1 and replica2, expects the
operation to fail but it does not because the connection between
master and replica2 exists

the output from the testrun with the patch applied:


-bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
denied: 'yacctab.py'

test session starts
=

platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ...


3 passed in 2156.82 seconds
=




I don't care about test output until there is no valid description of
problem, fixing test may just cover real issue.
Martin^2


Martin^2








--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.
From 6be984e1ff3ffa0dcbe3bc9fc415b7355a833c24 Mon Sep 17 00:00:00 2001
From: Oleg Fayans 
Date: Wed, 24 Aug 2016 13:48:56 +0200
Subject: [PATCH] Fixed segment naming in topology tests

As the segment name is a stochastic valu, which can have either of the two
nodes as the left node, we need to adapt the tests to not expect some
particular segment name but rather to calculate it dynamically based on node
names and the output of topologysegment-find ipa call
---
 ipatests/test_integration/test_topology.py | 33 --
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/ipatests/test_integration/test_topology.py b/ipatests/test_integration/test_topology.py
index e956563c27eafd84deed5786274a73d0d3594642..a3e0488eacc116d5ac3fe83b021b8bf85bcc2ef3 100644
--- a/ipatests/test_integration/test_topology.py
+++ b/ipatests/test_integration/test_topology.py
@@ -15,6 +15,18 @@ from ipatests.util import assert_deepequal
 config = get_global_config()
 reasoning = "Topology plugin disabled due to domain level 0"

Re: [Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

2016-08-24 Thread Oleg Fayans 

And here is how the run looks like:

$ ipa-run-tests test_integration/test_topology.py
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13] 
Permission denied: 'lextab.py'

WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission 
denied: 'yacctab.py'
 
test session starts 
=

platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ..x

=== 
2 passed, 1 xfailed in 1558.66 seconds 
===




On 08/12/2016 04:05 PM, Martin Basti wrote:



On 12.08.2016 15:48, Oleg Fayans wrote:

Hi Martin,



On 08/11/2016 10:05 AM, Martin Basti wrote:



On 10.08.2016 20:32, Oleg Fayans wrote:





Hello,

before we jump into fixing tests, my question is: Was this planned
change and not reflected by test, or switched values are unwanted side
effect and thus bug for us?


That's a marvelous question! The test used to pass, which means that
at some point the convention of naming the segments must have changed.
Is it a bug? I do not think so: the feature still works as expected.


Ludwig, do you know details about this change, why positions of server
names are different than used to be in topology name?





Ticket contains almost no info, except a traceback and it says nothing.
Commit message says at least something.

I'm not sure if this patch fixes that ticket, because traceback in test
shows error message that "removal of segment will disconnect topology",
but this patch only swap order of replica names in segment name. I would
expect that you should get different error, something like segment does
not exist.

Which I do get in jenkins job N 37: "segment not found"

In fact, the error in the issue is unrelated to the fix, you are right.



To tell the truth, I just put a random error from one of the jenkins
topology testruns into the issue.

This is very good way how to report tickets:
* nobody knows what happened
* nobody can search in current tickets,  what is wrong without proper
description
* developers cannot investigate issue, because there is even no name of
exact test in ticket, no steps to reproduce, nothing
* without proper tickets it is hard to backport patches correctly, if
patch fixes different issue than is reported

I'm closing ticket as invalid, please follow
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html and file a new
proper ticket.


This particular error message was caused by a previous replica
installation failure, which resulted in existing only one segment
instead of three:
master <-> replica1
instead of:
master <-> replica1,
master <-> replica2
replica1 <-> replica2

In fact the patch supplied fixes 2 tests at once:
The first test tries to remove the unexisting segment master <->
replica2 and fails, the second test expects the line topology
master <-> replica1 <-> replica2.
It removes the connection between replica1 and replica2, expects the
operation to fail but it does not because the connection between
master and replica2 exists

the output from the testrun with the patch applied:


-bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
denied: 'yacctab.py'

test session starts
=

platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ...


3 passed in 2156.82 seconds
=




I don't care about test output until there is no valid description of
problem, fixing test may just cover real issue.
Martin^2


Martin^2








--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

2016-08-12 Thread Martin Basti 



On 12.08.2016 15:48, Oleg Fayans wrote:

Hi Martin,



On 08/11/2016 10:05 AM, Martin Basti wrote:



On 10.08.2016 20:32, Oleg Fayans wrote:





Hello,

before we jump into fixing tests, my question is: Was this planned
change and not reflected by test, or switched values are unwanted side
effect and thus bug for us?


That's a marvelous question! The test used to pass, which means that 
at some point the convention of naming the segments must have changed. 
Is it a bug? I do not think so: the feature still works as expected.


Ludwig, do you know details about this change, why positions of server 
names are different than used to be in topology name?






Ticket contains almost no info, except a traceback and it says nothing.
Commit message says at least something.

I'm not sure if this patch fixes that ticket, because traceback in test
shows error message that "removal of segment will disconnect topology",
but this patch only swap order of replica names in segment name. I would
expect that you should get different error, something like segment does
not exist.

Which I do get in jenkins job N 37: "segment not found"

In fact, the error in the issue is unrelated to the fix, you are right. 


To tell the truth, I just put a random error from one of the jenkins 
topology testruns into the issue. 

This is very good way how to report tickets:
* nobody knows what happened
* nobody can search in current tickets,  what is wrong without proper 
description
* developers cannot investigate issue, because there is even no name of 
exact test in ticket, no steps to reproduce, nothing
* without proper tickets it is hard to backport patches correctly, if 
patch fixes different issue than is reported


I'm closing ticket as invalid, please follow 
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html and file a new 
proper ticket.


This particular error message was caused by a previous replica 
installation failure, which resulted in existing only one segment 
instead of three:

master <-> replica1
instead of:
master <-> replica1,
master <-> replica2
replica1 <-> replica2

In fact the patch supplied fixes 2 tests at once:
The first test tries to remove the unexisting segment master <-> 
replica2 and fails, the second test expects the line topology

master <-> replica1 <-> replica2.
It removes the connection between replica1 and replica2, expects the 
operation to fail but it does not because the connection between 
master and replica2 exists


the output from the testrun with the patch applied:


-bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13] 
Permission denied: 'lextab.py'

WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission 
denied: 'yacctab.py'
 
test session starts 
=

platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ...

 
3 passed in 2156.82 seconds 
=




I don't care about test output until there is no valid description of 
problem, fixing test may just cover real issue.

Martin^2


Martin^2






--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

2016-08-12 Thread Oleg Fayans 

Hi Martin,



On 08/11/2016 10:05 AM, Martin Basti wrote:



On 10.08.2016 20:32, Oleg Fayans wrote:





Hello,

before we jump into fixing tests, my question is: Was this planned
change and not reflected by test, or switched values are unwanted side
effect and thus bug for us?


That's a marvelous question! The test used to pass, which means that at 
some point the convention of naming the segments must have changed. Is 
it a bug? I do not think so: the feature still works as expected.




Ticket contains almost no info, except a traceback and it says nothing.
Commit message says at least something.

I'm not sure if this patch fixes that ticket, because traceback in test
shows error message that "removal of segment will disconnect topology",
but this patch only swap order of replica names in segment name. I would
expect that you should get different error, something like segment does
not exist.

Which I do get in jenkins job N 37: "segment not found"

In fact, the error in the issue is unrelated to the fix, you are right. 
To tell the truth, I just put a random error from one of the jenkins 
topology testruns into the issue. This particular error message was 
caused by a previous replica installation failure, which resulted in 
existing only one segment instead of three:

master <-> replica1
instead of:
master <-> replica1,
master <-> replica2
replica1 <-> replica2

In fact the patch supplied fixes 2 tests at once:
The first test tries to remove the unexisting segment master <-> 
replica2 and fails, the second test expects the line topology

master <-> replica1 <-> replica2.
It removes the connection between replica1 and replica2, expects the 
operation to fail but it does not because the connection between master 
and replica2 exists


the output from the testrun with the patch applied:


-bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13] 
Permission denied: 'lextab.py'

WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission 
denied: 'yacctab.py'
 
test session starts 
=

platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ...

 
3 passed in 2156.82 seconds 
=




Martin^2




--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

2016-08-11 Thread Martin Basti 



On 10.08.2016 20:32, Oleg Fayans wrote:





Hello,

before we jump into fixing tests, my question is: Was this planned 
change and not reflected by test, or switched values are unwanted side 
effect and thus bug for us?


Ticket contains almost no info, except a traceback and it says nothing. 
Commit message says at least something.


I'm not sure if this patch fixes that ticket, because traceback in test 
shows error message that "removal of segment will disconnect topology", 
but this patch only swap order of replica names in segment name. I would 
expect that you should get different error, something like segment does 
not exist.


Martin^2


-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code