[JIRA] (OVIRT-2252) The s390x slave is used in parallel by both the staging and the production CI systems
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=40057#comment-40057
]
Evgheni Dereveanchin commented on OVIRT-2252:
-
we have dedicated s390x VMs for Staging and production now. Is this still an
issue?
> The s390x slave is used in parallel by both the staging and the production CI
> systems
> -
>
> Key: OVIRT-2252
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2252
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Components: Jenkins Slaves
>Reporter: Barak Korren
>Assignee: infra
>
> Since we only have one s390x slave, it is currently attached to both the
> staging and the production CI systems, and while they use separate user
> accounts, it turns out this is not enough to isolate them from one another.
> The are several issues that are caused by this configuration:
> # Tests that allocate a fixed network port can fail if they are run by both
> systems at the same time - this happens in practice when sending Python
> patchs to the '{{jenkins}}' repo because the {{mirror_client.py}} tests start
> a web server on port 8675.
> # The {{mock_cleanup.sh}} script that is being run by one system can time out
> trying to umount things from a mock environment that was created and is being
> used by the other system.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100118)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/OXRSUWQOXZ7RHO6OEBA3Q5QLF64CJFFT/
[JIRA] (OVIRT-2252) The s390x slave is used in parallel by both the staging and the production CI systems
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39149#comment-39149
]
Barak Korren commented on OVIRT-2252:
-
we cannot, because of the way stdciv2 works, if we drop the slave, the jobs for
the 'jenkins' repo will never finish.
I wouldn't want to just drop the 'jenkins' repo jobs, because that is the only
way we have ATM to pre-test some things.
I think at this point the solution we should aim for is to containerize the
s390x host, that would resolve this issue and would enable us to drop support
for not-containerized hosts in the future. I actually had a chat with the
host's maintainer a while ago, and he was willing to install docker there. The
intiative died out because I did not have time to push it further.
> The s390x slave is used in parallel by both the staging and the production CI
> systems
> -
>
> Key: OVIRT-2252
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2252
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Components: Jenkins Slaves
>Reporter: Barak Korren
>Assignee: infra
>
> Since we only have one s390x slave, it is currently attached to both the
> staging and the production CI systems, and while they use separate user
> accounts, it turns out this is not enough to isolate them from one another.
> The are several issues that are caused by this configuration:
> # Tests that allocate a fixed network port can fail if they are run by both
> systems at the same time - this happens in practice when sending Python
> patchs to the '{{jenkins}}' repo because the {{mirror_client.py}} tests start
> a web server on port 8675.
> # The {{mock_cleanup.sh}} script that is being run by one system can time out
> trying to umount things from a mock environment that was created and is being
> used by the other system.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100099)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/44Q6QHOXI4J4A4I7GZZQGN4TAWQ6AK7V/
[JIRA] (OVIRT-2252) The s390x slave is used in parallel by both the staging and the production CI systems
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39148#comment-39148
]
Eyal Edri commented on OVIRT-2252:
--
Can we drop it from staging? its best to have one working slave on production
and if we really need it on staging, we can ask for one more
> The s390x slave is used in parallel by both the staging and the production CI
> systems
> -
>
> Key: OVIRT-2252
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2252
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Components: Jenkins Slaves
>Reporter: Barak Korren
>Assignee: infra
>
> Since we only have one s390x slave, it is currently attached to both the
> staging and the production CI systems, and while they use separate user
> accounts, it turns out this is not enough to isolate them from one another.
> The are several issues that are caused by this configuration:
> # Tests that allocate a fixed network port can fail if they are run by both
> systems at the same time - this happens in practice when sending Python
> patchs to the '{{jenkins}}' repo because the {{mirror_client.py}} tests start
> a web server on port 8675.
> # The {{mock_cleanup.sh}} script that is being run by one system can time out
> trying to umount things from a mock environment that was created and is being
> used by the other system.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100099)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ROZJNCJGPWSP3UFTBAOBXJ2YAOO7TSPB/
[JIRA] (OVIRT-2252) The s390x slave is used in parallel by both the staging and the production CI systems
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barak Korren updated OVIRT-2252:
Component/s: (was: CI Mirrors)
Jenkins Slaves
> The s390x slave is used in parallel by both the staging and the production CI
> systems
> -
>
> Key: OVIRT-2252
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2252
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Components: Jenkins Slaves
>Reporter: Barak Korren
>Assignee: infra
>
> Since we only have one s390x slave, it is currently attached to both the
> staging and the production CI systems, and while they use separate user
> accounts, it turns out this is not enough to isolate them from one another.
> The are several issues that are caused by this configuration:
> # Tests that allocate a fixed network port can fail if they are run by both
> systems at the same time - this happens in practice when sending Python
> patchs to the '{{jenkins}}' repo because the {{mirror_client.py}} tests start
> a web server on port 8675.
> # The {{mock_cleanup.sh}} script that is being run by one system can time out
> trying to umount things from a mock environment that was created and is being
> used by the other system.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100087)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/KA7JPL6XATXVP2FV2O6YIN3E7GD65WN4/
[JIRA] (OVIRT-2252) The s390x slave is used in parallel by both the staging and the production CI systems
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barak Korren updated OVIRT-2252:
Description: "Since we only have one s390x slave, it is currently attached
to both the staging and the production CI systems, and while they use separate
user accounts, it turns out this is not enough to isolate them from one
another.\r\n\r\nThe are several issues that are caused by this
configuration:\r\n# Tests that allocate a fixed network port can fail if they
are run by both systems at the same time - this happens in practice when
sending Python patchs to the '{{jenkins}}' repo because the
{{mirror_client.py}} tests start a web server on port 8675.\r\n# The
{{mock_cleanup.sh}} script that is being run by one system can time out trying
to umount things from a mock environment that was created and is being used by
the other system.\r\n\r\n\r\n\r\n" (was: One of the tests of the CI mirrors
python package is starting up an HTTP server on a fixed port number.
This can cause an issue if the same test is running in parallel on the same
machine. This typically cannot be an issue on the CI infra because it never
runs more then one STDCI thread at a time on a given slave.
This issue does arise specifically on the s390x slave because there the same
slave is used both by the staging and the production CI systems. In that case
the two systems might be trying to run the same test at the same time, so this
can lead to port allocation exceptions.
)
> The s390x slave is used in parallel by both the staging and the production CI
> systems
> -
>
> Key: OVIRT-2252
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2252
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Components: CI Mirrors
>Reporter: Barak Korren
>Assignee: infra
>
> Since we only have one s390x slave, it is currently attached to both the
> staging and the production CI systems, and while they use separate user
> accounts, it turns out this is not enough to isolate them from one another.
> The are several issues that are caused by this configuration:
> # Tests that allocate a fixed network port can fail if they are run by both
> systems at the same time - this happens in practice when sending Python
> patchs to the '{{jenkins}}' repo because the {{mirror_client.py}} tests start
> a web server on port 8675.
> # The {{mock_cleanup.sh}} script that is being run by one system can time out
> trying to umount things from a mock environment that was created and is being
> used by the other system.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100087)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/S2KN5WFPMEKBWVDJETAJ37VAIMXURMOT/
[JIRA] (OVIRT-2252) The s390x slave is used in parallel by both the staging and the production CI systems
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barak Korren updated OVIRT-2252: Summary: The s390x slave is used in parallel by both the staging and the production CI systems (was: Rece condition in python tests in jenkins repo on s390x) > The s390x slave is used in parallel by both the staging and the production CI > systems > - > > Key: OVIRT-2252 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2252 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: CI Mirrors >Reporter: Barak Korren >Assignee: infra > > One of the tests of the CI mirrors python package is starting up an HTTP > server on a fixed port number. > This can cause an issue if the same test is running in parallel on the same > machine. This typically cannot be an issue on the CI infra because it never > runs more then one STDCI thread at a time on a given slave. > This issue does arise specifically on the s390x slave because there the same > slave is used both by the staging and the production CI systems. In that case > the two systems might be trying to run the same test at the same time, so > this can lead to port allocation exceptions. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087) ___ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/YY6ZMI26KRYT4OCVED64PIFTNDXJUCRT/
