Re: Frequent failures in standby test
In the meantime, I configured those tests not to run when they are scheduled under a "beam" node. I will send a separate mail to builds@ and put oak-dev@ in CC. 2016-12-08 11:23 GMT+01:00 Robert Munteanu: > Hi, > > On Thu, 2016-12-08 at 10:01 +0100, Francesco Mari wrote: >> I analysed the latest builds to figure out what's going on with some >> tests related to Cold Standby. It seems that the problem is >> restricted >> to some Jenkins nodes - see [1] for more details. Can we configure >> the >> Jenkins job not to run on nodes labelled "beam"? >> >> [1]: https://issues.apache.org/jira/browse/OAK- >> 5017?focusedCommentId=15731590=com.atlassian.jira.plugin.system. >> issuetabpanels:comment-tabpanel#comment-15731590 > > The job is configured to only run on 'ubuntu' nodes, but it seems that > this restriction is not taken into account. The following Jenkins issue > might be related > > https://issues.jenkins-ci.org/browse/JENKINS-23459 > > Perhaps a question better suited to builds@ ? They would be interested > in proper resource utilisation. > > Robert > >> >> 2016-11-25 13:25 GMT+01:00 Francesco Mari : >> > Every failure in build 1298 is due to "java.net.BindException: >> > Address >> > already in use". According to the build log, the port wasn't bound >> > by >> > a server started during a test. The first failing test >> > (ExternalSharedStoreIT) was also the first test in the build to >> > start >> > a server. The issue persisted for some tests (the whole range of >> > tests >> > in ExternalSharedStoreIT and the first test in FailoverSslTestIT), >> > so >> > I can't deduce any random behaviour. Once the port (presumingly >> > bound >> > to an external process) was released, every other integration test >> > worked without any problem. >> > >> > 2016-11-25 5:38 GMT+01:00 Chetan Mehrotra > > m>: >> > > Per https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20mat >> > > rix/1298/ >> > > the test again failed but mostly on Jdk 1.7. The test on Jdk 1.8 >> > > looks >> > > like passed. >> > > Chetan Mehrotra >> > > >> > > >> > > On Tue, Nov 22, 2016 at 12:48 PM, Chetan Mehrotra >> > > wrote: >> > > > They are from oak-segment-tar. See >> > > > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matri >> > > > x/1295/#showFailuresLink >> > > > Chetan Mehrotra >> > > > >> > > > >> > > > On Tue, Nov 22, 2016 at 12:42 PM, Francesco Mari >> > > > wrote: >> > > > > Are those from oak-tarmk-standby or oak-segment-tar? >> > > > > >> > > > > 2016-11-22 6:11 GMT+01:00 Chetan Mehrotra > > > > > ail.com>: >> > > > > > Hi Team, >> > > > > > >> > > > > > Since last 4-6 builds I am seeing a recurring failure of >> > > > > > few test in >> > > > > > standby module >> > > > > > >> > > > > > * FailoverIPRangeIT >> > > > > > * ExternalPrivateStoreIT >> > > > > > * StandbyTestIT >> > > > > > >> > > > > > Probably something to be looked into >> > > > > > >> > > > > > Chetan Mehrotra >
Re: Frequent failures in standby test
Hi, On Thu, 2016-12-08 at 10:01 +0100, Francesco Mari wrote: > I analysed the latest builds to figure out what's going on with some > tests related to Cold Standby. It seems that the problem is > restricted > to some Jenkins nodes - see [1] for more details. Can we configure > the > Jenkins job not to run on nodes labelled "beam"? > > [1]: https://issues.apache.org/jira/browse/OAK- > 5017?focusedCommentId=15731590=com.atlassian.jira.plugin.system. > issuetabpanels:comment-tabpanel#comment-15731590 The job is configured to only run on 'ubuntu' nodes, but it seems that this restriction is not taken into account. The following Jenkins issue might be related https://issues.jenkins-ci.org/browse/JENKINS-23459 Perhaps a question better suited to builds@ ? They would be interested in proper resource utilisation. Robert > > 2016-11-25 13:25 GMT+01:00 Francesco Mari: > > Every failure in build 1298 is due to "java.net.BindException: > > Address > > already in use". According to the build log, the port wasn't bound > > by > > a server started during a test. The first failing test > > (ExternalSharedStoreIT) was also the first test in the build to > > start > > a server. The issue persisted for some tests (the whole range of > > tests > > in ExternalSharedStoreIT and the first test in FailoverSslTestIT), > > so > > I can't deduce any random behaviour. Once the port (presumingly > > bound > > to an external process) was released, every other integration test > > worked without any problem. > > > > 2016-11-25 5:38 GMT+01:00 Chetan Mehrotra > m>: > > > Per https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20mat > > > rix/1298/ > > > the test again failed but mostly on Jdk 1.7. The test on Jdk 1.8 > > > looks > > > like passed. > > > Chetan Mehrotra > > > > > > > > > On Tue, Nov 22, 2016 at 12:48 PM, Chetan Mehrotra > > > wrote: > > > > They are from oak-segment-tar. See > > > > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matri > > > > x/1295/#showFailuresLink > > > > Chetan Mehrotra > > > > > > > > > > > > On Tue, Nov 22, 2016 at 12:42 PM, Francesco Mari > > > > wrote: > > > > > Are those from oak-tarmk-standby or oak-segment-tar? > > > > > > > > > > 2016-11-22 6:11 GMT+01:00 Chetan Mehrotra > > > > ail.com>: > > > > > > Hi Team, > > > > > > > > > > > > Since last 4-6 builds I am seeing a recurring failure of > > > > > > few test in > > > > > > standby module > > > > > > > > > > > > * FailoverIPRangeIT > > > > > > * ExternalPrivateStoreIT > > > > > > * StandbyTestIT > > > > > > > > > > > > Probably something to be looked into > > > > > > > > > > > > Chetan Mehrotra
Re: Frequent failures in standby test
I analysed the latest builds to figure out what's going on with some tests related to Cold Standby. It seems that the problem is restricted to some Jenkins nodes - see [1] for more details. Can we configure the Jenkins job not to run on nodes labelled "beam"? [1]: https://issues.apache.org/jira/browse/OAK-5017?focusedCommentId=15731590=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15731590 2016-11-25 13:25 GMT+01:00 Francesco Mari: > Every failure in build 1298 is due to "java.net.BindException: Address > already in use". According to the build log, the port wasn't bound by > a server started during a test. The first failing test > (ExternalSharedStoreIT) was also the first test in the build to start > a server. The issue persisted for some tests (the whole range of tests > in ExternalSharedStoreIT and the first test in FailoverSslTestIT), so > I can't deduce any random behaviour. Once the port (presumingly bound > to an external process) was released, every other integration test > worked without any problem. > > 2016-11-25 5:38 GMT+01:00 Chetan Mehrotra : >> Per https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1298/ >> the test again failed but mostly on Jdk 1.7. The test on Jdk 1.8 looks >> like passed. >> Chetan Mehrotra >> >> >> On Tue, Nov 22, 2016 at 12:48 PM, Chetan Mehrotra >> wrote: >>> They are from oak-segment-tar. See >>> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1295/#showFailuresLink >>> Chetan Mehrotra >>> >>> >>> On Tue, Nov 22, 2016 at 12:42 PM, Francesco Mari >>> wrote: Are those from oak-tarmk-standby or oak-segment-tar? 2016-11-22 6:11 GMT+01:00 Chetan Mehrotra : > Hi Team, > > Since last 4-6 builds I am seeing a recurring failure of few test in > standby module > > * FailoverIPRangeIT > * ExternalPrivateStoreIT > * StandbyTestIT > > Probably something to be looked into > > Chetan Mehrotra
Re: Frequent failures in standby test
Every failure in build 1298 is due to "java.net.BindException: Address already in use". According to the build log, the port wasn't bound by a server started during a test. The first failing test (ExternalSharedStoreIT) was also the first test in the build to start a server. The issue persisted for some tests (the whole range of tests in ExternalSharedStoreIT and the first test in FailoverSslTestIT), so I can't deduce any random behaviour. Once the port (presumingly bound to an external process) was released, every other integration test worked without any problem. 2016-11-25 5:38 GMT+01:00 Chetan Mehrotra: > Per https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1298/ > the test again failed but mostly on Jdk 1.7. The test on Jdk 1.8 looks > like passed. > Chetan Mehrotra > > > On Tue, Nov 22, 2016 at 12:48 PM, Chetan Mehrotra > wrote: >> They are from oak-segment-tar. See >> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1295/#showFailuresLink >> Chetan Mehrotra >> >> >> On Tue, Nov 22, 2016 at 12:42 PM, Francesco Mari >> wrote: >>> Are those from oak-tarmk-standby or oak-segment-tar? >>> >>> 2016-11-22 6:11 GMT+01:00 Chetan Mehrotra : Hi Team, Since last 4-6 builds I am seeing a recurring failure of few test in standby module * FailoverIPRangeIT * ExternalPrivateStoreIT * StandbyTestIT Probably something to be looked into Chetan Mehrotra
Re: Frequent failures in standby test
Per https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1298/ the test again failed but mostly on Jdk 1.7. The test on Jdk 1.8 looks like passed. Chetan Mehrotra On Tue, Nov 22, 2016 at 12:48 PM, Chetan Mehrotrawrote: > They are from oak-segment-tar. See > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1295/#showFailuresLink > Chetan Mehrotra > > > On Tue, Nov 22, 2016 at 12:42 PM, Francesco Mari > wrote: >> Are those from oak-tarmk-standby or oak-segment-tar? >> >> 2016-11-22 6:11 GMT+01:00 Chetan Mehrotra : >>> Hi Team, >>> >>> Since last 4-6 builds I am seeing a recurring failure of few test in >>> standby module >>> >>> * FailoverIPRangeIT >>> * ExternalPrivateStoreIT >>> * StandbyTestIT >>> >>> Probably something to be looked into >>> >>> Chetan Mehrotra
Re: Frequent failures in standby test
They are from oak-segment-tar. See https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1295/#showFailuresLink Chetan Mehrotra On Tue, Nov 22, 2016 at 12:42 PM, Francesco Mariwrote: > Are those from oak-tarmk-standby or oak-segment-tar? > > 2016-11-22 6:11 GMT+01:00 Chetan Mehrotra : >> Hi Team, >> >> Since last 4-6 builds I am seeing a recurring failure of few test in >> standby module >> >> * FailoverIPRangeIT >> * ExternalPrivateStoreIT >> * StandbyTestIT >> >> Probably something to be looked into >> >> Chetan Mehrotra
Re: Frequent failures in standby test
Are those from oak-tarmk-standby or oak-segment-tar? 2016-11-22 6:11 GMT+01:00 Chetan Mehrotra: > Hi Team, > > Since last 4-6 builds I am seeing a recurring failure of few test in > standby module > > * FailoverIPRangeIT > * ExternalPrivateStoreIT > * StandbyTestIT > > Probably something to be looked into > > Chetan Mehrotra
Frequent failures in standby test
Hi Team, Since last 4-6 builds I am seeing a recurring failure of few test in standby module * FailoverIPRangeIT * ExternalPrivateStoreIT * StandbyTestIT Probably something to be looked into Chetan Mehrotra