[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377812#comment-16377812
]
Steve Rowe edited comment on LUCENE-8106 at 2/26/18 11:59 PM:
--
{quote}Sarowe's Jenkins uses ANT_OPTS environment variable. But this is
something completely different: It is just options that are passed to ANT's JVM
(the JVM that runs the build scripts). Those won't be passed to test runners,
as not even the build.xml would see them. Contrary, in the python script they
are passed like other ant arguments, which is wrong. [...] The ANT_OPTS stuff
should be removed from the script, it has nothing to do with reproducing tests
{quote}
Thanks, this is done.
I also did the following:
* fixed a path-based regex that was triggering failures on Windows (fixed to
handle backslashes).
* changed the windows batch script to only attempt to move directories if they
exist (missing directories were causing script failures)
* added the repro script to the 7.x windows job
I'll keep an eye on the windows builds, hopefully they're fully working after
these changes.
was (Author: steve_rowe):
bq. Sarowe's Jenkins uses ANT_OPTS environment variable. But this is something
completely different: It is just options that are passed to ANT's JVM (the JVM
that runs the build scripts). Those won't be passed to test runners, as not
even the build.xml would see them. Contrary, in the python script they are
passed like other ant arguments, which is wrong. [...] The ANT_OPTS stuff
should be removed from the script, it has nothing to do with reproducing tests
Thanks, this is done.
I also did the following:
* fixed a path-based regex that was triggering failures on Windows (fixed to
handle backslashes).
* changed the windows batch script to only attempt to move directories if they
exist (missing directories were causing script failures)
* added the repro script to the 7.x windows job
I'll keep an eye on the windows builds, hopefully they're fully working after
these changes.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375511#comment-16375511
]
Uwe Schindler edited comment on LUCENE-8106 at 2/24/18 11:35 AM:
-
Hi patch looks fine, but it's not fully correct:
- Policeman and ASF jenkins use "-Dargs=...". Those arguments are accessible
from inside Ant and are also passed to the underlying test JVM runners.
- Sarowe's Jenkins uses {{ANT_OPTS}} environment variable. But this is
something completely different: It is just options that are passed to ANT's JVM
(the JVM that runs the build scripts). Those won't be passed to test runners,
as not even the build.xml would see them. Contrary, in the python script they
are passed like other ant arguments, which is wrong.
The ANT_OPTS stuff should be removed from the script, it has nothing to do with
reproducing tests. It's just some tuning of Ant's JVM (which should no longer
necessary for Sarowe Jenkins, unless it runs with Java 7). If Jenkins would use
{{ANT_ARGS}} env var instead, then you could pass it.
See documentation here: https://ant.apache.org/manual/running.html
was (Author: thetaphi):
Hi patch looks fine, but it's not fully correct:
- Policeman and ASF jenkins use "-Dargs=...". Those arguments are passed to the
underlying test JVM runners.
- Sarowe's Jenkins uses {{ANT_OPTS}} environment variable. But this is
something completely different. It is just options that are passed to ANT's JVM
(the JVM that runs the build scripts). Those won't be passed to test runners.
Contrary, in the python script they are passed like other ant arguments, which
is wrong.
The ANT_OPTS stuff should be removed from the script, it has nothing to do with
reproducing tests. It's just some tuning of Ant's JVM (which should no longer
necessary for Sarowe Jenkins, unless it runs with Java 7). If Jenkins would use
{{ANT_ARGS}} env var instead, then you could pass it.
See documentation here: https://ant.apache.org/manual/running.html
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375511#comment-16375511
]
Uwe Schindler edited comment on LUCENE-8106 at 2/24/18 11:34 AM:
-
Hi patch looks fine, but it's not fully correct:
- Policeman and ASF jenkins use "-Dargs=...". Those arguments are passed to the
underlying test JVM runners.
- Sarowe's Jenkins uses {{ANT_OPTS}} environment variable. But this is
something completely different. It is just options that are passed to ANT's JVM
(the JVM that runs the build scripts). Those won't be passed to test runners.
Contrary, in the python script they are passed like other ant arguments, which
is wrong.
The ANT_OPTS stuff should be removed from the script, it has nothing to do with
reproducing tests. It's just some tuning of Ant's JVM (which should no longer
necessary for Sarowe Jenkins, unless it runs with Java 7). If Jenkins would use
{{ANT_ARGS}} env var instead, then you could pass it.
See documentation here: https://ant.apache.org/manual/running.html
was (Author: thetaphi):
Hi patch looks fine, but it's not fully correct:
- Policeman and ASF jenkins use "-Dargs=...". Those arguments are passed to the
usndelying test JVM runners.
- Sarowe's Jenkins uses {{ANT_OPTS}} environment variable. But this is
something completely different. It is just options that are passed to ANT's JVM
(the JVM that runs the build scripts). Those won't be passed to test runners.
Contrary, in the python script they are passed like other ant arguments, which
is wrong.
The ANT_OPTS stuff should be removed from the script, it has nothing to do with
reproducing tests. It's just some tuning of Ant's JVM (which should no longer
necessary for Sarowe Jenkins, unless it runs with Java 7). If Jenkins would use
{{ANT_ARGS}} env var instead, then you could pass it.
See documentation here: https://ant.apache.org/manual/running.html
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374051#comment-16374051
]
Uwe Schindler edited comment on LUCENE-8106 at 2/23/18 7:55 AM:
Ok done! I replaced all scripts by:
{code:bash}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE "${BUILD_URL}consoleText"
if grep -q 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON3_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-git
file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}
was (Author: thetaphi):
Ok done: I replaced all scripty by:
{code:bash}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE "${BUILD_URL}consoleText"
if grep -q 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON3_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-git
file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374043#comment-16374043
]
Uwe Schindler edited comment on LUCENE-8106 at 2/23/18 7:51 AM:
bq. Powershell is now there! I think the biggest problem on Windows is also to
read the properties file, as it uses escaping for backslashes I am thinking
about a better solution here. Unfortunately you can't add Python as a "Tool" so
you can use "Tools Environment" to add a variable. Digging about that...
I have a solution: You can set environment variables per node in Jenkins
config. I would just set a variable PYTHON3_EXE and remove the cryptic greps on
lucene.build.properties.
was (Author: thetaphi):
bq. Powershell is now there! I think the biggest problem on Windows is also to
read the properties file, as it uses escaping for backslashes I am thinking
about a better solution here. Unfortunately you can't add Python as a "Tool" so
you can use "Tools Environment" to add a variable. Digging about that...
I have a solution: You can set environment variables per node in Jenkins
config. I would just set a variable PYTHON3_EXECUTABLE and remove the cryptic
greps on lucene.build.properties.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374033#comment-16374033
]
Uwe Schindler edited comment on LUCENE-8106 at 2/23/18 7:25 AM:
Powershell is now there! I think the biggest problem on Windows is also to read
the properties file, as it uses escaping for backslashes :-( I am thinking
about a better solution here. Unfortunately you can't add Python as a "Tool" so
you can use "Tools Environment" to add a variable. Digging about that...
was (Author: thetaphi):
Powershell is there! I think the biggest problem on Windows is also to read the
properties file, as it uses escaping for backslashes :-( I am thinking about a
better solution here. Unfortunately you can't add Python as a "Tool" so you can
use "Tools Environment" to add a variable. Digging about that...
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372126#comment-16372126
]
Uwe Schindler edited comment on LUCENE-8106 at 2/21/18 10:39 PM:
-
That's now the Jenkins script:
{code:bash}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE "${BUILD_URL}consoleText"
if grep --quiet 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -z $PYTHON32_EXE ] && PYTHON32_EXE=python3
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -n $GIT_EXE ] && export PATH=$GIT_EXE:$PATH
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON32_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}
was (Author: thetaphi):
That's now the Jenkins script:
{code:bash}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE "$BUILD_URL/consoleText"
if grep --quiet 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -z $PYTHON32_EXE ] && PYTHON32_EXE=python3
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -n $GIT_EXE ] && export PATH=$GIT_EXE:$PATH
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON32_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369834#comment-16369834
]
Uwe Schindler edited comment on LUCENE-8106 at 2/20/18 9:16 AM:
On Linux, they don't have entries for those tools, so it falls back to
build.xml's default behaviour. git.exe is undefined everywhere - except the
master Jenkins (Linux), which has GIT, but no other slave machine has GIT
installed. They only use Jenkins' internal JGit client, no command line. There
is no need to have Git for running Lucene builds (except packaging and jar
version numbers, but Policeman does not use this - because it's optional for
test builds).
{noformat}
jenkins@serv1:~$ pwd
/home/jenkins
jenkins@serv1:~$ cat lucene.build.properties
tests.jvms=3
tests.multiplier=3
{noformat}
FYI, on Windows it looks like that:
{noformat}
C:\Users\jenkins>type lucene.build.properties
python32.exe=C:\\Python35\\python.exe
perl.exe=C:\\Perl64\\bin\\perl.exe
tests.jvms=2
C:\Users\jenkins>git
'git' is not recognized as an internal or external command,
operable program or batch file.
{noformat}
Same on MacOSX and Solaris.
Sorry for not being responsive, I am a bit busy today. Can you send me your SSH
key privately? I will put them on all 4 machines into jenkins home dir.
was (Author: thetaphi):
On Linux, they don't have entries for those tools, so it falls back to
build.xml's default behaviour. git.exe is undefined everywhere as except the
master Jenkins (Linux) no slave machine has GIT installed. They only use
Jenkins' internal JGit client, no command line.
{noformat}
jenkins@serv1:~$ pwd
/home/jenkins
jenkins@serv1:~$ cat lucene.build.properties
tests.jvms=3
tests.multiplier=3
{noformat}
FYI, on Windows it looks like that:
{noformat}
C:\Users\jenkins>type lucene.build.properties
python32.exe=C:\\Python35\\python.exe
perl.exe=C:\\Perl64\\bin\\perl.exe
tests.jvms=2
C:\Users\jenkins>git
'git' is not recognized as an internal or external command,
operable program or batch file.
{noformat}
Same on MacOSX and Solaris.
Sorry for not being responsive, I am a bit busy today. Can you send me your SSH
key privately? I will put them on all 4 machines into jenkins home dir.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368370#comment-16368370
]
Steve Rowe edited comment on LUCENE-8106 at 2/20/18 1:22 AM:
-
{quote}
bq. I'd include the repro stuff into the main build job somehow!
I'll try making it a build step (instead of a post-build action).
{quote}
I added the following as an "Execute shell" build step to the
{{Lucene-Solr-master-Linux}} project - we'll see how it goes:
{noformat}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE
https://jenkins.thetaphi.de/job/$JOB_NAME/$BUILD_NUMBER/consoleText
if grep --quiet 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
PATH=$GIT_EXE:$PATH $PYTHON32_EXE -u
dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{noformat}
was (Author: steve_rowe):
{quote}
bq. I'd include the repro stuff into the main build job somehow!
I'll try making it a build step (instead of a post-build action).
{quote}
I added the following as an "Execute shell" build step to the
{{Lucene-Solr-master-Linux}} project - we'll see how it goes:
{noformat}
set -x # Log commands
if grep --quiet 'reproduce with' ../builds/$BUILD_NUMBER/log ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
cd ..
PARENT_DIR=`pwd`
cd workspace
PATH=$GIT_EXE:$PATH $PYTHON32_EXE -u
dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
file://$PARENT_DIR/builds/$BUILD_NUMBER/log
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{noformat}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368370#comment-16368370
]
Steve Rowe edited comment on LUCENE-8106 at 2/17/18 10:09 PM:
--
{quote}
bq. I'd include the repro stuff into the main build job somehow!
I'll try making it a build step (instead of a post-build action).
{quote}
I added the following as an "Execute shell" build step to the
{{Lucene-Solr-master-Linux}} project - we'll see how it goes:
{noformat}
set -x # Log commands
if grep --quiet 'reproduce with' ../builds/$BUILD_NUMBER/log ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
cd ..
PARENT_DIR=`pwd`
cd workspace
PATH=$GIT_EXE:$PATH $PYTHON32_EXE -u
dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
file://$PARENT_DIR/builds/$BUILD_NUMBER/log
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{noformat}
was (Author: steve_rowe):
{quote}
bq. I'd include the repro stuff into the main build job somehow!
I'll try making it a build step (instead of a post-build action).
{quote}
I added the following as an "Execute shell" build step to the
{{Lucene-Solr-master-Linux}} project - we'll see how it goes:
{noformat}
set -x # Log commands
if grep 'reproduce with' ../builds/$BUILD_NUMBER/log ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
PATH=$GIT_EXE:$PATH $PYTHON32_EXE -u
dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
../builds/$BUILD_NUMBER/log
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{noformat}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368370#comment-16368370
]
Steve Rowe edited comment on LUCENE-8106 at 2/17/18 9:37 PM:
-
{quote}
bq. I'd include the repro stuff into the main build job somehow!
I'll try making it a build step (instead of a post-build action).
{quote}
I added the following as an "Execute shell" build step to the
{{Lucene-Solr-master-Linux}} project - we'll see how it goes:
{noformat}
set -x # Log commands
if grep 'reproduce with' ../builds/$BUILD_NUMBER/log ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
PATH=$GIT_EXE:$PATH $PYTHON32_EXE -u
dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
../builds/$BUILD_NUMBER/log
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{noformat}
was (Author: steve_rowe):
{quote}
bq. I'd include the repro stuff into the main build job somehow!
I'll try making it a build step (instead of a post-build action).
{quote}
I added the following as an "Execute shell" build step to the
{{Lucene-Solr-master-Linux}} project - we'll see how it goes:
{noformat}
set -x # Log commands
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*=" ~/lucene.build.properties |
cut -d'=' -f2`
PATH=$GIT_EXE:$PATH $PYTHON32_EXE -u
dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
../builds/$BUILD_NUMBER/log
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
{noformat}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368178#comment-16368178
]
Uwe Schindler edited comment on LUCENE-8106 at 2/17/18 10:41 AM:
-
Hi Steve,
I think it should work. The problem you havre is much simpler:
bq. When I tried to configure the Lucene-Solr-master-Linux project to trigger
building the new project, the "Build other projects" item in the "Add
post-build action" menu is greyed out.
Its grayed out because the master build job already triggers other jobs (the
Jenkins setup on Policeman is a chain of projects). Just scroll up and look for
the input line! You may clean it up, because there are some disabled/deleted
Jobs in it.
Currently you cannot parameterize the randomized JDK. So on reproducing, you
get another JDK/GC/bitness setting - sorry. Therefore I'd include the repro
stuff into the main build job somehow!
was (Author: thetaphi):
Hi Steve,
I think it should work. The problem you havre is much simpler:
bq. When I tried to configure the Lucene-Solr-master-Linux project to trigger
building the new project, the "Build other projects" item in the "Add
post-build action" menu is greyed out.
Its grayed out because the master build job already triggers other jobs (the
Jenkins setup on Policeman is a chain of projects). Just scroll up and look for
the input line!
Currently you cannot parameterize the randomized JDK. So on reproducing, you
get another JDK/GC/bitness setting - sorry. Therefore I'd include the repro
stuff into the main build job somehow!
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366831#comment-16366831
]
Uwe Schindler edited comment on LUCENE-8106 at 2/16/18 10:47 AM:
-
Hi,
it might work out of box on the master slave, as it's running linux, but
definitely not on the others.
In general this Jenkins does not use native operating system's GIT at all, it
uses Jenkins JGIT plugin to handle checkouts. So git is not available on the
slaves with operating systems != linux. Python is available, but requires the
entries in {{lucene.build.properties}} file, as on the other operating systems
this is not in "PATH". That's the way how it's also setup on ASF jenkins (going
back to times of FreeBSD).
I'd like to make the Jenkins build as generic as possible, so everything is
configured per node in lucene.build.properties, so the script should read it
like the main build scripts.
IMHO, I'd add a top-level jenkins job that runs the python script (similar to
the smoke tester) using the standardized build properties, so it des not
require separate per-node-config:
{noformat}
$ ant run-jenkins-repro
{noformat}
This won't solve the missing "git" command problem.
Uwe
was (Author: thetaphi):
Hi,
it might work out of box on the master slave, as it's running linux, but
definitely not on the others.
In general this Jenkins does not use native operating system's GIT at all, it
uses Jenkins JGIT plugin to handle checkouts. So git is not available on the
slaves with operating systems != linux. Python is available, but requires the
entries in lucene.build.xml properties file, as on the other operating systems
this is not in "PATH". That's the way how it's also setup on ASF jenkins.
I'd like to make the Jenkins build as generic as possible, so everything is
configured per node in lucene.build.properties, so the scriupt should read it
like the main jenkins.
IMHO, I'd add a top-level jenkins job that runs the python script (similar to
the smoke tester) using the standardized build properties, so it des not
require separate per-node-config:
{noformat}
$ ant run-jenkins-repro
{noformat}
This won't solve the missing "git" command problem.
Uwe
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366054#comment-16366054
]
Steve Rowe edited comment on LUCENE-8106 at 2/15/18 6:23 PM:
-
I've set up a new job using the repro script on ASF Jenkins:
[https://builds.apache.org/view/L/view/Lucene/job/Lucene-Solr-repro/], and
configured the Test, NightlyTest and SmokeTester jobs on master and branch_7x
to trigger the new job for Unstable/Failure runs.
[~thetaphi], what do you think of setting up a job for this on your Jenkins?
I'm not sure how to set up the same environment (OS/JVM) for the job though...
Maybe instead of triggering another job to run the repro script, the existing
jobs could be configured to run the script as a build step?
was (Author: steve_rowe):
I've set up a new job using the repro script on ASF Jenkins:
[https://builds.apache.org/view/L/view/Lucene/job/Lucene-Solr-repro/], and
configured it to be triggered by Unstable/Failure results for Test, NightlyTest
and SmokeTester jobs on master and branch_7x.
[~thetaphi], what do you think of setting up a job for this on your Jenkins?
I'm not sure how to set up the same environment (OS/JVM) for the job though...
Maybe instead of triggering another job to run the repro script, the existing
jobs could be configured to run the script as a build step?
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363479#comment-16363479
]
Steve Rowe edited comment on LUCENE-8106 at 2/14/18 4:45 AM:
-
Reopening to add some more functionality to the script:
# Add cmdline option parsing
# Run {{git fetch}} by default prior to running tests, and allow opt-out (for
Jenkins) via cmdline option {{--no-fetch}}
# Add cmdline option for test iterations, defaulting to 5
# Sort failure report by reverse failure count, so that highest counts are on
the bottom, with secondary sort on test suite name
# When all iterations fail for a test, re-test at branch tip (with the seed)
# When a test also fails 100% at branch tip, re-attempt without the seed
was (Author: steve_rowe):
Reopening to add some more functionality to the script:
# Add cmdline option parsing
# Run {{git fetch}} by default prior to running tests, and allow opt-out (for
Jenkins) via cmdline option {{--no-fetch}}
# Add cmdline option for test iterations, defaulting to 5
# Sort failure report by reverse failure count, so that highest counts are on
the bottom, with secondary sort on test suite name
# When all iterations fail for a test, re-test at branch tip with the seed
# When a test also fails at branch tip, re-attempt without the seed
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
