[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513458#comment-16513458
]
Adrien Grand commented on LUCENE-8106:
--
[~steve_rowe] Can this issue be marked as resolved again?
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.part5.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419483#comment-16419483
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 06e43084a57d29fd9dc176dfec145c148c2b5e50 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=06e4308 ]
LUCENE-8106: add missing import
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.3, master (8.0)
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.part5.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419484#comment-16419484
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 1ce72537b8b7577657c275dd7a6bfbb081392575 in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1ce7253 ]
LUCENE-8106: add missing import
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.3, master (8.0)
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.part5.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406476#comment-16406476
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 0c61c857c89ba0003890fc8a2b5378801b57e0ee in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0c61c85 ]
LUCENE-8106: Handle IncompleteRead exceptions while downloading the Jenkins log
by retrying a limited number of times
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.3, master (8.0)
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.part5.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406474#comment-16406474
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 7ea6569121d78e3683f658b962b1ea1ec9744daa in lucene-solr's branch
refs/heads/branch_7_3 from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7ea6569 ]
LUCENE-8106: Handle IncompleteRead exceptions while downloading the Jenkins log
by retrying a limited number of times
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.3, master (8.0)
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.part5.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406475#comment-16406475
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 7735f2c911c8c3fb7cc02f9ab9c8dbe4fb70f9cd in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7735f2c ]
LUCENE-8106: Handle IncompleteRead exceptions while downloading the Jenkins log
by retrying a limited number of times
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.3, master (8.0)
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.part5.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406468#comment-16406468
]
Steve Rowe commented on LUCENE-8106:
I attached a patch to address {{IncompleteRead}} exceptions that have been
occurring occasionally during download of the Jenkins log, e.g. from
[https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/492/consoleText]:
{noformat}
[repro] Jenkins log URL:
https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/492/consoleText
[repro] Revision: db1360fac40fe472f881a37d6a0f0187491e308c
[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC"
Traceback (most recent call last):
File "/usr/lib/python3.4/http/client.py", line 614, in _readinto_chunked
chunk_left = self._read_next_chunk_size()
File "/usr/lib/python3.4/http/client.py", line 559, in _read_next_chunk_size
return int(line, 16)
ValueError: invalid literal for int() with base 16: b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "dev-tools/scripts/reproduceJenkinsFailures.py", line 276, in
main()
File "dev-tools/scripts/reproduceJenkinsFailures.py", line 226, in main
tests = fetchAndParseJenkinsLog(config.url)
File "dev-tools/scripts/reproduceJenkinsFailures.py", line 108, in
fetchAndParseJenkinsLog
for rawLine in consoleText:
File "/usr/lib/python3.4/http/client.py", line 500, in read
return super(HTTPResponse, self).read(amt)
File "/usr/lib/python3.4/http/client.py", line 529, in readinto
return self._readinto_chunked(b)
File "/usr/lib/python3.4/http/client.py", line 618, in _readinto_chunked
raise IncompleteRead(bytes(b[0:total_bytes]))
http.client.IncompleteRead: IncompleteRead(0 bytes read)
Build step 'Execute shell' marked build as failure
{noformat}
Committing shortly.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.3, master (8.0)
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.part5.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377812#comment-16377812
]
Steve Rowe commented on LUCENE-8106:
bq. Sarowe's Jenkins uses ANT_OPTS environment variable. But this is something
completely different: It is just options that are passed to ANT's JVM (the JVM
that runs the build scripts). Those won't be passed to test runners, as not
even the build.xml would see them. Contrary, in the python script they are
passed like other ant arguments, which is wrong. [...] The ANT_OPTS stuff
should be removed from the script, it has nothing to do with reproducing tests
Thanks, this is done.
I also did the following:
* fixed a path-based regex that was triggering failures on Windows (fixed to
handle backslashes).
* changed the windows batch script to only attempt to move directories if they
exist (missing directories were causing script failures)
* added the repro script to the 7.x windows job
I'll keep an eye on the windows builds, hopefully they're fully working after
these changes.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377800#comment-16377800
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 606e91c2aeb2cb8678fcd40918e04cd3aef3022c in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=606e91c ]
LUCENE-8106: fix module regex to recognize Windows path backslashes
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377799#comment-16377799
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 67d17644ff85bc4e9dde8d2659e7eac2db3d862d in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=67d1764 ]
LUCENE-8106: stop parsing ANT_OPTS from Jenkins log
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377798#comment-16377798
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 443cc19db2afa28ab258a577d4bbd6e9e0b24b57 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=443cc19 ]
LUCENE-8106: fix module regex to recognize Windows path backslashes
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377801#comment-16377801
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 41af8bd16a4d5151adebe3e4dfe847854ab5bde3 in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=41af8bd ]
LUCENE-8106: stop parsing ANT_OPTS from Jenkins log
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375511#comment-16375511
]
Uwe Schindler commented on LUCENE-8106:
---
Hi patch looks fine, but it's not fully correct:
- Policeman and ASF jenkins use "-Dargs=...". Those arguments are passed to the
usndelying test JVM runners.
- Sarowe's Jenkins uses {{ANT_OPTS}} environment variable. But this is
something completely different. It is just options that are passed to ANT's JVM
(the JVM that runs the build scripts). Those won't be passed to test runners.
Contrary, in the python script they are passed like other ant arguments, which
is wrong.
The ANT_OPTS stuff should be removed from the script, it has nothing to do with
reproducing tests. It's just some tuning of Ant's JVM (which should no longer
necessary for Sarowe Jenkins, unless it runs with Java 7). If Jenkins would use
{{ANT_ARGS}} env var instead, then you could pass it.
See documentation here: https://ant.apache.org/manual/running.html
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375162#comment-16375162
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 07f7a7b5ddd08aa86dd65f31cf475546da084204 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=07f7a7b ]
LUCENE-8106: collect Ant test sysprops from Jenkins log and add them to test
repro lines
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375163#comment-16375163
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 88e00d08894cd58f8cf1f51f9de6f36ce3183031 in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=88e00d0 ]
LUCENE-8106: collect Ant test sysprops from Jenkins log and add them to test
repro lines
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375160#comment-16375160
]
Steve Rowe commented on LUCENE-8106:
bq. Maybe the reproducer script should use a regex to catch the initial ANT
call at the beginning of the log file and extract extra command line arguments
from it. Then it would be universal and also works for ASF jenkins (if we add
similar stuff there, too).
I attached a patch ({{\*\-part4.patch}}) that implements this idea. Committing
shortly.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106-part4.patch, LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374839#comment-16374839
]
Uwe Schindler commented on LUCENE-8106:
---
Another approach that might be worth to investigate: Maybe pass the extra args
separately to the runner JVMs as a sysprop so the repo line can be extended by
"extra stuff" coming form the outside. The extra sysprop with all extra
arguments as a single string is just to work around the problem that you cannot
get all command line args and sysprops easily from inside the JVM. So the ant
script could create a sysprop "tests.repo.extra_args" that contains the {{}} which are passed (mainly the args sysprop). Then the test reproduce
line could include that info, if sysprop is set.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374784#comment-16374784
]
Steve Rowe commented on LUCENE-8106:
bq. Maybe the reproducer script should use a regex to catch the initial ANT
call at the beginning of the log file and extract extra command line arguments
from it. Then it would be universal and also works for ASF jenkins (if we add
similar stuff there, too).
Good idea! I'll look into it.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374776#comment-16374776
]
Uwe Schindler commented on LUCENE-8106:
---
Oh yes, I also noticed this. It does not pass the JVM properties to the runner,
so the reproducing script runs with default garbage collector and compressed
oops. Maybe the reproducer script should use a regex to catch the initial ANT
call at the beginning of the log file and extract extra command line arguments
from it. Then it would be universal and also works for ASF jenkins (if we add
similar stuff there, too).
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374741#comment-16374741
]
Steve Rowe commented on LUCENE-8106:
Thanks [~thetaphi]!
Any thoughts about this question I had for you?:
{quote}
Uwe, I'm working on adding the script to the jobs for MacOS and Solaris
(Windows will need a batch/cmd script instead of a shell script, right?), and I
noticed that the jenkins-hourly Ant build step passes in sysprop
args=${TEST_JVM_ARGS} - am I right in thinking that without these, we're not
reproducing the same as the initial tests? Not sure how to fix - maybe
reproduceJenkinsFailures.py could be made aware of this environment variable
and pass it on in its Ant invocations?
{quote}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374051#comment-16374051
]
Uwe Schindler commented on LUCENE-8106:
---
Ok done: I replaced all scripty by:
{code:bash}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE "${BUILD_URL}consoleText"
if grep -q 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON3_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-git
file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374043#comment-16374043
]
Uwe Schindler commented on LUCENE-8106:
---
bq. Powershell is now there! I think the biggest problem on Windows is also to
read the properties file, as it uses escaping for backslashes I am thinking
about a better solution here. Unfortunately you can't add Python as a "Tool" so
you can use "Tools Environment" to add a variable. Digging about that...
I have a solution: You can set environment variables per node in Jenkins
config. I would just set a variable PYTHON3_EXECUTABLE and remove the cryptic
greps on lucene.build.properties.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374041#comment-16374041
]
Uwe Schindler commented on LUCENE-8106:
---
I replaced the long option "--quiet" by the more portable "-q" one everywhere.
Reals BSD's don't know any long options on grep.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374037#comment-16374037
]
Uwe Schindler commented on LUCENE-8106:
---
Solaris does not like your usage of grep:
{noformat}
+ grep --quiet 'reproduce with' /tmp/tmp.y6aGln
grep: illegal option -- quiet
Usage: grep [-c|-l|-q] -bhinsvw pattern file . . .
+ rm -f /tmp/tmp.y6aGln
{noformat}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374033#comment-16374033
]
Uwe Schindler commented on LUCENE-8106:
---
Powershell is there! I think the biggest problem on Windows is also to read the
properties file, as it uses escaping for backslashes :-( I am thinking about a
better solution here. Unfortunately you can't add Python as a "Tool" so you can
use "Tools Environment" to add a variable. Digging about that...
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373820#comment-16373820
]
Steve Rowe commented on LUCENE-8106:
Uwe:
bq. Windows will need a batch/cmd script instead of a shell script, right?
Actually, I think Jenkins's Powershell plugin is the right thing here, since
Powershell can natively download web files, and knows how to create temporary
files. It's not installed on Policeman Jenkins right now though.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373793#comment-16373793
]
Steve Rowe commented on LUCENE-8106:
bq. I'm working on adding the script to the jobs for MacOS and Solaris
Done for the master branch jobs for these two.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373789#comment-16373789
]
Steve Rowe commented on LUCENE-8106:
Uwe, I'm working on adding the script to the jobs for MacOS and Solaris
(Windows will need a batch/cmd script instead of a shell script, right?), and I
noticed that the {{jenkins-hourly}} Ant build step passes in sysprop
{{args=$\{TEST_JVM_ARGS\}}} - am I right in thinking that without these, we're
not reproducing the same as the initial tests? Not sure how to fix - maybe
{{reproduceJenkinsFailures.py}} could be made aware of this environment
variable and pass it on in its Ant invocations?
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373528#comment-16373528
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit edd54e551d5ae9cd675482bab042b0cce8fb02e2 in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=edd54e5 ]
LUCENE-8106: add cmdline option --no-git to avoid all git usage; replaces
--no-fetch cmdline option
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373527#comment-16373527
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 27f5ca75ad811532e5b46b32ba858b5997ed87b4 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=27f5ca7 ]
LUCENE-8106: add cmdline option --no-git to avoid all git usage; replaces
--no-fetch cmdline option
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373507#comment-16373507
]
Steve Rowe commented on LUCENE-8106:
{quote}
So I have one small question to Steve Rowe: Can't we make a variant of the
script for Jenkins that does not deal with Git at all? For reproduing the
failure directly after running the main build job there is no need to change
branches or merge in changes. We just leave the checkout as-is. As there are no
external forces changing the checkout, why would we need GIT?
So let's add a "replacement" option "\-no\-git" instead of "\-no\-fetch", that
completely disables GIT in the python script.
{quote}
I agree; when the script is invoked in the same run as the failing tests, there
is no need for the script to do anything with GIT. I've attached a patch
(...\-part3.patch) to do this via the replacement {{\-\-no\-git}} cmdline
option. I'll commit shortly, then modify Jenkins jobs to invoke properly.
However, on all other Jenkins besides Policeman, I intend to *not* include
{{\-\-no\-git}} when invoking the script, since it's run as a separate job that
can be scheduled hours or even days after the initial run that produced the
test failures.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106-part3.patch,
> LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372126#comment-16372126
]
Uwe Schindler commented on LUCENE-8106:
---
That's now the Jenkins script:
{code:bash}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE "$BUILD_URL/consoleText"
if grep --quiet 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -z $PYTHON32_EXE ] && PYTHON32_EXE=python3
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -n $GIT_EXE ] && export PATH=$GIT_EXE:$PATH
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON32_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370869#comment-16370869
]
Steve Rowe commented on LUCENE-8106:
Now that the repro stuff is working on the Policeman Jenkins
{{Lucene-Solr-master-Linux}} project, I also added the script ^^ as a build
step to the {{Lucene-Solr-7.x-Linux}} project.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370863#comment-16370863
]
Steve Rowe commented on LUCENE-8106:
Looks like Policeman {{Lucene-Solr-master-Linux}} is working now - see e.g.
[https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21501/consoleText].
Here's the final script; I had to add Jenkins's {{ant}} to the PATH to get it
running:
{noformat}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE
https://jenkins.thetaphi.de/job/$JOB_NAME/$BUILD_NUMBER/consoleText
if grep --quiet 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -z $PYTHON32_EXE ] && PYTHON32_EXE=python3
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -n $GIT_EXE ] && export PATH=$GIT_EXE:$PATH
export
ANT_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.4
export PATH=$ANT_HOME/bin:$PATH
$PYTHON32_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{noformat}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370574#comment-16370574
]
Steve Rowe commented on LUCENE-8106:
BTW I noticed a side-effect of running the script as a Build step on Policeman
Jenkins: if the Ant build step fails (e.g. precommit), then none of the
following Build steps will be invoked. Maybe this is ok?
An alternative would be to switch from a Build step to a Post-Build step, but
this would require installing the Post Build Script plugin:
https://github.com/jenkinsci/postbuildscript-plugin
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370226#comment-16370226
]
Steve Rowe commented on LUCENE-8106:
bq. git.exe is undefined everywhere - except the master Jenkins (Linux), which
has GIT, but no other slave machine has GIT installed. They only use Jenkins'
internal JGit client, no command line. There is no need to have Git for running
Lucene builds (except packaging and jar version numbers, but Policeman does not
use this - because it's optional for test builds).
{{reproduceJenkinsFailures.py}} modifies the checked out revision in three
different ways:
# Checks out the revision at which the original failure(s) occurred (phase:
initial setup)
# Checks out the tip of the branch on which the original failure occurred
(phase: If any test reproduces 100%)
# Checks out the original workspace revision (phase: cleanup)
What do you think ought to be done on OS VMs without git?
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370215#comment-16370215
]
Steve Rowe commented on LUCENE-8106:
bq. On Linux, they don't have entries for those tools, so it falls back to
build.xml's default behaviour. git.exe is undefined everywhere - except the
master Jenkins (Linux), which has GIT, but no other slave machine has GIT
installed.
Thanks, I've tried to capture that in a modified version of the script now on
{{Lucene-Solr-master-Linux}}, we'll see how it goes next time it runs:
{noformat}
set -x # Log commands
TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT # Delete the temp file on SIGEXIT
curl -o $TMPFILE
https://jenkins.thetaphi.de/job/$JOB_NAME/$BUILD_NUMBER/consoleText
if grep --quiet 'reproduce with' $TMPFILE ; then
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -z $PYTHON32_EXE ] && PYTHON32_EXE=python3
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
[ -n $GIT_EXE ] && export PATH=$GIT_EXE:$PATH
$PYTHON32_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
file://$TMPFILE
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{noformat}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370148#comment-16370148
]
Steve Rowe commented on LUCENE-8106:
bq. On master jenkins i have a public key from you: {{ssh
jenk...@jenkins.thetaphi.de}} should work.
Indeed it does, thanks! I'd attempted (and failed) using a different acct.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369836#comment-16369836
]
Uwe Schindler commented on LUCENE-8106:
---
On master jenkins i have a public key from you: {{ssh
jenk...@jenkins.thetaphi.de}} should work.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369834#comment-16369834
]
Uwe Schindler commented on LUCENE-8106:
---
On Linux, they don't have entries for those tools, so it falls back to
build.xml's default behaviour. git.exe is undefined everywhere as except the
master Jenkins (Linux) no slave machine has GIT installed. They only use
Jenkins' internal JGit client, no command line.
{noformat}
jenkins@serv1:~$ pwd
/home/jenkins
jenkins@serv1:~$ cat lucene.build.properties
tests.jvms=3
tests.multiplier=3
{noformat}
FYI, on Windows it looks like that:
{noformat}
C:\Users\jenkins>type lucene.build.properties
python32.exe=C:\\Python35\\python.exe
perl.exe=C:\\Perl64\\bin\\perl.exe
tests.jvms=2
C:\Users\jenkins>git
'git' is not recognized as an internal or external command,
operable program or batch file.
{noformat}
Same on MacOSX and Solaris.
Sorry for not being responsive, I am a bit busy today. Can you send me your SSH
key privately? I will put them on all 4 machines into jenkins home dir.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369634#comment-16369634
]
Steve Rowe commented on LUCENE-8106:
[~thetaphi]: I've run into trouble getting the script ^^ to work. First, the
local log isn't where it is on my Jenkins
({{workspace/../builds/$BUILD_NUMBER/log}}, so I switched to fetching the log
via HTTPS and storing it in a temp file. That seems to work. But now there
are problems dealing with {{lucene.build.properties}}:
>From
>[https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21494/consoleText]:
{noformat}
+ set -x
+ mktemp
+ TMPFILE=/tmp/tmp.GedGCMtTog
+ trap rm -f /tmp/tmp.GedGCMtTog EXIT
+ curl -o /tmp/tmp.GedGCMtTog
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21494/consoleText
[...]
+ grep --quiet reproduce with /tmp/tmp.GedGCMtTog
+ mv lucene/build lucene/build.orig
+ mv solr/build solr/build.orig
+ grep ^[[:space:]]*python32\.exe[[:space:]]*=
/home/jenkins/lucene.build.properties
+ cut -d= -f2
+ PYTHON32_EXE=
+ grep ^[[:space:]]*git\.exe[[:space:]]*= /home/jenkins/lucene.build.properties
+ cut -d= -f2
+ GIT_EXE=
+
PATH=:/home/jenkins/tools/java/64bit/jdk-10-ea+43/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
-u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
file:///tmp/tmp.GedGCMtTog
/tmp/jenkins9073460340081845275.sh: 17: /tmp/jenkins9073460340081845275.sh: -u:
not found
+ rm -f /tmp/tmp.GedGCMtTog
{noformat}
>From ^^, it looks to me like {{/home/jenkins/lucene.build.properties}} either
>doesn't exist, or doesn't have entries for {{git.exe}} and {{python32.exe}}.
>I apparently no longer have login access to the VMs (I tried).
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368370#comment-16368370
]
Steve Rowe commented on LUCENE-8106:
{quote}
bq. I'd include the repro stuff into the main build job somehow!
I'll try making it a build step (instead of a post-build action).
{quote}
I added the following as an "Execute shell" build step to the
{{Lucene-Solr-master-Linux}} project - we'll see how it goes:
{noformat}
set -x # Log commands
# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig
PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*="
~/lucene.build.properties | cut -d'=' -f2`
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*=" ~/lucene.build.properties |
cut -d'=' -f2`
PATH=$GIT_EXE:$PATH $PYTHON32_EXE -u
dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
../builds/$BUILD_NUMBER/log
# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro
# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
{noformat}
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368233#comment-16368233
]
Steve Rowe commented on LUCENE-8106:
bq. I'd include the repro stuff into the main build job somehow!
I'll try making it a build step (instead of a post-build action).
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368178#comment-16368178
]
Uwe Schindler commented on LUCENE-8106:
---
Hi Steve,
I think it should work. The problem you havre is much simpler:
bq. When I tried to configure the Lucene-Solr-master-Linux project to trigger
building the new project, the "Build other projects" item in the "Add
post-build action" menu is greyed out.
Its grayed out because the master build job already triggers other jobs (the
Jenkins setup on Policeman is a chain of projects). Just scroll up and look for
the input line!
Currently you cannot parameterize the randomized JDK. So on reproducing, you
get another JDK/GC/bitness setting - sorry. Therefore I'd include the repro
stuff into the main build job somehow!
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16367724#comment-16367724
]
Uwe Schindler commented on LUCENE-8106:
---
I am out of office at the moment, so I have to look. Yes, the parameterized
build plugin is not there. The Randomization is done by a groovy script. To
repeat it with same setting it needs to be done different. The Randomization
stuff is saved nowhere, so it needs more work.
On the slaves, the python executable is saved in lucene.build.properties. No
git available on any slave.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16367696#comment-16367696
]
Steve Rowe commented on LUCENE-8106:
bq. [~thetaphi], I'll try to set up a job for your master slave.
I did this, after copying the configuration for the
{{Lucene-Solr-master-Linux}} project:
https://jenkins.thetaphi.de/job/Lucene-Solr-repro-Linux/
But I ran into three problems:
# The Parameterized Trigger plugin is not installed, but AFAIK is required to
pass information from the upstream job (to know where the upstream job's build
log is)
# When I tried to configure the {{Lucene-Solr-master-Linux}} project to trigger
building the new project, the "Build other projects" item in the "Add
post-build action" menu is greyed out.
# I'm not sure how to pass the randomized JDK+cmdline options to the repro
script - I suppose via the Parameterized Trigger plugin?
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16367516#comment-16367516
]
Steve Rowe commented on LUCENE-8106:
[~thetaphi], I'll try to set up a job for your master slave.
I'm not sure which entries you're referring to in {{lucene.build.properties}}?
Here's how ASF Jenkins is populated (from
[https://builds.apache.org/view/L/view/Lucene/job/Lucene-Ivy-Bootstrap/configure]):
{noformat}
tests.multiplier=2
smokeTestRelease.testArgs=-Dtests.slow=false
tests.jvms=3
{noformat}
AFAICT none of those refer to Python.
The repro job on ASF Jenkins invokes the script like this:
{noformat}
/usr/bin/python3 -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch
https://builds.apache.org/job/$UPSTREAM_JOB_NAME/$UPSTREAM_BUILD_NUMBER/consoleText
{noformat}
(On my Jenkins the URL points to the local log file instead of using HTTPS.)
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16367489#comment-16367489
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 1a6d896dfc1bdafff3067a513bade205f6e1ad11 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1a6d896 ]
LUCENE-8106: always fast-forward merge after checkout
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16367488#comment-16367488
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit e71286c8263646efe465ac7230a2fd7b34f9 in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e71286c ]
LUCENE-8106: always fast-forward merge after checkout
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16366831#comment-16366831
]
Uwe Schindler commented on LUCENE-8106:
---
Hi,
it might work out of box on the master slave, as it's running linux, but
definitely not on the others.
In general this Jenkins does not use native operating system's GIT at all, it
uses Jenkins JGIT plugin to handle checkouts. So git is not available on the
slaves with operating systems != linux. Python is available, but requires the
entries in lucene.build.xml properties file, as on the other operating systems
this is not in "PATH". That's the way how it's also setup on ASF jenkins.
I'd like to make the Jenkins build as generic as possible, so everything is
configured per node in lucene.build.properties, so the scriupt should read it
like the main jenkins.
IMHO, I'd add a top-level jenkins job that runs the python script (similar to
the smoke tester) using the standardized build properties, so it des not
require separate per-node-config:
{noformat}
$ ant run-jenkins-repro
{noformat}
This won't solve the missing "git" command problem.
Uwe
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16366054#comment-16366054
]
Steve Rowe commented on LUCENE-8106:
I've set up a new job using the repro script on ASF Jenkins:
[https://builds.apache.org/view/L/view/Lucene/job/Lucene-Solr-repro/], and
configured it to be triggered by Unstable/Failure results for Test, NightlyTest
and SmokeTester jobs on master and branch_7x.
[~thetaphi], what do you think of setting up a job for this on your Jenkins?
I'm not sure how to set up the same environment (OS/JVM) for the job though...
Maybe instead of triggering another job to run the repro script, the existing
jobs could be configured to run the script as a build step?
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363485#comment-16363485
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit a07493d508dce52bcfc584d001a0240cd835f9ba in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a07493d ]
LUCENE-8106: reproduceJenkinsFailures.py improvements: add cmdline option
parsing; sort max failures to the bottom of the report; retest at branch tip
first with and then without the seed if 100% of iterations fail.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363484#comment-16363484
]
ASF subversion and git services commented on LUCENE-8106:
-
Commit 08b9de9970a90d31dfec07e36faf8c1bc0a6c593 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=08b9de9 ]
LUCENE-8106: reproduceJenkinsFailures.py improvements: add cmdline option
parsing; sort max failures to the bottom of the report; retest at branch tip
first with and then without the seed if 100% of iterations fail.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363483#comment-16363483
]
Steve Rowe commented on LUCENE-8106:
Committing shortly.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16317536#comment-16317536
]
Steve Rowe commented on LUCENE-8106:
bq. So are you going to add Jobs to Apache Jenkins for this?
Yes, I was planning on it.
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
> Attachments: LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log
[
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16315050#comment-16315050
]
David Smiley commented on LUCENE-8106:
--
This is really cool Steve!
So are you going to add Jobs to Apache Jenkins for this?
> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Steve Rowe
> Attachments: LUCENE-8106.patch, LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
> python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
