[
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399885#comment-16399885
]
Steve Rowe edited comment on SOLR-10912 at 3/15/18 4:01 AM:
------------------------------------------------------------
{quote}The current change is here:
[https://github.com/apache/lucene-solr/compare/master...manokovacs:feature/SOLR-10912]
{quote}
I've attached a patch with a modified version of Mano's branch, and I plan on
committing it shortly in order to start testing the Jenkins job I've set up at
[https://builds.apache.org/view/PreCommit+Builds/job/PreCommit-LUCENE-Build/]
(the script in this job's config, a copy of which is in the patch, expects the
Yetus personality to be committed to the Git repo).
h2. From Mano's "Steps remaining":
{quote}Multiple module test currently duplicates the failed test result, fixing
shortly.
{quote}
Fixed.
{quote}Add test to call -validate-source-patterns
{quote}
Added. FYI, I had to rename the Ant task to remove the leading dash so that it
can be invoked from the cmdline.
{quote}Finalize the runner script to setup Yetus, similarly to Hadoop.
{quote}
Done.
{quote}Verify the test-patch with Github PR
{quote}
Since [~aw] wrote above that "The job on Jenkins that feeds test-patch is NOT
github aware", I don't plan on doing this verification. I'll include this on a
TODO list below.
{quote}Add Patch Available status for SOLR project (and update the script to
look for that).
{quote}
Done.
{quote}Add Precommit-SOLR and Precommit-LUCENE jenkins jobs
{quote}
I've added a PreCommit-LUCENE-Build job (linked above), and once I've committed
the patch I've attached here (on master only initially), I'll manually invoke
it for testing (via "Build with Parameters").
h2. From Mano's "Open questions":
{quote}1. As you can see, a patch with two log entries had 6 (flaky) test
failures. Assuming flaky tests will not go away very soon, would it still be
useful to have this test-patch?
{quote}
I think now is a perfect time to do this, given the current efforts focused on
reducing test flakiness.
{quote}2. I propose starting with the smallest set of tests (ie. affected
modules) and extend them later to dependent modules.
{quote}
+1.
h2. Stuff I changed from Mano's branch that is not already mentioned above:
# Renamed the personality file (was {{solr-yetus-personality.sh}}, now
{{lucene-solr-yetus-personality.sh}}).
# Improved handling of Lucene modules.
# Added basic ref guide validation, via {{ant bare-bones-html-validation}};
note that this is not the kind of missing-doc validation discussed above in
this issue; that idea was spun off as SOLR-11419
# Standardized Lucene/Solr-specific plugins to make them run only if they need
to.
# Added some user- and dev-level documentation to the local runner script and
personality files.
h2. Short-term TODO:
# Commit current patch to master
# Manually test the Jenkins precommit job
# Iterate above two steps until testing is successful
# Backport the patch to branch_7x
# Copy the Precommit-LUCENE-Build job to a new PreCommit-SOLR-Build job.
# Request ASF Infrastructure to add LUCENE and SOLR to the list of projects
that use the PreCommit-Admin Jenkins job to enqueue precommit runs for new
patches on LUCENE/SOLR JIRAs with the "Patch Available" state. (I'll make a
JIRA for this and link it to this issue.)
# Attach a patch with the finalized Lucene/Solr personality on HADOOP-14538,
for inclusion in future Yetus releases
*edit*: Added Shawn's improvement ideas to the TODO list below:
h2. Longer-term TODO (to be dealt with on further issues):
# Solr missing-doc validation: SOLR-11419.
# Add handling of module dependencies and build ordering.
# Currently when any test or non-test file is changed in a module, all unit
tests are run in that module. I think a faster version of this is possible when
there are test-only changes: just the changed test suites could be run, rather
than all of the module's tests.
# If there's not an entry in CHANGES.txt that mentions the issue number
(either the lucene or solr file as appropriate), that should be a -1 (or maybe
-0?)
# How about a -1 if a SOLR patch makes changes to lucene, or vice versa? If
there is an entry in the appropriate CHANGES.txt file for the issue, turn that
into a -0. That way, we have better assurance that if a commit for one part of
the project requires changes to the other part, there will be a release note.
was (Author: steve_rowe):
{quote}The current change is here:
[https://github.com/apache/lucene-solr/compare/master...manokovacs:feature/SOLR-10912]
{quote}
I've attached a patch with a modified version of Mano's branch, and I plan on
committing it shortly in order to start testing the Jenkins job I've set up at
[https://builds.apache.org/view/PreCommit+Builds/job/PreCommit-LUCENE-Build/]
(the script in this job's config, a copy of which is in the patch, expects the
Yetus personality to be committed to the Git repo).
h2. From Mano's "Steps remaining":
{quote}Multiple module test currently duplicates the failed test result, fixing
shortly.
{quote}
Fixed.
{quote}Add test to call -validate-source-patterns
{quote}
Added. FYI, I had to rename the Ant task to remove the leading dash so that it
can be invoked from the cmdline.
{quote}Finalize the runner script to setup Yetus, similarly to Hadoop.
{quote}
Done.
{quote}Verify the test-patch with Github PR
{quote}
Since [~aw] wrote above that "The job on Jenkins that feeds test-patch is NOT
github aware", I don't plan on doing this verification. I'll include this on a
TODO list below.
{quote}Add Patch Available status for SOLR project (and update the script to
look for that).
{quote}
Done.
{quote}Add Precommit-SOLR and Precommit-LUCENE jenkins jobs
{quote}
I've added a PreCommit-LUCENE-Build job (linked above), and once I've committed
the patch I've attached here (on master only initially), I'll manually invoke
it for testing (via "Build with Parameters").
h2. From Mano's "Open questions":
{quote}1. As you can see, a patch with two log entries had 6 (flaky) test
failures. Assuming flaky tests will not go away very soon, would it still be
useful to have this test-patch?
{quote}
I think now is a perfect time to do this, given the current efforts focused on
reducing test flakiness.
{quote}2. I propose starting with the smallest set of tests (ie. affected
modules) and extend them later to dependent modules.
{quote}
+1.
h2. Stuff I changed from Mano's branch that is not already mentioned above:
# Renamed the personality file (was {{solr-yetus-personality.sh}}, now
{{lucene-solr-yetus-personality.sh}}).
# Improved handling of Lucene modules.
# Added basic ref guide validation, via {{ant bare-bones-html-validation}};
note that this is not the kind of missing-doc validation discussed above in
this issue; that idea was spun off as SOLR-11419
# Standardized Lucene/Solr-specific plugins to make them run only if they need
to.
# Added some user- and dev-level documentation to the local runner script and
personality files.
h2. Short-term TODO:
# Commit current patch to master
# Manually test the Jenkins precommit job
# Iterate above two steps until testing is successful
# Backport the patch to branch_7x
# Copy the Precommit-LUCENE-Build job to a new PreCommit-SOLR-Build job.
# Request ASF Infrastructure to add LUCENE and SOLR to the list of projects
that use the PreCommit-Admin Jenkins job to enqueue precommit runs for new
patches on LUCENE/SOLR JIRAs with the "Patch Available" state. (I'll make a
JIRA for this and link it to this issue.)
# Attach a patch with the finalized Lucene/Solr personality on HADOOP-14538,
for inclusion in future Yetus releases
h2. Longer-term TODO (to be dealt with on further issues):
# Solr missing-doc validation: SOLR-11419.
# Add handling of module dependencies and build ordering.
# Currently when any test or non-test file is changed in a module, all unit
tests are run in that module. I think a faster version of this is possible when
there are test-only changes: just the changed test suites could be run, rather
than all of the module's tests.
> Adding automatic patch validation
> ---------------------------------
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
> Reporter: Mano Kovacs
> Assignee: Steve Rowe
> Priority: Major
> Attachments: SOLR-10912.ok-patch-in-core.patch, SOLR-10912.patch,
> SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or
> other Apache projects are using (see link). This would ensure that every
> patch passes a certain set of criterions before getting approved. It would
> save time for developer (faster feedback loop), save time for committers
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be
> a good direction to start. This jira could be the board of discussing the
> preferred solution.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
