In theory, if there are 10 pending patches, we would combine the 10 into one patch (or find a group that applies together) and then run the tests once. Assuming the number of failures are a fraction of these tests, the failed tests can be then run with each of these patches one at a time. If the number of failures are very small, we could even do use a binary search style method to find the patch that caused the failure. (More heuristics can be added such as first running a smoke-test suite before including the patch in a group).
On Sun, Feb 16, 2014 at 9:16 PM, Brock Noland <br...@cloudera.com> wrote: > On Sun, Feb 16, 2014 at 11:11 PM, Navis류승우 <navis....@nexr.com> wrote: > > bq. even if a JIRA is in the queue twice it will only be tested once. > > Good to know! > > > > bq. removing order-by clauses just for conforming purpose (my comment) > > > > I've tested it in https://issues.apache.org/jira/browse/HIVE-6438, > making > > 556 sec -> 418 sec for join_filters.q. Would it be worthwhile to rewrite > > and update so many tests/results? > > Faster is always better :) I'll look at 6438 tomorrow. > > > > > > > > > 2014-02-14 15:58 GMT+09:00 Brock Noland <br...@cloudera.com>: > > > >> Hi, > >> > >> The pre-commit tests: > >> > >> 1) only test the latest attachment > >> 2) post the attachment id to the JIRA > >> 3) Verify the attachment id has not been tested before running > >> > >> This means that even if a JIRA is in the queue twice it will only be > tested > >> once. > >> > >> Below are relevant portions of the script: > >> > >> curl -s -S --location --retry 3 > "${JIRA_ROOT_URL}/jira/browse/${JIRA_NAME}" > >> > $JIRA_TEXT > >> ... > >> PATCH_URL=$(grep -o '"/jira/secure/attachment/[0-9]*/[^"]*' $JIRA_TEXT > | \ > >> grep -v -e 'htm[l]*$' | sort | tail -1 | \ > >> grep -o '/jira/secure/attachment/[0-9]*/[^"]*') > >> ... > >> # ensure attachment has not already been tested > >> ATTACHMENT_ID=$(basename $(dirname $PATCH_URL)) > >> if grep -q "ATTACHMENT ID: $ATTACHMENT_ID" $JIRA_TEXT > >> then > >> echo "Attachment $ATTACHMENT_ID is already tested for $JIRA_NAME" > >> exit 1 > >> fi > >> > >> > >> > >> > >> > >> On Fri, Feb 14, 2014 at 12:51 AM, Navis류승우 <navis....@nexr.com> wrote: > >> > >> > Recently, precommit test takes more than 1 day (including queue time). > >> > > >> > Deduping work queue (currently, HIVE-6403 and HIVE-6418 is queued > twice) > >> > can make this better. Rewriting some test queries simpler (I'm > thinking > >> of > >> > removing order-by clauses just for conforming purpose). Any other > ideas? > >> > > >> > > >> > 2014-02-14 6:46 GMT+09:00 Thejas Nair <the...@hortonworks.com>: > >> > > >> > > I see a new job now running there. Maybe there is nothing wrong with > >> the > >> > > infra and builds actually finished (except for the 3 aborted ones). > >> > > Can't complain about a shorter queue ! :) > >> > > > >> > > > >> > > > >> > > On Thu, Feb 13, 2014 at 1:30 PM, Thejas Nair < > the...@hortonworks.com> > >> > > wrote: > >> > > > >> > > > Is the jenkins infra used for hive precommit tests under > maintenance > >> > ? I > >> > > > see that the long queue has suddenly disappeared. The last few > test > >> > > builds > >> > > > have been aborted. > >> > > > > >> > > > The jenkins used for hive precommit tests - > >> > > > http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/ > >> > > > > >> > > > Thanks, > >> > > > Thejas > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > -- > >> > > CONFIDENTIALITY NOTICE > >> > > NOTICE: This message is intended for the use of the individual or > >> entity > >> > to > >> > > which it is addressed and may contain information that is > confidential, > >> > > privileged and exempt from disclosure under applicable law. If the > >> reader > >> > > of this message is not the intended recipient, you are hereby > notified > >> > that > >> > > any printing, copying, dissemination, distribution, disclosure or > >> > > forwarding of this communication is strictly prohibited. If you have > >> > > received this communication in error, please contact the sender > >> > immediately > >> > > and delete it from your system. Thank You. > >> > > > >> > > >> > >> > >> > >> -- > >> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org > >> > > > > -- > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
