Thanks Szehon for your time and contribution in fixing test infra! Looks like things are good now.
On Wed, May 21, 2014 at 2:09 PM, Szehon Ho <sze...@cloudera.com> wrote: > The pre-commit builds should be officially good now. The test logs are > properly downloaded, and correctly displayed on the HiveQA JIRA comment as > the link under "TestReport", and all logs themselves in raw form are at > "Test logs", see HIVE-6937 and HIVE-7080 for details. > > Also as we had troubles with the patch-upload trigger job on Apache Jenkins > having outages and not triggering the test, so we also moved that to the > Hive Jenkins on the same server: > > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-Admin/ > , > which now runs the hadoopQA script every 10 minutes. > I'll continue to monitor this one, as its still new. > > Also as some might know already, the Pre-commit build is now using > -Phadoop-2 and JDK1.7. > > Thanks > Szehon > < > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-Admin/ > > > > > On Tue, Apr 22, 2014 at 4:15 PM, Szehon Ho <sze...@cloudera.com> wrote: > > > Patch upload auto-trigger is now enabled, thanks Ashutosh for helping me > > fix the Jenkins url from Apache-Infra side. Now there's no need to > > manually submit Pre-commit builds, unless it got missed somehow. And in > > this Jenkins, you can easily see all the JIRA's in the queue: > > > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/ > > > > Sorry, please use this URL and not the first one, as job was renamed due > > to the auto-trigger expecting this format. > > > > And to repeat, Test Reporting and the 'Logs located at:' link is still > > broken and tracked in HIVE-6937. For now, to see the logs of a failed > > build, you have to manually navigate. > > Example, for PreCommit-Hive-BUILD#6, logs are at: > > > http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-6/ > < > http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-6/ > >. > > The HiveQA comment on the JIRA should mention the build number. > > > > Thanks, > > Szehon > > > > > > On Sun, Apr 20, 2014 at 11:07 PM, Szehon Ho <sze...@cloudera.com> wrote: > > > >> It looks great, thanks Lefty! > >> > >> > >> On Sun, Apr 20, 2014 at 2:22 PM, Lefty Leverenz < > leftylever...@gmail.com>wrote: > >> > >>> Nice doc, Szehon. I did some minor editing so you might want to make > >>> sure > >>> I didn't introduce any errors. > >>> > >>> > >>> > https://cwiki.apache.org/confluence/display/Hive/Hive+PTest2+Infrastructure > >>> > >>> -- Lefty > >>> > >>> > >>> On Sat, Apr 19, 2014 at 9:45 PM, Szehon Ho <sze...@cloudera.com> > wrote: > >>> > >>> > Migration is done, I updated the wiki to add all the details of the > new > >>> > setup: > >>> > > >>> > https://cwiki.apache.org/confluence/display/Hive/Hive+PTest2+Infrastructure > >>> > > >>> > New Jenkins URL to submit pre-commit jobs: > >>> > > >>> > > >>> > http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/precommit-hive/ > >>> > Again, this has to be done manually for time being, by clicking on > >>> 'build > >>> > with parameters', and entering the issue number as a parameter. I've > >>> > submitted some already. I'll reach out to some committers to get the > >>> > auto-trigger working. > >>> > > >>> > As I mentioned, there is some work to fix the test-reporting, due to > >>> the > >>> > framework using old url scheme. I am tracking it at > >>> > HIVE-6937<https://issues.apache.org/jira/browse/HIVE-6937>. > >>> > For now I am hosting log directory separately, if you want to see > test > >>> > logs, you have to manually go the url corresponding to your build, > >>> like: > >>> > > >>> > http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/precommit-hive-11/for > >>> > run#11. Sorry about that. > >>> > > >>> > Let me know if you see other issues, thanks! > >>> > Szehon > >>> > > >>> > > >>> > On Fri, Apr 18, 2014 at 2:11 PM, Thejas Nair <the...@hortonworks.com > > > >>> > wrote: > >>> > > >>> > > Sounds good. > >>> > > Thanks Szehon! > >>> > > > >>> > > > >>> > > On Fri, Apr 18, 2014 at 10:17 AM, Ashutosh Chauhan < > >>> hashut...@apache.org > >>> > > > >>> > > wrote: > >>> > > > +1 Thanks Szehon! > >>> > > > > >>> > > > > >>> > > > On Fri, Apr 18, 2014 at 6:29 AM, Xuefu Zhang < > xzh...@cloudera.com> > >>> > > wrote: > >>> > > > > >>> > > >> +1. Thanks for taking care of this. > >>> > > >> > >>> > > >> > >>> > > >> On Thu, Apr 17, 2014 at 11:00 PM, Szehon Ho < > sze...@cloudera.com> > >>> > > wrote: > >>> > > >> > >>> > > >> > Hi, > >>> > > >> > > >>> > > >> > This week the machine running Hive builds at > >>> > > >> > http://bigtop01.cloudera.org:8080/view/Hive/? ran out of > >>> space, so > >>> > > new > >>> > > >> > jobs like Precommit tests stopped. Its still not resolved > >>> there, > >>> > > there > >>> > > >> was > >>> > > >> > another email today on Bigtop list, but there's very few > people > >>> with > >>> > > root > >>> > > >> > access to that host, and they still haven't responded. > >>> > > >> > > >>> > > >> > I chatted with Brock, he has also seen various issues with > >>> Bigtop > >>> > > jenkins > >>> > > >> > in the past, so I am thinking to move the Jenkins jobs to the > >>> PTest > >>> > > >> master > >>> > > >> > itself, where some PMC already have access and can admin if > >>> needed. > >>> > > >> > Currently I am hosting the pre-commit Jenkins job on my own > EC2 > >>> > > instance > >>> > > >> > as stop-gap. > >>> > > >> > > >>> > > >> > Other advantages of hosting our own Jenkins: > >>> > > >> > 1. No need to wait for other Bigtop jobs to run. > >>> > > >> > 2. Bigtop is using a version of Jenkins that doesnt show > >>> parameters > >>> > > like > >>> > > >> > JIRA number for queued jobs, so impossible to tell whether a > >>> patch > >>> > got > >>> > > >> > picked up and where it is in queue. > >>> > > >> > 3. Eliminate network hop from Bigtop box to our PTest master. > >>> > > >> > > >>> > > >> > The disadvantage is: > >>> > > >> > 1. We don't have much experience doing Jenkins admin, but it > >>> doesn't > >>> > > look > >>> > > >> > too bad. Mostly, restart if there's issue and clean up if out > >>> of > >>> > > space. > >>> > > >> > > >>> > > >> > I wonder what people think, and if there's any objections to > >>> this? > >>> > If > >>> > > >> not, > >>> > > >> > I'll try setting up this weekend. Then, there is some > follow-up > >>> > work, > >>> > > >> like > >>> > > >> > changing the Jenkins url's displayed in the test report. > >>> > > >> > > >>> > > >> > Thanks! > >>> > > >> > Szehon > >>> > > >> > > >>> > > >> > >>> > > > >>> > > -- > >>> > > CONFIDENTIALITY NOTICE > >>> > > NOTICE: This message is intended for the use of the individual or > >>> entity > >>> > to > >>> > > which it is addressed and may contain information that is > >>> confidential, > >>> > > privileged and exempt from disclosure under applicable law. If the > >>> reader > >>> > > of this message is not the intended recipient, you are hereby > >>> notified > >>> > that > >>> > > any printing, copying, dissemination, distribution, disclosure or > >>> > > forwarding of this communication is strictly prohibited. If you > have > >>> > > received this communication in error, please contact the sender > >>> > immediately > >>> > > and delete it from your system. Thank You. > >>> > > > >>> > > >>> > >> > >> > > >
