Ah, another problem... java.lang.NoSuchMethodError: No such DSL method 'sshTransfer' found among steps
Seems on the new jenkins we do not have the support for transfering data to nightlies... 张铎(Duo Zhang) <[email protected]> 于2022年1月22日周六 17:04写道： > Thank you Gavin. The builds are started now. > > Let's wait for the result. > > And on installing plugins, I haven't seen any system management related > button on the https://ci-hbase.apache.org/ page... > On the https://jenkins-ccos.apache.org/job/masters/job/HBase/ page, click > the 'Config Files' button, there is a 'Manage Jenkins' button but I have no > permission to click it... > > Another thing is hbase20 is offline because of readonly filesystem. PTAL. > > Thanks a lot. > > Gavin McDonald <[email protected]> 于2022年1月22日周六 16:42写道： > >> Hi, >> Timestamper plugin installed. >> >> By the way I thought I had given the PMC members permissions to manage >> Jenkins >> on ci-hbase including installing plugins etc, can someone check? >> >> >> On Sat, Jan 22, 2022 at 7:34 AM 张铎(Duo Zhang) <[email protected]> >> wrote: >> >>> We faced a strange issue.. >>> >>> rg.codehaus.groovy.control.MultipleCompilationErrorsException: startup >>> failed: >>> WorkflowScript: 29: Invalid option type "timestamps". Valid option types: >>> [buildDiscarder, catchError, checkoutToSubdirectory, >>> disableConcurrentBuilds, disableResume, durabilityHint, >>> githubProjectProperty, newContainerPerStage, overrideIndexTriggers, >>> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, >>> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, >>> waitUntil, warnError, withChecks, withContext, withCredentials, withEnv, >>> wrap, ws] @ line 29, column 5. >>> timestamps() >>> ^ >>> >>> 1 error >>> >>> at >>> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310) >>> at >>> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085) >>> at >>> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603) >>> at >>> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581) >>> at >>> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558) >>> at >>> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) >>> at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) >>> at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688) >>> at groovy.lang.GroovyShell.parse(GroovyShell.java:700) >>> at >>> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142) >>> at >>> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127) >>> at >>> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571) >>> at >>> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523) >>> at >>> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:334) >>> at hudson.model.ResourceController.execute(ResourceController.java:97) >>> at hudson.model.Executor.run(Executor.java:429) >>> >>> >>> After googling, it seems a restart of jenkins can fix it, or we do not >>> have timestamper plugin installed? Mind taking a look Gavin? >>> >>> Thanks. >>> >>> 张铎(Duo Zhang) <[email protected]> 于2022年1月22日周六 11:40写道： >>> >>>> Oh, just saw that we could do this by ourselves. >>>> >>>> Let me have a try. >>>> >>>> 张铎(Duo Zhang) <[email protected]> 于2022年1月22日周六 11:18写道： >>>> >>>>> OK, so while copying you could keep the old jenkins builds? >>>>> That's awesome. >>>>> >>>>> Then please help 'copying' HBase-Find-Flaky-Tests >>>>> and HBase-Flaky-Tests to ci-hbase too? We need the history builds to >>>>> calucate the flaky test list so if all the old build data can be kept then >>>>> it will help us a lot. >>>>> >>>>> Thanks~ >>>>> >>>>> 张铎(Duo Zhang) <[email protected]> 于2022年1月22日周六 11:03写道： >>>>> >>>>>> Thank you Gavin. >>>>>> >>>>>> Let me first move some light weighted job first to see if everything >>>>>> is OK. >>>>>> >>>>>> HBase Nightly, some flaky related jobs, and also the main pre commit >>>>>> job for HBase are tied together. We need to move them at once. >>>>>> >>>>>> Gavin McDonald <[email protected]> 于2022年1月22日周六 02:44写道： >>>>>> >>>>>>> Thanks for your patience >>>>>>> >>>>>>> I have 'copied' over a job - >>>>>>> https://ci-hbase.apache.org/job/HBase%20Nightly/ >>>>>>> >>>>>>> which of course made me aware of what plugins I needed to install. >>>>>>> >>>>>>> Feel free to delete that, or use it and delete the old, and let me >>>>>>> know if have any issues >>>>>>> with any other missing plugins. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Jan 20, 2022 at 1:11 PM 张铎(Duo Zhang) <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I’ve also replied on the jira issue. >>>>>>>> >>>>>>>> You can try to copy a job to the new Jenkins. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> Gavin McDonald <[email protected]>于2022年1月20日 周四18:45写道： >>>>>>>> >>>>>>>>> Im taking a look ... >>>>>>>>> >>>>>>>>> On Thu, Jan 20, 2022 at 6:54 AM Sean Busbey <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> > Hi Gavin >>>>>>>>> > >>>>>>>>> > I went to start migrating jobs to the ci-hbase controller, but >>>>>>>>> the new job >>>>>>>>> > workflow doesn't include pipelines or multibranch pipelines. We >>>>>>>>> use both of >>>>>>>>> > these job types for nearly all of our work. Can they be enabled? >>>>>>>>> > >>>>>>>>> > On Tue, Jan 18, 2022 at 5:04 AM Gavin McDonald < >>>>>>>>> [email protected]> >>>>>>>>> > wrote: >>>>>>>>> > >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> On 2022/01/18 09:20:24 "张铎(Duo Zhang)" wrote: >>>>>>>>> >> > Thank you Gavin. >>>>>>>>> >> > >>>>>>>>> >> > We have some urls in our jenkins files which references the >>>>>>>>> artifacts of >>>>>>>>> >> > the jenkins jobs, so a simple copy or move can not work. For >>>>>>>>> example, >>>>>>>>> >> here: >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> >> >>>>>>>>> https://github.com/apache/hbase/blob/c9bcd87b34a15d200a55ec7fdc2b1d86e3367a8c/dev-support/Jenkinsfile_GitHub#L47 >>>>>>>>> >> > >>>>>>>>> >> > So I prefer we migrate the jobs by ourselves. Will file a >>>>>>>>> HBase issue >>>>>>>>> >> to do >>>>>>>>> >> > this. >>>>>>>>> >> > >>>>>>>>> >> > And will report any problems on INFRA_22760 while migrating. >>>>>>>>> When the >>>>>>>>> >> > migration is half done, you can move for example, 5 hbase >>>>>>>>> nodes from >>>>>>>>> >> > ci-hadoop to ci-hbase, and once the migration is finally >>>>>>>>> done, you can >>>>>>>>> >> move >>>>>>>>> >> > all the remaining hbase nodes from ci-hadoop to ci-hbase. >>>>>>>>> >> > >>>>>>>>> >> > WDYT? >>>>>>>>> >> >>>>>>>>> >> However you want to do it is fine with me :) >>>>>>>>> >> >>>>>>>>> >> You have 10 nodes on there right now to get you started. >>>>>>>>> >> >>>>>>>>> >> Let me know how you get on >>>>>>>>> >> >>>>>>>>> >> > >>>>>>>>> >> > Thanks. >>>>>>>>> >> > >>>>>>>>> >> > Gavin McDonald <[email protected]> 于2022年1月18日周二 16:16写道： >>>>>>>>> >> > >>>>>>>>> >> > > Hi All, >>>>>>>>> >> > > >>>>>>>>> >> > > I created an Infra ticket for tracking. >>>>>>>>> >> > > >>>>>>>>> >> > > https://issues.apache.org/jira/browse/INFRA-22760 >>>>>>>>> >> > > >>>>>>>>> >> > > For now, I left your current nodes hbase1 to hbase10 in >>>>>>>>> ci-hadoop.a.o >>>>>>>>> >> > > >>>>>>>>> >> > > I have created for you 10 new nodes so far out of our >>>>>>>>> Hetzner DC >>>>>>>>> >> > > of a similar specifications to the hbase1-10. >>>>>>>>> >> > > >>>>>>>>> >> > > Those are hbase11-20 and are available right now for >>>>>>>>> testing on >>>>>>>>> >> > > ci-hbase.apache.org >>>>>>>>> >> > > >>>>>>>>> >> > > I do not know your workflow well enough to decide the next >>>>>>>>> steps for >>>>>>>>> >> you >>>>>>>>> >> > > for testing. >>>>>>>>> >> > > >>>>>>>>> >> > > Options include: >>>>>>>>> >> > > >>>>>>>>> >> > > 1. I can 'move' all jobs out of ci-hadoop onto ci-hbase >>>>>>>>> >> > > 2. I can 'move' one or a number of jobs out of ci-hadoop >>>>>>>>> onto ci-hbase >>>>>>>>> >> > > 3. Similar to 1 or 2, but 'copy' any number of jobs over >>>>>>>>> for testing. >>>>>>>>> >> > > - I like copy, however quite a few projects jobs would >>>>>>>>> not be >>>>>>>>> >> > > appropriate >>>>>>>>> >> > > to have the same job running twice, including those >>>>>>>>> that >>>>>>>>> >> deploy, or >>>>>>>>> >> > > those that >>>>>>>>> >> > > build from PRs and continue a workflow to report >>>>>>>>> back, or >>>>>>>>> >> approve >>>>>>>>> >> > > for merge. >>>>>>>>> >> > > So for a copy, would need to be a job that just >>>>>>>>> 'builds and >>>>>>>>> >> tests' >>>>>>>>> >> > > with no outflow. >>>>>>>>> >> > > 4. You guys create new jobs on ci-hbase.apache.org to test >>>>>>>>> the nodes >>>>>>>>> >> > > setup and the >>>>>>>>> >> > > main controller setup, without affecting current jobs. >>>>>>>>> >> > > >>>>>>>>> >> > > Obviously, you may to tweak version controlled script files >>>>>>>>> such as a >>>>>>>>> >> > > JenkinsFile, which makes the copy options harder unless >>>>>>>>> testing from a >>>>>>>>> >> > > branch created for the purpose. >>>>>>>>> >> > > >>>>>>>>> >> > > I may be overthinking things, and option 1 , the simplest >>>>>>>>> and most >>>>>>>>> >> > > straight forward from my end would also suit you guys and >>>>>>>>> we deal >>>>>>>>> >> with any >>>>>>>>> >> > > breakage as it happens (I'm thinking of integrations with >>>>>>>>> other >>>>>>>>> >> services, >>>>>>>>> >> > > Credentials/Secrets, Plugins I haven't installed that are >>>>>>>>> needed etc.) >>>>>>>>> >> > > >>>>>>>>> >> > > Let me know please ASAP how you want to proceed so we can >>>>>>>>> get using >>>>>>>>> >> these >>>>>>>>> >> > > nodes, after which I can then proceed to move hbase1-10 >>>>>>>>> nodes over >>>>>>>>> >> also. >>>>>>>>> >> > > And then start adding more into the mix. >>>>>>>>> >> > > >>>>>>>>> >> > > Oh. The 'git-websites' label should be a no-op. It is >>>>>>>>> available on >>>>>>>>> >> > > ci-hbase.a.o as the nodes with that label are shared and >>>>>>>>> leased out to >>>>>>>>> >> > > controllers as needed upon request of a job. >>>>>>>>> >> > > >>>>>>>>> >> > > Thanks >>>>>>>>> >> > > >>>>>>>>> >> > > Gav... >>>>>>>>> >> > > >>>>>>>>> >> > > >>>>>>>>> >> > > On 2021/12/09 15:07:26 "张铎(Duo Zhang)" wrote: >>>>>>>>> >> > > > Thanks you Gavin. >>>>>>>>> >> > > > >>>>>>>>> >> > > > You can do it in your convenience, just let me know so I >>>>>>>>> could >>>>>>>>> >> change the >>>>>>>>> >> > > > label in the jenkinsfile of our jenkins jobs, at least >>>>>>>>> our pre >>>>>>>>> >> commit job >>>>>>>>> >> > > > still uses the Hadoop label. >>>>>>>>> >> > > > >>>>>>>>> >> > > > And another thing is about the hbase_generate_website >>>>>>>>> job, it needs >>>>>>>>> >> to be >>>>>>>>> >> > > > executed on the nodes under the git-websites label. I'm >>>>>>>>> not sure if >>>>>>>>> >> the >>>>>>>>> >> > > > label is still available after the migration. >>>>>>>>> >> > > > >>>>>>>>> >> > > > Gavin McDonald <[email protected]> 于2021年12月8日周三 >>>>>>>>> 17:28写道： >>>>>>>>> >> > > > >>>>>>>>> >> > > > > Hi HBase devs. >>>>>>>>> >> > > > > >>>>>>>>> >> > > > > This is to let you know that Infra has created a >>>>>>>>> dedicated Jenkins >>>>>>>>> >> > > > > Controller at https://ci-hbase.apache.org . >>>>>>>>> >> > > > > >>>>>>>>> >> > > > > You have new incoming donated agents which will get >>>>>>>>> connected to >>>>>>>>> >> this >>>>>>>>> >> > > new >>>>>>>>> >> > > > > controller by Infra. >>>>>>>>> >> > > > > >>>>>>>>> >> > > > > You also have 10 or so agents connected to >>>>>>>>> >> > > https://ci-hadoop.apache.org - >>>>>>>>> >> > > > > I >>>>>>>>> >> > > > > would like to move these agents - and all of your jobs >>>>>>>>> - over to >>>>>>>>> >> the >>>>>>>>> >> > > new >>>>>>>>> >> > > > > controller as soon as possible. >>>>>>>>> >> > > > > >>>>>>>>> >> > > > > You may or may not need to reconfigure your jobs and or >>>>>>>>> >> JenkinsFile or >>>>>>>>> >> > > > > other. >>>>>>>>> >> > > > > >>>>>>>>> >> > > > > Can I get the go ahead to move these at my convenience >>>>>>>>> or suggest >>>>>>>>> >> > > please a >>>>>>>>> >> > > > > timeline for Infra to move these agents. >>>>>>>>> >> > > > > >>>>>>>>> >> > > > > Thanks! >>>>>>>>> >> > > > > >>>>>>>>> >> > > > > -- >>>>>>>>> >> > > > > >>>>>>>>> >> > > > > *Gavin McDonald* >>>>>>>>> >> > > > > Systems Administrator >>>>>>>>> >> > > > > ASF Infrastructure Team >>>>>>>>> >> > > > > >>>>>>>>> >> > > > >>>>>>>>> >> > > >>>>>>>>> >> > >>>>>>>>> >> >>>>>>>>> > >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> *Gavin McDonald* >>>>>>>>> Systems Administrator >>>>>>>>> ASF Infrastructure Team >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *Gavin McDonald* >>>>>>> Systems Administrator >>>>>>> ASF Infrastructure Team >>>>>>> >>>>>> >> >> -- >> >> *Gavin McDonald* >> Systems Administrator >> ASF Infrastructure Team >> >
