Yep, we’ve finally received it. It’s done. Current issue is that Jenkins is unable to set Github build status. I’ve added repo:status permission, but it’s also asking to be member of the project/organization and not sure how to do that.
Andor > On 2020. Jul 29., at 4:10, 张铎(Duo Zhang) <[email protected]> wrote: > > Seems you have already made it? > > Usually there are several moderators for the private list, you need to ask > them to let the GitHub registration go through. > > Andor Molnar <[email protected]> 于2020年7月29日周三 上午1:03写道： > >> Thanks Duo, that’s very helpful. >> I cannot set private@zookeeper as a verified e-mail address, because the >> verification e-mail cannot be sent to the list. Isn’t that restricted for >> members only (by default)? >> >> Andor >> >> >> >>> On 2020. Jul 28., at 3:15, 张铎(Duo Zhang) <[email protected]> wrote: >>> >>> Hi Andor, >>> >>> The Apache-HBase account is registered by me, using the private@hbase >>> mailing list, so all the PMC members can maintain the password. >>> >>> I generated an access token and added it to our jenkins, so we can use it >>> to post comments back to GitHub. >>> >>> I think you could do the same to register an Apache-ZooKeeper account? Or >>> if you want to use the hadoop-yetus account, you'd better ask the hadoop >>> PMC members or Gavin to add the token to jenkins so you can use it. >>> >>> Thanks. >>> >>> Andor Molnar <[email protected]> 于2020年7月28日周二 上午3:56写道： >>> >>>> Hi Duo, >>>> >>>> I’m trying to create a similar job for Apache ZooKeeper, but >> unfortunately >>>> haven’t got too much help on the Apache builds@ list so far, so I’m >>>> rather asking you if you don’t mind. >>>> >>>> First, how have you set up the Hbase Github account that you use in this >>>> job to access the repo? >>>> >>>> Thanks, >>>> Andor >>>> >>>> >>>> >>>>> On 2020. Jul 27., at 2:22, 张铎(Duo Zhang) <[email protected]> >> wrote: >>>>> >>>>> The pre commit job has been migrated to c-hadoop.a.o. >>>>> >>>>> I have disabled periodical scan for the old job on builds.a.o, as we >>>> still >>>>> need to view the pre commit result on it do not delete for now. Will >>>> delete >>>>> it later, maybe after several weeks. >>>>> >>>>> The new job is here >>>>> >>>>> https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/ >>>>> >>>>> Thanks. >>>>> >>>>> 张铎(Duo Zhang) <[email protected]> 于2020年7月25日周六 下午9:44写道： >>>>> >>>>>> >>>>>> >>>> >> https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/5/console >>>>>> >>>>>> >>>>>> We successfully finished a nightly build. >>>>>> >>>>>> But seems the jiraComment did not work. I haven't seen the comment >>>>>> on HBASE-24757... >>>>>> >>>>>> 张铎(Duo Zhang) <[email protected]> 于2020年7月25日周六 下午4:51写道： >>>>>> >>>>>>> After installing two new jenkins plugins, the pre commit job seems >> fine >>>>>>> now. >>>>>>> >>>>>>> The last failure is because of a timeout, I assume the problem is >> that >>>> we >>>>>>> do not have enough executors so all the jobs are executed >> sequentially. >>>>>>> >>>>>>> Maybe we could move the pre commit job to the new env first? The >>>> nightly >>>>>>> job and flaky job require more resources, and we need the output of >>>> these >>>>>>> jenkins jobs(the flaky test list). >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> >>>>>>> 张铎(Duo Zhang) <[email protected]> 于2020年7月24日周五 下午4:36写道： >>>>>>> >>>>>>>> The problem seems because of this: >>>>>>>> >>>>>>>> https://issues.jenkins-ci.org/browse/JENKINS-48556 >>>>>>>> >>>>>>>> I triggered the job again, it passed the timestamps call, and will >>>> keep >>>>>>>> an eye on it. >>>>>>>> >>>>>>>> 张铎(Duo Zhang) <[email protected]> 于2020年7月21日周二 上午11:18写道： >>>>>>>> >>>>>>>>> On the sponsors, we could have a try. >>>>>>>>> >>>>>>>>> The problem here is the process of the donation? IIRC there is a >>>> thread >>>>>>>>> on the infra mailing list about how to donate machines to a >> specific >>>>>>>>> project and the discussion did not go well... >>>>>>>>> >>>>>>>>> Sean Busbey <[email protected]> 于2020年7月21日周二 上午11:13写道： >>>>>>>>> >>>>>>>>>> We could check with ASF infra for the current state of things wrt >>>>>>>>>> GitHub >>>>>>>>>> actions. I believe there is a queue set up across ASF projects. >>>>>>>>>> >>>>>>>>>> It has the same resource issue Travis had; things are fine until >>>> some >>>>>>>>>> critical mass of projects seeking better perf realize some new >>>> option >>>>>>>>>> is >>>>>>>>>> available and then quickly all available resources are consumed. >>>>>>>>>> >>>>>>>>>> AFAICT the only option that gets us the same or better as the H* >>>> nodes >>>>>>>>>> will >>>>>>>>>> be finding sponsors and running our own. >>>>>>>>>> >>>>>>>>>> On Mon, Jul 20, 2020, 21:55 张铎(Duo Zhang) <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I think our nightly, flakey, and pre commit jobs should be >>>>>>>>>> transferred as a >>>>>>>>>>> whole? They depend on each other. >>>>>>>>>>> >>>>>>>>>>> I offer my help on the transition. >>>>>>>>>>> >>>>>>>>>>> And on github CI, does ASF have a special deal with github? If >> not, >>>>>>>>>> I do >>>>>>>>>>> not think the default resource can fit our requirements... >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Sean Busbey <[email protected]> 于2020年7月21日周二 上午1:49写道： >>>>>>>>>>> >>>>>>>>>>>> Hi folks! >>>>>>>>>>>> >>>>>>>>>>>> Back in April there was a brief discussion[1] about ASF Infra's >>>>>>>>>>>> notification that builds.a.o is going away and we are currently >>>>>>>>>> slated >>>>>>>>>>>> to migrate to a set of CI servers for "Hadoop and related >>>>>>>>>> projects". >>>>>>>>>>>> This is the ci farm that will contain the bulk of the H* worker >>>>>>>>>> nodes >>>>>>>>>>>> that are donated by Yahoo!, which are the nodes we've been >> running >>>>>>>>>> on >>>>>>>>>>>> for ages[2]. >>>>>>>>>>>> >>>>>>>>>>>> Migration discussion still happens on the >> [email protected] >>>>>>>>>>>> list[3] and recently ASF Infra set a target date of August 15th >>>> for >>>>>>>>>>>> turning off the existing builds.a.o server[4]. >>>>>>>>>>>> >>>>>>>>>>>> That gives us a little under 4 weeks to have things up and >> working >>>>>>>>>> on >>>>>>>>>>>> the new ci-hadoop.a.o jenkins coordinator[5]. it’s not clear to >> me >>>>>>>>>>>> that the level of effort we’ll need to spend is worth what we >> get >>>>>>>>>> out >>>>>>>>>>>> of a continuation of the status quo on builds.a.o. I did a quick >>>>>>>>>> test >>>>>>>>>>>> by updating the nightly job on ci-hadoop.a.o to run just >> branch-2, >>>>>>>>>>>> since that has been stable on builds.a.o. It failed with a >> Jenkins >>>>>>>>>>>> pipeline DSL syntax error[6] so I'm assuming migrating will be a >>>>>>>>>> slog. >>>>>>>>>>>> >>>>>>>>>>>> As far as I can see our options are: >>>>>>>>>>>> >>>>>>>>>>>> * Do nothing. Have no testing or automated website publication >> in >>>>>>>>>> mid >>>>>>>>>>>> August. >>>>>>>>>>>> * Transition website publication and nothing else (probably can >> be >>>>>>>>>>>> done in a day) >>>>>>>>>>>> * Transition just precommit testing for various repos (probably >>>>>>>>>> can be >>>>>>>>>>>> done in a few days) >>>>>>>>>>>> * Transition everything (no idea how long it takes due to >> nightly, >>>>>>>>>>>> flaky stuff, etc) >>>>>>>>>>>> >>>>>>>>>>>> The alternatives if we do not transition any given job to >>>>>>>>>> ci-hadoop: >>>>>>>>>>>> >>>>>>>>>>>> * Try to move to GitHub Actions >>>>>>>>>>>> * Try to move to Travis CI >>>>>>>>>>>> * Try to move to Jenkins infra we maintain ourselves (presumably >>>> by >>>>>>>>>>>> soliciting project specific donations for worker nodes on cloud >>>>>>>>>>>> vendors) >>>>>>>>>>>> >>>>>>>>>>>> It's important to remember that as a project we have a heavy >>>>>>>>>> footprint >>>>>>>>>>>> wherever our nightly tests run. For context, a given branch's >>>>>>>>>> nightly >>>>>>>>>>>> can keep 3-4 executors busy for 6+ hours on the current >> builds.a.o >>>>>>>>>>>> setup. There's been a bunch of great work lately on bringing >> down >>>>>>>>>> what >>>>>>>>>>>> it takes to run the full test suite, but applying that work to >>>>>>>>>> nightly >>>>>>>>>>>> is itself a significant undertaking. >>>>>>>>>>>> >>>>>>>>>>>> What are folks thinking? Most importantly who is ready to work >>>>>>>>>> towards >>>>>>>>>>>> any given approach? >>>>>>>>>>>> >>>>>>>>>>>> [1] [DISCUSS] Migrating HBase to new CI Master >>>>>>>>>>>> https://s.apache.org/fux1o >>>>>>>>>>>> >>>>>>>>>>>> [2] https://builds.apache.org/view/H-L/view/HBase/ >>>>>>>>>>>> >>>>>>>>>>>> [3] >>>>>>>>>>> >>>>>>>>>> >>>> https://lists.apache.org/[email protected] >>>>>>>>>>>> >>>>>>>>>>>> [4] [IMPORTANT] - 2 more HADOOP nodes migrated over to ci-hadoop >>>>>>>>>>>> https://s.apache.org/7e1nq >>>>>>>>>>>> >>>>>>>>>>>> [5] https://ci-hadoop.apache.org/job/HBase/ >>>>>>>>>>>> >>>>>>>>>>>> [6] >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>> >> https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/2/console >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>> >>>> >> >>
