The CI currently running on cloudera master is still using old images w/o trunk prefix, hence it's still not maven 3.3.3 yet. I'd prefer to switch to new images on our new CI master. In new CI env w/ sufficient disk we don't need to have bunch of jobs for each package separately. Using a multi-parameter build job would be much easier to maintain.
My observation on HBase build is it is flipping. Sometimes it's good, sometimes not. I guess it's the upstream issue... BTW, I noticed that BIGTOP-2059 <https://issues.apache.org/jira/browse/BIGTOP-2059>, BIGTOP-2063 <https://issues.apache.org/jira/browse/BIGTOP-2063> have been pushed w/o +1. Are you doing CTR, Cos? If so that's fine but we should have a short announcement to officially start the trial. :) 2015-10-09 1:51 GMT+08:00 Olaf Flebbe <[email protected]>: > hi, > > The build dockers are updated. Be sure to use bigtop/slaves:trunk-centos-7 > for instance. The trunk-... containers denote the current git trunk built > by docker/build-slaves/... recipies. > > -olaf > > > > Am 08.10.2015 um 19:32 schrieb Konstantin Boudnik <[email protected]>: > > > > Indeed, but I don't think we have upgraded the build dockers just yet. > > Otherwise, the build would be just fine, I think. > > > > Cos > > > > On Thu, Oct 08, 2015 at 10:11AM, Jonathan Kelly wrote: > >> Maven should already be 3.3.3 as of > >> https://issues.apache.org/jira/browse/BIGTOP-1953, right? > >> > >> On Thu, Oct 8, 2015 at 9:32 AM, Konstantin Boudnik <[email protected]> > wrote: > >> > >>> Looks like we are pretty much all-green on the CI side. Two issues > left: > >>> - Mahout expects Maven >=3.3.3 (which I think is coming with the new > >>> containers, right?) > >>> - Hbase is failing weirdly and don't know what to make out of it > >>> 08:57:25 [ERROR] Failed to execute goal > >>> org.apache.maven.plugins:maven-site-plugin:3.3:site (default-site) on > >>> project > >>> hbase: Error during page generation: Error rendering Maven report: > Failed > >>> during checkstyle execution: Unable to find suppressions file at > location: > >>> hbase/checkstyle-suppressions.xml: Could not find resource > >>> 'hbase/checkstyle-suppressions.xml'. -> [Help 1] > >>> > >>> It seems like an upstream issue, isn't it? Andrew, any suggestions? > >>> > >>> Thanks, > >>> Cos > >>> > >>> On Mon, Sep 28, 2015 at 10:23PM, Evans Ye wrote: > >>>> Fix tez build, which was failing because of some docker daemon issue > on > >>> the > >>>> build slave. :) > >>>> > >>>> 2015-09-25 19:06 GMT+02:00 Evans Ye <[email protected]>: > >>>> > >>>>> With nodeps setting added now our CI is almost back to normal. > >>>>> There're still 5 components failing but the other's are good. > >>>>> I need to update our build slaves so that mahout can resume to > normal. > >>>>> I don't know why the others still failing. > >>>>> Will look into them when back from budapest ;) > >>>>> > >>>>> 2015-09-22 2:50 GMT+08:00 Konstantin Boudnik <[email protected]>: > >>>>> > >>>>>> I think for now it would be reasonable to just turn off the > dependency > >>>>>> graph > >>>>>> building. To do that we need to add > >>>>>> -Dbuildnodeps=true > >>>>>> to the gradle command line. > >>>>>> > >>>>>> Cos > >>>>>> > >>>>>> On Tue, Sep 22, 2015 at 01:40AM, Evans Ye wrote: > >>>>>>> Yup, and I think we've discussed that before in BIGTOP-1906. > >>>>>>> > >>>>>>> Right now I don't have any clue how to build that server up, but > >>> that's > >>>>>> a > >>>>>>> good way to go since our users would somehow need to set there > >>> internal > >>>>>>> jars repository for hadoop apps developer to download dependencies. > >>>>>>> > >>>>>>> At this point, I'm thinking that maybe Cos can give me a hint of > >>> code > >>>>>>> snippet how to turn the dependency build off. If there's no such > >>>>>> feature I > >>>>>>> can try to add it so that we can resume the CI status. > >>>>>>> > >>>>>>> > >>>>>>> 2015-09-21 20:36 GMT+08:00 Jay Vyas <[email protected]>: > >>>>>>> > >>>>>>>> Thanks Evans. I like your idea of separate Bigtop artifacts > >>> stored > >>>>>>>> somewhere... > >>>>>>>> > >>>>>>>> If Bigtop can publish its own jars to its own nexus (or else to > >>>>>> upstream > >>>>>>>> apaches maven server) that is ideal I guess! > >>>>>>>> > >>>>>>>>> On Sep 21, 2015, at 4:19 AM, Evans Ye <[email protected]> > >>> wrote: > >>>>>>>>> > >>>>>>>>> Hi all ! > >>>>>>>>> > >>>>>>>>> I spent some time looking into this and found that the massive > >>> job > >>>>>>>> failing > >>>>>>>>> is telling us that the build dependency feature is now working! > >>>>>> Which is > >>>>>>>>> because of every build that has dependency setting start from > >>>>>> building > >>>>>>>>> upstream packages locally. That fills up the disk size on every > >>>>>> build > >>>>>>>>> slaves. > >>>>>>>>> As I mentioned earlier, our CI build env is a sub-optimal design > >>>>>> because > >>>>>>>> of > >>>>>>>>> limited disk size on instances. I have no choice to split > >>> components > >>>>>>>>> equally on 4 slaves manually to ease the disk consumption. > >>>>>>>>> > >>>>>>>>> Maybe I should start trying nexus server to put built jars > >>> there? > >>>>>>>>> Or I can specify an option to tell gradle to fetch jars directly > >>>>>> from > >>>>>>>>> public maven instead of building upstream components locally? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Evans > >>>>>>>> > >>>>>> > >>>>> > >>>>> > >>> > >
