Alex, Thanks for reporting the issue, but let's not drive the discussion in this way. This is an open source project and we're just trying to make software for public good. We don't have the resource to compete with commercial distribution vendor(s). If you have an idea to improve the build quality. We're happy to learn from you :)
IMHO either way our latest release can't build. Let's figure out how to fix it first. Then we can come up with a quick bugfix release with just the apache tar.gz release. No need to publish the artifacts/repos. I was trying to fix the problem but I have limited time so it can be slow. Alex if you have some finding please share to us. Technical discussions are always welcome. Evans Konstantin Boudnik <[email protected]> 於 2020年1月27日 週一 上午5:55寫道： > Actually, it isn't about Bigtop build. Bigtop is using whatever > source/build > materials are particular component is supplying as a part of their > release' source > tree. Yes, we do an alternation or two when the "official" stuff is totally > broken but it is rather an exception. > > In this particular case, Hadoop's builds have fixed on the fly by our own > 'do-component-build' script or to be fixed either in the upstream project. > > -- > Cos > > On Sun, Jan 26, 2020 at 12:47PM, Alexander Frolushkin wrote: > > So, someone need to remade src.rpm for Hadoop, replacing url in .pom for > > this resource to become available? And now Hadoop cannot be build by any > > version of Bigtop? > > > > On Fri, Jan 24, 2020, 12:04 Olaf Flebbe <[email protected]> wrote: > > > > > hi > > > > > > the failed downloads are http:// downloads > > > > > > http download from nexus are now turned off for security reasons. > > > > > > there was another report for oozie with the same problem on this list > iirc. > > > > > > imo this can only be solved by using additional tooling, smthg like a > > > proxy changing http to https . nexus might have a property for that. > or new > > > maven might have smthg in place for working around broken > dependencies, but > > > i am speculating > > > > > > best > > > olaf > > > > > > > > > > > > > Am 24.01.2020 um 05:47 schrieb Konstantin Boudnik <[email protected]>: > > > > > > > > Looks like some sort of trouble with one of the dependencies of > httpfs: > > > > > > > > > > > > Downloading from apache.snapshots.https: > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar > > > > Downloading from repository.jboss.org: > > > > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar > > > > [WARNING] Checksum validation failed, expected but is > > > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss. > > > > org for > > > > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar > > > > [WARNING] Could not validate integrity of download from > > > > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar > : > > > Checksum validation failed, expected but is > > > da39a3ee5e6b4b0d3255bfef95601890afd80709 > > > > [WARNING] Checksum validation failed, expected but is > > > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.org for > > > > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar > > > > Downloaded from repository.jboss.org: > > > > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar > > > (0 B at 0 B/s) > > > > > > > > And then build fails right at that module. Doesn't look like a Bigtop > > > issue, methink. I am on the road right now with somewhat limited > internet, > > > will be back over the weekend and try to look more. Perhaps in the > > > meanwhile someone on the list would have any other ideas. > > > > > > > > -- > > > > Cos > > > > > > > >> On 1/23/20 11:40 PM, Konstantin Boudnik wrote: > > > >> Well, I am not familiar with Gradle enterprise - looks like it has > > > everything indeed, thanks! Let me dig and see what I can figure out. > > > >> > > > >> -- > > > >> Cos > > > >> > > > >>> On 1/23/20 11:28 PM, Alexander Frolushkin wrote: > > > >>> Thank you for responding, Konstantin. > > > >>> > > > >>> Honestly, I was thinking --scan gradle will publish all necessary > > > >>> information about its execution, which may be required to > inverstigate > > > the > > > >>> problem. > > > >>> > > > >>> Anyway, I'm ready to provide maximum data for this dicsussion. > Firstly, > > > >>> here is a complete console output of entire hadoop-pkg-ind task > till > > > the > > > >>> error - > > > >>> > > > > https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk > > > >>> > > > >>> If I need to collect additional data, please direct me, what > exactly I > > > need > > > >>> to do to get it for processing here. > > > >>> > > > >>> I really want to build hadoop repo by bigtop-1.4.0. > > > >>> > > > >>>> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <[email protected]> > > > wrote: > > > >>> > > > >>>> Thanks, Alexander, for bringing this up. > > > >>>> > > > >>>> I don't see anything wrong with the way you're starting the build, > > > >>>> except that official CI is now using "hadoop-pkg-ind" target and > build > > > >>>> then pulls in a correct container automatically. But I see you > already > > > >>>> tried that one. > > > >>>> > > > >>>> As a side note, I am looking at [1] and see the following error > (which > > > >>>> seems to be there for some time): > > > >>>> > > > >>>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not > found > > > in > > > >>>> any of the following sources: > > > >>>> > > > >>>> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace) > > > >>>> - Plugin Repositories (could not resolve plugin artifact > > > >>>> > 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0') > > > >>>> Searched in the following repositories: > > > >>>> Gradle Central Plugin Repository > > > >>>> > > > >>>> Unfortunately, it is hard to tell what kind of error you're > getting in > > > >>>> your build. Any chance you can publish a snippet of the actual > logs, > > > >>>> rather than a screenshot? > > > >>>> > > > >>>> -- > > > >>>> Thanks, > > > >>>> Cos > > > >>>> > > > >>>> [1] > > > >>>> > > > >>>> > > > > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console > > > >>>> > > > >>>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote: > > > >>>>> Hello everyone! > > > >>>>> > > > >>>>> Sorry for repeating this in dev list, but I really need help in > this > > > >>>>> situation. > > > >>>>> > > > >>>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error: > > > >>>>> > > > >>>>> Failed to execute goal > > > >>>>> > > > >>>> > > > > org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies > > > >>>>> (default) on project hadoop-hdfs-httpfs: An error has occurred in > > > >>>>> Dependencies report generation.: zip file is empty -> [Help 1] > > > >>>>> > > > >>>>> Launched in clean bigtop-1.4.0 dir, with command docker run -v > > > `pwd`:/ws > > > >>>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg > > > >>>>> > > > >>>>> I made a --scan for this attempt - > > > https://gradle.com/s/7iu4z3fsbrh62 > > > >>>>> > > > >>>>> Maybe I'm using not a valid way to get it perform correctly? > > > >>>>> > > > >>>>> Or this can be bug and I need to fill a jira ticket for this? > > > >>>>> > > > >>>>> Also I tryed to use direct building by ./gradlew hadoop-rpm and > via > > > >>>> docker > > > >>>>> ./gradlew hadoop-pkg-ind, got the same error. If I use > bigtop-trunk, > > > this > > > >>>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've been > > > told, > > > >>>>> currently hadoop is unbuildable in master branch. > > > >>>>> > > > >>>>> Plus, the same error I get on bigtop-1.3.0. > > > >>>>> > > > >>>>> Cleaning the local maven repository does not fix it. > > > >>>>> > > > >>>>> So, likely, the problem is in building environment. How I can > > > configure > > > >>>> it > > > >>>>> correctly for successful hadoop building in bigtop? > > > >>>>> > > > >>>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <[email protected]> wrote: > > > >>>>> > > > >>>>>> Currently building Hadoop across all platform on master branch > is > > > >>>> failing: > > > >>>>>> > > > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ > > > >>>>>> > > > >>>>>> I'm not sure whether the master code can be built successfully > since > > > >>>> we're > > > >>>>>> in transition of upgrading zookeeper. > > > >>>>>> > > > >>>>>> If you just want to use bigtop, please get our latest release > 1.4.0. > > > >>>>>> > > > >>>>>> If you are willing to contribute, you can report a bug on our > jira > > > first > > > >>>>>> and we ca work on a fix together. > > > >>>>>> > > > >>>>>> Let us know if any question. > > > >>>>>> > > > >>>>>> Evan's > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> Yuqi Gu <[email protected]> 於 2020年1月22日 週三 16:31 寫道： > > > >>>>>> > > > >>>>>>> Hi Alexander, > > > >>>>>>> > > > >>>>>>> It seems bigtop_toolchain didn't deploy correctly. > > > >>>>>>> Please try to build bigtop components in docker with > bigtop/slaves > > > >>>> docker > > > >>>>>>> images < > > > https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos > > > >>>>> . > > > >>>>>>> Run: > > > >>>>>>> cd BIGTOP_SRC_DIR > > > >>>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l > -c 'cd > > > >>>> /ws ; > > > >>>>>>> gradle hadoop-pkg' > > > >>>>>>> > > > >>>>>>> BR, > > > >>>>>>> Yuqi > > > >>>>>>> > > > >>>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin < > > > [email protected] > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails > with > > > other > > > >>>>>>>> error. > > > >>>>>>>> > > > >>>>>>>> Also, in webtop-trunk this particular error appeard while > building > > > >>>>>> hadoop > > > >>>>>>>> in docker (trunk-centos-7) image (by ./gradlew > hadoop-pkg-ind) and > > > >>>>>>> without > > > >>>>>>>> it (by ./gradlew hadoop-rpm). > > > >>>>>>>> > > > >>>>>>>> Performed steps: > > > >>>>>>>> > > > >>>>>>>> I was trying to follow the available guides, do a clean > centos 7 > > > >>>>>> install, > > > >>>>>>>> upgrade to latest official rpms, install openjdk 7 and unzip > from > > > >>>> repo. > > > >>>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply > > > >>>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include > > > >>>>>>>> bigtop_toolchain::installer". > > > >>>>>>>> > > > >>>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew > hadoop-rpm > > > and > > > >>>> got > > > >>>>>>>> this error. > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) < > [email protected]> > > > >>>>>> wrote: > > > >>>>>>>>> Hi Alexander, > > > >>>>>>>>> > > > >>>>>>>>> Which version or branch of Bigtop do you use? and also can > you > > > >>>>>> provide > > > >>>>>>> us > > > >>>>>>>>> the steps you did? > > > >>>>>>>>> > > > >>>>>>>>> Thanks, > > > >>>>>>>>> Youngwoo > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin < > > > >>>>>>> [email protected] > > > >>>>>>>>> wrote: > > > >>>>>>>>> > > > >>>>>>>>>> Hello everyone. > > > >>>>>>>>>> > > > >>>>>>>>>> Looks like Bigtop mailing lists is not very active, but I > still > > > >>>>>> hope > > > >>>>>>>>>> someone could help me. > > > >>>>>>>>>> > > > >>>>>>>>>> We are trying to employ BigTop to build own hadoop > repository > > > >>>>>>>> (./gradlew > > > >>>>>>>>>> hadoop-rpm), but after all efforts still getting error on > > > zookeeper > > > >>>>>>>>> version > > > >>>>>>>>>> dependency: > > > >>>>>>>>>> > > > >>>>>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5 > > > >>>>>>>>>> > > > >>>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6 > > > >>>>>>>>>> > > > >>>>>>>>>> while all other conponents depends on > > > >>>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13 > > > >>>>>>>>>> > > > >>>>>>>>>> so we got: > > > >>>>>>>>>> > > > >>>>>>>>>> [ERROR] Failed to execute goal > > > >>>>>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce > > > >>>>>>> (depcheck) > > > >>>>>>>>> on > > > >>>>>>>>>> project hadoop-yarn-server-tests > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> Please advise me some solution for this show stopper, if > > > possible. > > > >>>>>>>>> Thanks! > > > > > > >
