Hi, Olaf, These downloading failures were observed on other archs as well. Just randomly happend. And I saw several times the failure happened in zeppelin's zeppelin-web.
Regards, Jun Olaf Flebbe <[email protected]> 于2018年10月26日周五 下午12:52写道： > Hi Jun, > > these random artifact failures made me build the nexus artifact cache, > which I had to disable because it finally tripped over other problems, I > was aware but didn't show up before. I don't think just enabling it again > will help. I will work on a new solution. Will create a jira to track the > efforts. > > for the time please do cheat like copying together independent runs, I > would recommend. > > one question: does this random download failures still mostly show on > amd64 or do you observe it on ppc64le and aarch64 as well? > > Olaf > > Von meinem iPad gesendet > > > Am 26.10.2018 um 03:27 schrieb Jun HE <[email protected]>: > > > > Sorry for not updating for a while. > > > > Actually I was struggling in building the 1.3 RC2 for last two weeks. I > > cannot get the Bigtop-1.3.0 job successfully finished in one shot. > There've > > always been some random artifacts downloading issue, happened for > different > > arch and distros, caused some subjob failed. See fedora-26 and > > opensuse-42.3 on amd64 ( > https://ci.bigtop.apache.org/job/Bigtop-1.3.0/25/) > > > > I tried these two subjob in job-26 and they succeeded. And I'm intended > to > > use their outputs (job-25/26) to prepare RC2. Do you see any risk or > > problem it could be? > > > > Thanks, > > > > Jun > > > > Evans Ye <[email protected]> 于2018年10月26日周五 上午1:21写道： > > > >> Hi Jun, > >> > >> Could you share the status of 1.3.0 release. Anything you'd like to call > >> for help? > >> > >> Thanks, > >> Evans > >> > >> Jun HE <[email protected]> 於 2018年10月10日 週三 下午6:13寫道： > >> > >>> Hi, folks, > >>> > >>> I'm planning to start RC2 preparation on 10-12. Please kindly let me > know > >>> if you: > >>> * need more time to test > >>> * found problems in RC1, other than provisioners failure > >>> (centos-7/debian-9) > >>> * have anything which should be included in RC2 > >>> > >>> The plan is to start build on 10-12 20:00, Beijing time if no further > >>> comments are raised. > >>> > >>> Thanks & regards, > >>> > >>> Jun > >>> > >>> Jun HE <[email protected]> 于2018年10月8日周一 上午8:41写道： > >>> > >>>> > >>>> Sure, I will go to work on RC2. > >>>> > >>>> Before doing that, I'd like to know if there are other issues you > found > >>> in > >>>> branch-1.3? So I can fix them before baking RC2. :) > >>>> > >>>> Konstantin Boudnik <[email protected]> 于2018年10月8日周一 上午2:38写道： > >>>> > >>>>> So, looks like the vote needs to be stopped until RC2 is ready, > right? > >>>>> Otherwise, things will become confusing... > >>>>> > >>>>> Cos > >>>>> > >>>>>> On Wed, Oct 03, 2018 at 07:07PM, Jun HE wrote: > >>>>>> I've tested CentOS-7 with 4.12.0 puppetlabs-stdlib, will submit > >> patch > >>>>> soon. > >>>>>> For Debian, I'll update the config file as you suggested, and > >>>>> centos/fedora > >>>>>> config as well. > >>>>>> They should be ready in today for review. > >>>>>> > >>>>>> Evans Ye <[email protected]> 于2018年10月3日周三 下午1:48写道： > >>>>>> > >>>>>>> For provisioner part, I tested CentOS, Ubuntu, and Debian. > >>>>>>> Currently only Ubuntu works. > >>>>>>> * CentOS: Need to pin down the puppetlabs-stdlib to 4.12.0 as Jun > >> He > >>>>>>> suggested > >>>>>>> * Debian: Just need to update config file from debian 8 to debian > >> 9. > >>>>>>> > >>>>>>> Jun are you already working on this or you need my help fixing > >> them? > >>>>>>> > >>>>>>> > >>>>>>> Jun HE <[email protected]> 於 2018年9月30日 週日 下午2:30寫道： > >>>>>>> > >>>>>>>> Hi Olaf, > >>>>>>>> > >>>>>>>> For debian provision test, use bigtop/puppet:1.3.0-debian-9 > >> should > >>>>> work. > >>>>>>>> As for issue in centos-7, it should be the same of previous > >>>>> puppet-stdlib > >>>>>>>> dependency (See: > >>> https://tickets.puppetlabs.com/browse/MODULES-3962 > >>>>> ). > >>>>>>> Pin > >>>>>>>> puppet-stdlib to 4.12.0 should fix this problem. > >>>>>>>> I'll do test and submit fix if local verification is OK. > >>>>>>>> > >>>>>>>> > >>>>>>>> Olaf Flebbe <[email protected]> 于2018年9月30日周日 上午12:03写道： > >>>>>>>> > >>>>>>>>> Hi, > >>>>>>>>> > >>>>>>>>> I tried the docker provisioner and it failed for debian > >> because > >>>>>>>> /sbin/init > >>>>>>>>> not present in bigtop/puppet:trunk-debian-9 > >>>>>>>>> > >>>>>>>>> --- config -- > >>>>>>>>> > >>>>>>>>> docker: > >>>>>>>>> memory_limit: "4g" > >>>>>>>>> image: "bigtop/puppet:trunk-debian-9" > >>>>>>>>> > >>>>>>>>> repo: " > >>>>> http://repos.bigtop.apache.org/releases/1.3.0/debian/9/x86_64"; > >>>>>>>>> distro: debian > >>>>>>>>> components: [hdfs, yarn, mapreduce] > >>>>>>>>> enable_local_repo: false > >>>>>>>>> smoke_test_components: [hdfs, yarn, mapreduce] > >>>>>>>>> --------- > >>>>>>>>> I can live with it, since docker provisioner had many issues > >>> with > >>>>>>> systemd > >>>>>>>>> in the past: > >>>>>>>>> > >>>>>>>>> But it failed for centos-7 too. > >>>>>>>>> > >>>>>>>>> + docker exec > >>>>>>>>> > >> 8d3cf8113318d6b695a85742b985b65d6e146152a7019994dfb8fd8cf0017297 > >>>>> bash > >>>>>>> -c > >>>>>>>>> 'puppet apply --parser future > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>> > >> > --modulepath=/bigtop-home/bigtop-deploy/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules > >>>>>>>>> /bigtop-home/bigtop-deploy/puppet/manifests' > >>>>>>>>> + future='--parser future' > >>>>>>>>> + docker exec > >>>>>>>>> > >> c18b89c4cfd0847d71798b0bf5a86db01c6f3becb42b8ba293343227a71a8717 > >>>>> bash > >>>>>>> -c > >>>>>>>>> 'puppet apply --parser future > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>> > >> > --modulepath=/bigtop-home/bigtop-deploy/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules > >>>>>>>>> /bigtop-home/bigtop-deploy/puppet/manifests' > >>>>>>>>> + future='--parser future' > >>>>>>>>> + docker exec > >>>>>>>>> > >> 79b0a9ce1ae5d25f25cbf35c3e3b8e26131ca556a2b046eca56737020ffbd31e > >>>>> bash > >>>>>>> -c > >>>>>>>>> 'puppet apply --parser future > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>> > >> > --modulepath=/bigtop-home/bigtop-deploy/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules > >>>>>>>>> /bigtop-home/bigtop-deploy/puppet/manifests' > >>>>>>>>> Warning: Config file /etc/puppet/hiera.yaml not found, using > >>> Hiera > >>>>>>>> defaults > >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in > >> any > >>>>> Hiera > >>>>>>>> data > >>>>>>>>> file and no default supplied on node > >>>>> 8d3cf8113318.bigtop.apache.org > >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in > >> any > >>>>> Hiera > >>>>>>>> data > >>>>>>>>> file and no default supplied on node > >>>>> 8d3cf8113318.bigtop.apache.org > >>>>>>>>> Warning: Config file /etc/puppet/hiera.yaml not found, using > >>> Hiera > >>>>>>>> defaults > >>>>>>>>> Warning: Config file /etc/puppet/hiera.yaml not found, using > >>> Hiera > >>>>>>>> defaults > >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in > >> any > >>>>> Hiera > >>>>>>>> data > >>>>>>>>> file and no default supplied on node > >>>>> c18b89c4cfd0.bigtop.apache.org > >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in > >> any > >>>>> Hiera > >>>>>>>> data > >>>>>>>>> file and no default supplied on node > >>>>> c18b89c4cfd0.bigtop.apache.org > >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in > >> any > >>>>> Hiera > >>>>>>>> data > >>>>>>>>> file and no default supplied on node > >>>>> 79b0a9ce1ae5.bigtop.apache.org > >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in > >> any > >>>>> Hiera > >>>>>>>> data > >>>>>>>>> file and no default supplied on node > >>>>> 79b0a9ce1ae5.bigtop.apache.org > >>>>>>>>> > >>>>>>>>> ---- > >>>>>>>>> > >>>>>>>>> Evans, what do you think: Can you reproduce issues ? > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> > >>>>>>>>> Olaf > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> Am 29.09.2018 um 12:11 schrieb Jun HE <[email protected]>: > >>>>>>>>>> > >>>>>>>>>> Already updated. > >>>>>>>>>> > >>>>>>>>>> Jun HE <[email protected]> 于2018年9月29日周六 下午3:20写道： > >>>>>>>>>> > >>>>>>>>>>> Sure. Will update them today. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Evans Ye <[email protected]> 于2018年9月29日周六 下午2:32写道： > >>>>>>>>>>> > >>>>>>>>>>>> I need some extra time to perform evaluation and give my > >> +1 > >>>>> out. > >>>>>>>>>>>> I'll be back to Taiwan tomorrow which I'll have full > >> network > >>>>>>>>>>>> accessibility. > >>>>>>>>>>>> However, if you got enough vote for the release, go ahead > >>> w/o > >>>>>>> waiting > >>>>>>>>> me. > >>>>>>>>>>>> > >>>>>>>>>>>> BTW, I poked around for the checksum, it seems that our > >>>>> checksum > >>>>>>> for > >>>>>>>> a > >>>>>>>>>>>> release is gathered from nexus, which is probably > >> generated > >>> by > >>>>>>> maven. > >>>>>>>>>>>> Though there's no SHA256/512 available for download, we > >> can > >>>>> just > >>>>>>>>> generate > >>>>>>>>>>>> one by ourselves and commit it in SVN for releasing > >>>>>>>>>>>> Seems that Flink is doing in this way: > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>> > >> > https://github.com/apache/flink/blob/master/tools/releasing/create_binary_release.sh#L85 > >>>>>>>>>>>> > >>>>>>>>>>>> Jun, would you please update the checksum for the release? > >>>>>>>>>>>> > >>>>>>>>>>>> Best, > >>>>>>>>>>>> Evans > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Evans Ye <[email protected]> 於 2018年9月27日 週四 上午12:47寫道： > >>>>>>>>>>>> > >>>>>>>>>>>>> When trying to evaluate the release, I noticed that [1] > >>>>> requires > >>>>>>>>> SHA-256 > >>>>>>>>>>>>> or SHA-512 signature. > >>>>>>>>>>>>> While our release process currently produces SHA1 and > >> MD5. > >>>>> These 2 > >>>>>>>>> must > >>>>>>>>>>>>> not be supplied. > >>>>>>>>>>>>> I didn't dig in to it yet. > >>>>>>>>>>>>> Roman, Cos would mind to point out where to look for? > >>>>>>>>>>>>> > >>>>>>>>>>>>> [1] > >>>>> http://www.apache.org/dev/release-distribution#sigs-and-sums > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Naresh Bhat <[email protected]> 於 2018年9月26日 週三 > >>>>> 下午4:51寫道： > >>>>>>>>>>>>> > >>>>>>>>>>>>>> +1 > >>>>>>>>>>>>>> I have compiled v1.3 branch and executing the smoke > >> tests > >>> on > >>>>>>> ARM64 > >>>>>>>>>>>>>> machine. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Wed, 26 Sep 2018 at 08:44, Jun HE <[email protected]> > >>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> This is the vote for release 1.3.0 of Apache Bigtop. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> It fixes the following issues: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12338976&projectId=12311420 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The vote will be going for at least 72 hours and will > >> be > >>>>> closed > >>>>>>> on > >>>>>>>>>>>>>>> Saturday, > >>>>>>>>>>>>>>> September 29, 2018 at noon PDT. Please download, test > >>> and > >>>>> vote > >>>>>>>> with > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> [ ] +1, accept RC1 as the official 1.3.0 release of > >>> Apache > >>>>>>> Bigtop > >>>>>>>>>>>>>>> [ ] +0, I don't care either way, > >>>>>>>>>>>>>>> [ ] -1, do not accept RC1 as the official 1.3.0 release > >>> of > >>>>>>> Apache > >>>>>>>>>>>>>> Bigtop, > >>>>>>>>>>>>>>> because... > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Source and binary files: > >>>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>> https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.3.0-RC1/ > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Maven staging repo: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>> > >> https://repository.apache.org/content/repositories/orgapachebigtop-1019 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The git tag to be voted upon is release-1.3.0 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Bigtop's KEYS file containing PGP keys we use to sign > >> the > >>>>>>> release: > >>>>>>>>>>>>>>> > >>>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>>> > >>> > >> > >
