Hi Jun, these random artifact failures made me build the nexus artifact cache, which I had to disable because it finally tripped over other problems, I was aware but didn't show up before. I don't think just enabling it again will help. I will work on a new solution. Will create a jira to track the efforts.
for the time please do cheat like copying together independent runs, I would recommend. one question: does this random download failures still mostly show on amd64 or do you observe it on ppc64le and aarch64 as well? Olaf Von meinem iPad gesendet > Am 26.10.2018 um 03:27 schrieb Jun HE <[email protected]>: > > Sorry for not updating for a while. > > Actually I was struggling in building the 1.3 RC2 for last two weeks. I > cannot get the Bigtop-1.3.0 job successfully finished in one shot. There've > always been some random artifacts downloading issue, happened for different > arch and distros, caused some subjob failed. See fedora-26 and > opensuse-42.3 on amd64 (https://ci.bigtop.apache.org/job/Bigtop-1.3.0/25/) > > I tried these two subjob in job-26 and they succeeded. And I'm intended to > use their outputs (job-25/26) to prepare RC2. Do you see any risk or > problem it could be? > > Thanks, > > Jun > > Evans Ye <[email protected]> 于2018年10月26日周五 上午1:21写道： > >> Hi Jun, >> >> Could you share the status of 1.3.0 release. Anything you'd like to call >> for help? >> >> Thanks, >> Evans >> >> Jun HE <[email protected]> 於 2018年10月10日 週三 下午6:13寫道： >> >>> Hi, folks, >>> >>> I'm planning to start RC2 preparation on 10-12. Please kindly let me know >>> if you: >>> * need more time to test >>> * found problems in RC1, other than provisioners failure >>> (centos-7/debian-9) >>> * have anything which should be included in RC2 >>> >>> The plan is to start build on 10-12 20:00, Beijing time if no further >>> comments are raised. >>> >>> Thanks & regards, >>> >>> Jun >>> >>> Jun HE <[email protected]> 于2018年10月8日周一 上午8:41写道： >>> >>>> >>>> Sure, I will go to work on RC2. >>>> >>>> Before doing that, I'd like to know if there are other issues you found >>> in >>>> branch-1.3? So I can fix them before baking RC2. :) >>>> >>>> Konstantin Boudnik <[email protected]> 于2018年10月8日周一 上午2:38写道： >>>> >>>>> So, looks like the vote needs to be stopped until RC2 is ready, right? >>>>> Otherwise, things will become confusing... >>>>> >>>>> Cos >>>>> >>>>>> On Wed, Oct 03, 2018 at 07:07PM, Jun HE wrote: >>>>>> I've tested CentOS-7 with 4.12.0 puppetlabs-stdlib, will submit >> patch >>>>> soon. >>>>>> For Debian, I'll update the config file as you suggested, and >>>>> centos/fedora >>>>>> config as well. >>>>>> They should be ready in today for review. >>>>>> >>>>>> Evans Ye <[email protected]> 于2018年10月3日周三 下午1:48写道： >>>>>> >>>>>>> For provisioner part, I tested CentOS, Ubuntu, and Debian. >>>>>>> Currently only Ubuntu works. >>>>>>> * CentOS: Need to pin down the puppetlabs-stdlib to 4.12.0 as Jun >> He >>>>>>> suggested >>>>>>> * Debian: Just need to update config file from debian 8 to debian >> 9. >>>>>>> >>>>>>> Jun are you already working on this or you need my help fixing >> them? >>>>>>> >>>>>>> >>>>>>> Jun HE <[email protected]> 於 2018年9月30日 週日 下午2:30寫道： >>>>>>> >>>>>>>> Hi Olaf, >>>>>>>> >>>>>>>> For debian provision test, use bigtop/puppet:1.3.0-debian-9 >> should >>>>> work. >>>>>>>> As for issue in centos-7, it should be the same of previous >>>>> puppet-stdlib >>>>>>>> dependency (See: >>> https://tickets.puppetlabs.com/browse/MODULES-3962 >>>>> ). >>>>>>> Pin >>>>>>>> puppet-stdlib to 4.12.0 should fix this problem. >>>>>>>> I'll do test and submit fix if local verification is OK. >>>>>>>> >>>>>>>> >>>>>>>> Olaf Flebbe <[email protected]> 于2018年9月30日周日 上午12:03写道： >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I tried the docker provisioner and it failed for debian >> because >>>>>>>> /sbin/init >>>>>>>>> not present in bigtop/puppet:trunk-debian-9 >>>>>>>>> >>>>>>>>> --- config -- >>>>>>>>> >>>>>>>>> docker: >>>>>>>>> memory_limit: "4g" >>>>>>>>> image: "bigtop/puppet:trunk-debian-9" >>>>>>>>> >>>>>>>>> repo: " >>>>> http://repos.bigtop.apache.org/releases/1.3.0/debian/9/x86_64"; >>>>>>>>> distro: debian >>>>>>>>> components: [hdfs, yarn, mapreduce] >>>>>>>>> enable_local_repo: false >>>>>>>>> smoke_test_components: [hdfs, yarn, mapreduce] >>>>>>>>> --------- >>>>>>>>> I can live with it, since docker provisioner had many issues >>> with >>>>>>> systemd >>>>>>>>> in the past: >>>>>>>>> >>>>>>>>> But it failed for centos-7 too. >>>>>>>>> >>>>>>>>> + docker exec >>>>>>>>> >> 8d3cf8113318d6b695a85742b985b65d6e146152a7019994dfb8fd8cf0017297 >>>>> bash >>>>>>> -c >>>>>>>>> 'puppet apply --parser future >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >> --modulepath=/bigtop-home/bigtop-deploy/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules >>>>>>>>> /bigtop-home/bigtop-deploy/puppet/manifests' >>>>>>>>> + future='--parser future' >>>>>>>>> + docker exec >>>>>>>>> >> c18b89c4cfd0847d71798b0bf5a86db01c6f3becb42b8ba293343227a71a8717 >>>>> bash >>>>>>> -c >>>>>>>>> 'puppet apply --parser future >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >> --modulepath=/bigtop-home/bigtop-deploy/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules >>>>>>>>> /bigtop-home/bigtop-deploy/puppet/manifests' >>>>>>>>> + future='--parser future' >>>>>>>>> + docker exec >>>>>>>>> >> 79b0a9ce1ae5d25f25cbf35c3e3b8e26131ca556a2b046eca56737020ffbd31e >>>>> bash >>>>>>> -c >>>>>>>>> 'puppet apply --parser future >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >> --modulepath=/bigtop-home/bigtop-deploy/puppet/modules:/etc/puppet/modules:/usr/share/puppet/modules >>>>>>>>> /bigtop-home/bigtop-deploy/puppet/manifests' >>>>>>>>> Warning: Config file /etc/puppet/hiera.yaml not found, using >>> Hiera >>>>>>>> defaults >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in >> any >>>>> Hiera >>>>>>>> data >>>>>>>>> file and no default supplied on node >>>>> 8d3cf8113318.bigtop.apache.org >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in >> any >>>>> Hiera >>>>>>>> data >>>>>>>>> file and no default supplied on node >>>>> 8d3cf8113318.bigtop.apache.org >>>>>>>>> Warning: Config file /etc/puppet/hiera.yaml not found, using >>> Hiera >>>>>>>> defaults >>>>>>>>> Warning: Config file /etc/puppet/hiera.yaml not found, using >>> Hiera >>>>>>>> defaults >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in >> any >>>>> Hiera >>>>>>>> data >>>>>>>>> file and no default supplied on node >>>>> c18b89c4cfd0.bigtop.apache.org >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in >> any >>>>> Hiera >>>>>>>> data >>>>>>>>> file and no default supplied on node >>>>> c18b89c4cfd0.bigtop.apache.org >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in >> any >>>>> Hiera >>>>>>>> data >>>>>>>>> file and no default supplied on node >>>>> 79b0a9ce1ae5.bigtop.apache.org >>>>>>>>> Error: Could not find data item bigtop::hadoop_head_node in >> any >>>>> Hiera >>>>>>>> data >>>>>>>>> file and no default supplied on node >>>>> 79b0a9ce1ae5.bigtop.apache.org >>>>>>>>> >>>>>>>>> ---- >>>>>>>>> >>>>>>>>> Evans, what do you think: Can you reproduce issues ? >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> >>>>>>>>> Olaf >>>>>>>>> >>>>>>>>> >>>>>>>>>> Am 29.09.2018 um 12:11 schrieb Jun HE <[email protected]>: >>>>>>>>>> >>>>>>>>>> Already updated. >>>>>>>>>> >>>>>>>>>> Jun HE <[email protected]> 于2018年9月29日周六 下午3:20写道： >>>>>>>>>> >>>>>>>>>>> Sure. Will update them today. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Evans Ye <[email protected]> 于2018年9月29日周六 下午2:32写道： >>>>>>>>>>> >>>>>>>>>>>> I need some extra time to perform evaluation and give my >> +1 >>>>> out. >>>>>>>>>>>> I'll be back to Taiwan tomorrow which I'll have full >> network >>>>>>>>>>>> accessibility. >>>>>>>>>>>> However, if you got enough vote for the release, go ahead >>> w/o >>>>>>> waiting >>>>>>>>> me. >>>>>>>>>>>> >>>>>>>>>>>> BTW, I poked around for the checksum, it seems that our >>>>> checksum >>>>>>> for >>>>>>>> a >>>>>>>>>>>> release is gathered from nexus, which is probably >> generated >>> by >>>>>>> maven. >>>>>>>>>>>> Though there's no SHA256/512 available for download, we >> can >>>>> just >>>>>>>>> generate >>>>>>>>>>>> one by ourselves and commit it in SVN for releasing >>>>>>>>>>>> Seems that Flink is doing in this way: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >> https://github.com/apache/flink/blob/master/tools/releasing/create_binary_release.sh#L85 >>>>>>>>>>>> >>>>>>>>>>>> Jun, would you please update the checksum for the release? >>>>>>>>>>>> >>>>>>>>>>>> Best, >>>>>>>>>>>> Evans >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Evans Ye <[email protected]> 於 2018年9月27日 週四 上午12:47寫道： >>>>>>>>>>>> >>>>>>>>>>>>> When trying to evaluate the release, I noticed that [1] >>>>> requires >>>>>>>>> SHA-256 >>>>>>>>>>>>> or SHA-512 signature. >>>>>>>>>>>>> While our release process currently produces SHA1 and >> MD5. >>>>> These 2 >>>>>>>>> must >>>>>>>>>>>>> not be supplied. >>>>>>>>>>>>> I didn't dig in to it yet. >>>>>>>>>>>>> Roman, Cos would mind to point out where to look for? >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>> http://www.apache.org/dev/release-distribution#sigs-and-sums >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Naresh Bhat <[email protected]> 於 2018年9月26日 週三 >>>>> 下午4:51寫道： >>>>>>>>>>>>> >>>>>>>>>>>>>> +1 >>>>>>>>>>>>>> I have compiled v1.3 branch and executing the smoke >> tests >>> on >>>>>>> ARM64 >>>>>>>>>>>>>> machine. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, 26 Sep 2018 at 08:44, Jun HE <[email protected]> >>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> This is the vote for release 1.3.0 of Apache Bigtop. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> It fixes the following issues: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12338976&projectId=12311420 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The vote will be going for at least 72 hours and will >> be >>>>> closed >>>>>>> on >>>>>>>>>>>>>>> Saturday, >>>>>>>>>>>>>>> September 29, 2018 at noon PDT. Please download, test >>> and >>>>> vote >>>>>>>> with >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [ ] +1, accept RC1 as the official 1.3.0 release of >>> Apache >>>>>>> Bigtop >>>>>>>>>>>>>>> [ ] +0, I don't care either way, >>>>>>>>>>>>>>> [ ] -1, do not accept RC1 as the official 1.3.0 release >>> of >>>>>>> Apache >>>>>>>>>>>>>> Bigtop, >>>>>>>>>>>>>>> because... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Source and binary files: >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>> https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.3.0-RC1/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Maven staging repo: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >> https://repository.apache.org/content/repositories/orgapachebigtop-1019 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The git tag to be voted upon is release-1.3.0 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Bigtop's KEYS file containing PGP keys we use to sign >> the >>>>>>> release: >>>>>>>>>>>>>>> >>>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> >>
