On Wed, Nov 04, 2015 at 01:45AM, Evans Ye wrote: > Yes, I've updated the wiki. > https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide > > Then I guess we're all good. > Thank you both Andrew and Cos for fixing this. > > I'm thinking that we can take one step further which is to setup a CNAME > pointing to > repos.bigtop.apache.org --> bigtop-repos.s3.amazonaws.com
Great idea! Love it - INFRA-10710. > So that the default repo in puppet recipe as well as vagrantconfig.yaml can > be fixed w/o updating with releases. And this can be helpful for migration > if we find resources to host our artifacts. > > Evans > > > 2015-11-03 6:19 GMT+08:00 Konstantin Boudnik <[email protected]>: > > > And done, including the update for the dist repo files and the "How to > > Release" wiki page. Thanks for setting up the bucket, Andrew! > > > > Evans, do we need to update anything else? > > Cos > > > > On Mon, Nov 02, 2015 at 01:52PM, Andrew Purtell wrote: > > > Thanks Cos! > > > > > > > > > On Mon, Nov 2, 2015 at 12:53 PM, Konstantin Boudnik <[email protected]> > > wrote: > > > > > > > Centos bits are up (the rest are still in-flight). So if anyone wants > > to > > > > give > > > > it a try, new repos are at: > > > > > > > > [bigtop] > > > > name=Bigtop > > > > enabled=1 > > > > gpgcheck=1 > > > > type=NONE > > > > baseurl= > > > > http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64 > > > > gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS > > > > > > > > > > > > [bigtop] > > > > name=Bigtop > > > > enabled=1 > > > > gpgcheck=1 > > > > type=NONE > > > > baseurl= > > > > http://bigtop-repos.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64 > > > > gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS > > > > > > > > Once everything is uploaded I will update the repo file for the 1.0 on > > the > > > > dist. > > > > > > > > Please report here if you find any issues with these two. Thanks! > > > > Cos > > > > > > > > On Mon, Nov 02, 2015 at 06:28AM, Brian Long wrote: > > > > > Just wanted to volunteer to test as well, really appreciate the work > > you > > > > > guys are doing! > > > > > On Nov 1, 2015 9:14 PM, "Konstantin Boudnik" <[email protected]> wrote: > > > > > > > > > > > Thanks Andrew, I will test it in the morning and if everything is > > fine > > > > we > > > > > > should have at least official 1.0 back online. > > > > > > > > > > > > Cos > > > > > > > > > > > > On Sun, Nov 01, 2015 at 06:06PM, Andrew Purtell wrote: > > > > > > > I sent credentials for an IAM user with permissions on those > > buckets > > > > to > > > > > > > private@bigtop. > > > > > > > > > > > > > > > > > > > > > On Sat, Oct 31, 2015 at 12:33 PM, Konstantin Boudnik < > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Andrew, > > > > > > > > > > > > > > > > how we can upload the content to the buckets you have created? > > I > > > > think > > > > > > we > > > > > > > > need > > > > > > > > to push out signed 1.0 bits, that I still have locally, and put > > > > this > > > > > > > > inconvenience behind us. > > > > > > > > > > > > > > > > Please let me know how I can get the credentials for the > > bucket. > > > > > > Thanks! > > > > > > > > Cos > > > > > > > > > > > > > > > > On Thu, Oct 22, 2015 at 05:35PM, Andrew Purtell wrote: > > > > > > > > > I don't really care but what's wrong with the ones I already > > > > claimed? > > > > > > > > > > > > > > > > > > > On Oct 22, 2015, at 3:45 PM, Konstantin Boudnik < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > It might be some sort of name-retention policy in action > > on AWS > > > > > > part. > > > > > > > > Ah... > > > > > > > > > > well, a lesson for the future. > > > > > > > > > > > > > > > > > > > > Shall we get the bucket 'asfbigtop' to make it clear in the > > > > future? > > > > > > > > > > Cos > > > > > > > > > > > > > > > > > > > >> On Thu, Oct 22, 2015 at 03:07PM, Andrew Purtell wrote: > > > > > > > > > >> Then unfortunately we should just move on, unless we want > > to > > > > try > > > > > > and > > > > > > > > get it > > > > > > > > > >> from the current owner (using a trademark claim?). That > > would > > > > be > > > > > > at > > > > > > > > least > > > > > > > > > >> time consuming. Want to use one of the buckets I managed > > to > > > > > > reserve? > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >>> On Oct 22, 2015, at 2:58 PM, Sean Mackrory < > > > > [email protected] > > > > > > > > > > > > > > > wrote: > > > > > > > > > >>> > > > > > > > > > >>> Yeah I'm afraid the account is gone, and that bucket is > > not > > > > under > > > > > > > > any other > > > > > > > > > >>> account in our control. So if the bucket is not > > available, > > > > > > somebody > > > > > > > > else > > > > > > > > > >>> must have claimed it. > > > > > > > > > >>> > > > > > > > > > >>>> On Thu, Oct 22, 2015 at 9:54 AM, Evans Ye < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > >>>> > > > > > > > > > >>>> The best case is that Sean gets back with a news that > > the > > > > > > bucket is > > > > > > > > still > > > > > > > > > >>>> managed under Cloudera so that we might be able to > > backup > > > > > > things > > > > > > > > out and > > > > > > > > > >>>> then delete the bucket. > > > > > > > > > >>>> The worst case is someone took bigtop bucket so that we > > can > > > > only > > > > > > > > put 1.0 > > > > > > > > > >>>> packages in buckets created by Andrew and then update > > the > > > > 1.0 > > > > > > > > release repo > > > > > > > > > >>>> files. > > > > > > > > > >>>> > > > > > > > > > >>>> 2015-10-21 13:39 GMT+08:00 Konstantin Boudnik < > > > > [email protected]>: > > > > > > > > > >>>> > > > > > > > > > >>>>>> On Tue, Oct 20, 2015 at 09:00PM, Andrew Purtell wrote: > > > > > > > > > >>>>>> Thanks Evans. > > > > > > > > > >>>>>> > > > > > > > > > >>>>>> Cos: We do have the option to make new point releases > > and > > > > > > > > convenience > > > > > > > > > >>>>> repos > > > > > > > > > >>>>>> from older code if someone asks, but I suspect there > > > > won't be > > > > > > such > > > > > > > > > >>>>> demand. > > > > > > > > > >>>>> > > > > > > > > > >>>>> True, we can. There's already a JIRA about the absence > > of > > > > 0.6 - > > > > > > > > that's > > > > > > > > > >>>> why > > > > > > > > > >>>>> I > > > > > > > > > >>>>> brought up this point. > > > > > > > > > >>>>> > > > > > > > > > >>>>> Cos > > > > > > > > > >>>>> > > > > > > > > > >>>>>>> On Oct 20, 2015, at 5:25 PM, Konstantin Boudnik < > > > > > > [email protected]> > > > > > > > > > >>>>> wrote: > > > > > > > > > >>>>>>> > > > > > > > > > >>>>>>> Another part here is that _all_ our historical > > releases > > > > are > > > > > > gone > > > > > > > > ;( > > > > > > > > > >>>>>>> > > > > > > > > > >>>>>>> It might be not a huge issue as we clearly encourage > > our > > > > > > users to > > > > > > > > > >>>> stay > > > > > > > > > >>>>> on the > > > > > > > > > >>>>>>> later stuff, but still it's a blow to the project. > > One of > > > > > > those > > > > > > > > > >>>>> non-so-obvious > > > > > > > > > >>>>>>> things that are very clear when looked upon in the > > > > rear-view > > > > > > > > mirror. > > > > > > > > > >>>>>>> > > > > > > > > > >>>>>>> Cos > > > > > > > > > >>>>>>> > > > > > > > > > >>>>>>>> On Wed, Oct 21, 2015 at 02:34AM, Evans Ye wrote: > > > > > > > > > >>>>>>>> Thanks Andrew! > > > > > > > > > >>>>>>>> For temporarily dev/test usage, I've built packages > > > > back on > > > > > > our > > > > > > > > CI > > > > > > > > > >>>>> server: > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> http://ci.bigtop.apache.org:8080/view/Releases/ > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> And added some wiki for users: > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Bigtop+1.0.0+with+Bigtop+Provisioner > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> I think this is enough, if it won't take too long > > to get > > > > > > bigtop > > > > > > > > > >>>>> bucket back > > > > > > > > > >>>>>>>> online. > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> OTOH, BIGTOP-2092 < > > > > > > > > > >>>> https://issues.apache.org/jira/browse/BIGTOP-2092> > > > > > > > > > >>>>> reveals > > > > > > > > > >>>>>>>> that all the historical release artifacts are not > > > > available > > > > > > as > > > > > > > > well. > > > > > > > > > >>>>>>>> My thought is to just provide 1.0 release artifacts. > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> Cos, > > > > > > > > > >>>>>>>> right now I haven't used up the resource provided by > > > > Tom for > > > > > > > > CI. If > > > > > > > > > >>>>> needed > > > > > > > > > >>>>>>>> I think we can re-negotiate with Tom for > > rearrangement > > > > to > > > > > > get > > > > > > > > some > > > > > > > > > >>>> S3 > > > > > > > > > >>>>>>>> resources. > > > > > > > > > >>>>>>>> Anyhow, if you need help, just ping me. > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> 2015-10-21 0:22 GMT+08:00 Andrew Purtell < > > > > > > [email protected]>: > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>>> FWIW, I was able to create the following buckets > > under > > > > my > > > > > > > > account: > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> bigtop-repo > > > > > > > > > >>>>>>>>> bigtop-repos > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> If you want to go with one of these, even if only > > > > > > temporarily, > > > > > > > > let > > > > > > > > > >>>>> me know > > > > > > > > > >>>>>>>>> and I'll send the PMC access credentials of an IAM > > user > > > > > > with > > > > > > > > full > > > > > > > > > >>>>> perms > > > > > > > > > >>>>>>>>> over these buckets. > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> On Mon, Oct 19, 2015 at 2:08 PM, Sean Mackrory < > > > > > > > > > >>>> [email protected] > > > > > > > > > >>>>>> > > > > > > > > > >>>>>>>>> wrote: > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>>>>> Sean, could you figure out how this can be done? > > > > > > > > > >>>>>>>>>> > > > > > > > > > >>>>>>>>>> Sure. Although if the bucket ceased to exist when > > the > > > > > > account > > > > > > > > was > > > > > > > > > >>>>>>>>>> terminated, the name should have been free again > > with > > > > a > > > > > > > > couple of > > > > > > > > > >>>>> hours. > > > > > > > > > >>>>>>>>> So > > > > > > > > > >>>>>>>>>> either the bucket has NOT ceased to exist, or > > someone > > > > > > other > > > > > > > > than > > > > > > > > > >>>>> Cloudera > > > > > > > > > >>>>>>>>>> now has the bucket name. > > > > > > > > > >>>>>>>>>> > > > > > > > > > >>>>>>>>>> I'll find out which... > > > > > > > > > >>>>>>>>>> > > > > > > > > > >>>>>>>>>> On Mon, Oct 19, 2015 at 12:56 PM, Konstantin > > Boudnik < > > > > > > > > > >>>>> [email protected]> > > > > > > > > > >>>>>>>>>> wrote: > > > > > > > > > >>>>>>>>>> > > > > > > > > > >>>>>>>>>>>> On Sun, Oct 18, 2015 at 03:30PM, Evans Ye wrote: > > > > > > > > > >>>>>>>>>>>> Guys I've a bad news. > > > > > > > > > >>>>>>>>>>>> I'm guessing that our official released 1.0 > > repos > > > > on S3 > > > > > > are > > > > > > > > also > > > > > > > > > >>>>> tied > > > > > > > > > >>>>>>>>>>> with > > > > > > > > > >>>>>>>>>>>> Cloudera's credentials, which is why we no > > longer > > > > have > > > > > > S3 > > > > > > > > repos > > > > > > > > > >>>>>>>>>> available > > > > > > > > > >>>>>>>>>>>> for users to consume now(I've tried centos6 and > > > > > > debian8). > > > > > > > > > >>>>>>>>>>>> That's really bad in user experience. :( > > > > > > > > > >>>>>>>>>>>> Cos, > > > > > > > > > >>>>>>>>>>>> * Could you please confirm where we put 1.0 > > repos > > > > on? > > > > > > > > > >>>>>>>>>>>> * If my guess is right, do you still have > > copies of > > > > > > signed > > > > > > > > repos > > > > > > > > > >>>>> at > > > > > > > > > >>>>>>>>>> local > > > > > > > > > >>>>>>>>>>>> for restore? > > > > > > > > > >>>>>>>>>>> > > > > > > > > > >>>>>>>>>>> I do have the copy of the repos, so it should be > > an > > > > easy > > > > > > > > exercise > > > > > > > > > >>>>> to > > > > > > > > > >>>>>>>>>>> restore > > > > > > > > > >>>>>>>>>>> them to the new location. > > > > > > > > > >>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>> * I don't have knowledge of how our current S3 > > > > > > resources are > > > > > > > > > >>>> being > > > > > > > > > >>>>>>>>>>> managed. > > > > > > > > > >>>>>>>>>>>> And we don't have S3 resources available from > > Tom's > > > > > > team, > > > > > > > > hence > > > > > > > > > >>>> I > > > > > > > > > >>>>>>>>> think > > > > > > > > > >>>>>>>>>>> we > > > > > > > > > >>>>>>>>>>>> need to plan for this now. > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>> Evans > > > > > > > > > >>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>> On Sun, Oct 18, 2015 at 10:53AM, Andrew Purtell > > > > wrote: > > > > > > > > > >>>>>>>>>>>> Since the binary convenience artifacts are not > > an > > > > > > official > > > > > > > > > >>>> release > > > > > > > > > >>>>>>>>>>> artifact, > > > > > > > > > >>>>>>>>>>>> only the source tarball is, then any of us can > > feel > > > > > > free to > > > > > > > > use > > > > > > > > > >>>>> the > > > > > > > > > >>>>>>>>>>> official > > > > > > > > > >>>>>>>>>>>> 1.0 release tarball to generate a new set of > > > > packages, > > > > > > store > > > > > > > > > >>>> them > > > > > > > > > >>>>> at > > > > > > > > > >>>>>>>>>> new > > > > > > > > > >>>>>>>>>>>> locations, and update pointers to that location. > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>> The new location *could* be Apache dist. Other > > > > projects > > > > > > host > > > > > > > > > >>>> their > > > > > > > > > >>>>>>>>>>>> convenience artifacts there. We need to > > consider the > > > > > > impact > > > > > > > > on > > > > > > > > > >>>>>>>>>>>> infrastructure costs. I don't think there would > > be > > > > any > > > > > > > > > >>>> significant > > > > > > > > > >>>>>>>>>>> impact. > > > > > > > > > >>>>>>>>>>>> We could mail infrastructure to find out if they > > > > have > > > > > > any > > > > > > > > > >>>> concerns > > > > > > > > > >>>>>>>>>> given > > > > > > > > > >>>>>>>>>>> the > > > > > > > > > >>>>>>>>>>>> space requirement if you prefer this option. > > > > > > > > > >>>>>>>>>>> > > > > > > > > > >>>>>>>>>>> We have discussed it with INFRA in the early > > days of > > > > the > > > > > > > > project. > > > > > > > > > >>>>> And > > > > > > > > > >>>>>>>>>>> precisely the cost impact was the reason we have > > > > kept it > > > > > > > > > >>>>> elsewhere. We > > > > > > > > > >>>>>>>>>> are > > > > > > > > > >>>>>>>>>>> talking about ~0.7GB/repo x 5 platforms (at > > least) x > > > > > > number > > > > > > > > of > > > > > > > > > >>>>> Apache > > > > > > > > > >>>>>>>>>>> mirrors > > > > > > > > > >>>>>>>>>>> - it's pretty huge, really. > > > > > > > > > >>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>> Bintray is another option. I don't know anything > > > > about > > > > > > it. > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>> I also looked at creating a S3 bucket for Bigtop > > > > using > > > > > > my > > > > > > > > > >>>>> account. I > > > > > > > > > >>>>>>>>>>> have a > > > > > > > > > >>>>>>>>>>>> feeling monthly transfer charges will not be a > > > > problem. > > > > > > > > However > > > > > > > > > >>>>> the > > > > > > > > > >>>>>>>>>>> 'bigtop' > > > > > > > > > >>>>>>>>>>>> bucket is taken. Perhaps we could talk to Tom > > about > > > > > > getting > > > > > > > > > >>>>> ownership > > > > > > > > > >>>>>>>>>>>> transferred if you prefer this option. > > > > > > > > > >>>>>>>>>>> > > > > > > > > > >>>>>>>>>>> We need to ask Cloudera's infra team to transfer > > it > > > > to > > > > > > us. > > > > > > > > Sean, > > > > > > > > > >>>>> could > > > > > > > > > >>>>>>>>>> you > > > > > > > > > >>>>>>>>>>> figure out how this can be done? > > > > > > > > > >>>>>>>>>>> > > > > > > > > > >>>>>>>>>>> Thanks! > > > > > > > > > >>>>>>>>>>> Cos > > > > > > > > > >>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>> On Oct 18, 2015, at 4:42 AM, Olaf Flebbe < > > > > > > [email protected]> > > > > > > > > > >>>> wrote: > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>> Hi Cos, > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>> Is it correct that we should not put our > > release > > > > > > artifacts > > > > > > > > on > > > > > > > > > >>>>>>>>> apache > > > > > > > > > >>>>>>>>>>> dist, > > > > > > > > > >>>>>>>>>>>>> since Apache is about code not binaries? > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>> Can we use something like bintray.org ? Looks > > > > like the > > > > > > > > > >>>>>>>>> functionality > > > > > > > > > >>>>>>>>>>> we need: RPM and DEB repo, and does have an API. > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>> Olaf > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>>> Am 18.10.2015 um 09:30 schrieb Evans Ye < > > > > > > > > [email protected]>: > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>>> Guys I've a bad news. > > > > > > > > > >>>>>>>>>>>>>> I'm guessing that our official released 1.0 > > repos > > > > on > > > > > > S3 > > > > > > > > are > > > > > > > > > >>>> also > > > > > > > > > >>>>>>>>>> tied > > > > > > > > > >>>>>>>>>>> with > > > > > > > > > >>>>>>>>>>>>>> Cloudera's credentials, which is why we no > > longer > > > > > > have S3 > > > > > > > > > >>>> repos > > > > > > > > > >>>>>>>>>>> available > > > > > > > > > >>>>>>>>>>>>>> for users to consume now(I've tried centos6 > > and > > > > > > debian8). > > > > > > > > > >>>>>>>>>>>>>> That's really bad in user experience. :( > > > > > > > > > >>>>>>>>>>>>>> Cos, > > > > > > > > > >>>>>>>>>>>>>> * Could you please confirm where we put 1.0 > > repos > > > > on? > > > > > > > > > >>>>>>>>>>>>>> * If my guess is right, do you still have > > copies > > > > of > > > > > > signed > > > > > > > > > >>>> repos > > > > > > > > > >>>>>>>>> at > > > > > > > > > >>>>>>>>>>> local > > > > > > > > > >>>>>>>>>>>>>> for restore? > > > > > > > > > >>>>>>>>>>>>>> * I don't have knowledge of how our current S3 > > > > > > resources > > > > > > > > are > > > > > > > > > >>>>> being > > > > > > > > > >>>>>>>>>>> managed. > > > > > > > > > >>>>>>>>>>>>>> And we don't have S3 resources available from > > > > Tom's > > > > > > team, > > > > > > > > > >>>> hence > > > > > > > > > >>>>> I > > > > > > > > > >>>>>>>>>>> think we > > > > > > > > > >>>>>>>>>>>>>> need to plan for this now. > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > >>>>>>>>>>>>>> Evans > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> -- > > > > > > > > > >>>>>>>>> Best regards, > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> - Andy > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> Problems worthy of attack prove their worth by > > hitting > > > > > > back. - > > > > > > > > Piet > > > > > > > > > >>>>> Hein > > > > > > > > > >>>>>>>>> (via Tom White) > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best regards, > > > > > > > > > > > > > > - Andy > > > > > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. - > > Piet > > > > Hein > > > > > > > (via Tom White) > > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > > (via Tom White) > >
signature.asc
Description: Digital signature
