Maybe another thing we can add to post 1.0 and ongoing CI evolution is documenting to some extent must have and nice to have tests for each component, eventually having CI green/red have meaning interms of a bundle/release of bigtop
-----Original Message----- From: Jay Vyas [mailto:[email protected]] Sent: Friday, July 10, 2015 2:30 PM To: [email protected] Subject: Re: Brand new Bigtop AWS account My thought is no. Hive tests were broken in 0.8 and we released hive. We continue evolving on head so it's ok. But in pretty lax about this stuff Sent from my iPhone > On Jul 9, 2015, at 10:49 AM, Evans Ye <[email protected]> wrote: > > Sorry I didn't make it clear at the first place. > What I really want to ask is whether we need to have all the smoke > tests/integration tests to be tested and passed before we roll out 1.0. > For example, there might be 2 or 3 failed test in hadoop integration test. > But failed task doesn't mean that the component really has problems. > May I know what did we do in 0.8 release? > > 2015-07-01 2:43 GMT+08:00 Konstantin Boudnik <[email protected]>: > >>> On Mon, Jun 29, 2015 at 09:46PM, Evans Ye wrote: >>> I think Nate answered most of the questions, so thanks a lot! >>> To me the most critical task is to have 1.0 released, while CI >>> overhaul >> is >>> part of the task. >>> From my point of view, there're two thinks holding us back from release. >>> >>> 1. The 1.0 branch is still at bleeding age to support all the >>> claimed >> OSs. >>> The primary issues are in debian(should be fixed now) and oepnsuse >> working >>> as bigtop build slaves. >> >> Sorry guys - I was away for a while and still am catching up here. >> Evans, are you saying that there're issues that need to be fixed in >> the trunk and >> branch-1.0 as well? Or these are just CI related and do no require >> additional commits into the branch? >> >> The release criteria for us is pretty much what we define it to be ;) >> If we see 'good-enough' quality of the produced >> packages/tests/deployment mechanism >> - we can start the vote for the release any moment. Mind you - the >> packages aren't required for the release, as any ASF release is source code only. >> We do >> binaries as a extra service to our users. >> >> Cos >> >>> 2. Migrating the CI master to EC2 under EMR donated account. Since I >> can't >>> get in the old CI master, there's no way for me to retrieve current >>> job configs and restore them back to new CI master. Roman probably >>> has >> backups >>> but he might be busying on other things. >>> >>> 3. I'll let cos add more since I'm not quite familiar with release >>> criteria, but I'd love to help. >>> >>> I guess we can't have 2 solved in short period, so I might start to >>> work >> on >>> setting up repositories on old CI master directly. >>> >>> Thanks, >>> Evans >>> >>> 2015-06-27 3:46 GMT+08:00 <[email protected]>: >>> >>>> Still work in progress, will let Evans add some more details as he >>>> has >> been >>>> the one kickin but on CI tasks. Besides some more setup and >>>> planned enhancements focus is getting 1.0 out the door, then >>>> turning towards >> more >>>> foundational CI work for 1.x and beyond. >>>> >>>> Think we will be well served having more official bigtop repos >> serving, I >>>> setup a prototype off 0.8, serving from S3.., few things for us to >> figure >>>> out build/process wise to properly hook the CI into S3/repo >>>> publishing >>>> >>>> Evans created this jira for general tracking: >>>> https://issues.apache.org/jira/browse/BIGTOP-1897 >>>> >>>> Not all of which are required for new CI to be ready, for example >>>> newly added one for optimizing build process to reuse artifacts can >>>> start >> working >>>> on in parallel to new CI and 1.0 release but get completed after >>>> both >> are >>>> taken care of >>>> >>>> -----Original Message----- >>>> From: Zeng, Tom [mailto:[email protected]] >>>> Sent: Thursday, June 25, 2015 3:36 PM >>>> To: Evans Ye; Konstantin Boudnik >>>> Cc: [email protected] >>>> Subject: Re: Brand new Bigtop AWS account >>>> >>>> Hi Evans and Cos, what's the status of this, do we have the build >>>> CI environment set up yet? >>>> >>>> Thanks, >>>> Tom >>>> >>>> From: Evans Ye <[email protected]<mailto:[email protected]>> >>>> Date: Sunday, April 19, 2015 at 11:12 AM >>>> To: "[email protected]<mailto:[email protected]>" >>>> <[email protected]<mailto:[email protected]>>, "Zeng, Tom" >>>> <[email protected]<mailto:[email protected]>> >>>> Subject: Re: Brand new Bigtop AWS account >>>> >>>> Hi guys, >>>> >>>> I've copied security group settings from the old account to the new >>>> one provided by EMR team (thanks Tom!). >>>> I've also checked our existing jenkins and there are some useful >>>> jobs already defined. It seems that I only need to have some >>>> slightly modification on them to get it start working. >>>> My plan is to create a m3.xlarge instance and directly plug it on >>>> our existing jenkins as a slave so that I can start to make some progress. >>>> I'll keep you posted. Suggestions are welcome as well. :) >>>> >>>> Thanks, >>>> Evans >>>> >>>> >>>> 2015-04-09 13:52 GMT+08:00 Konstantin Boudnik >>>> <[email protected]<mailto:[email protected]>>: >>>> Cycling back: today we got a credit coupon from Amazon EMR team so >>>> we >> have >>>> everything to start building new CI infra. Please contact me >>>> directly >> if >>>> you >>>> plan to help with setting it up, so I can create an account with >>>> proper permissions for you. >>>> >>>> Special thanks are going to Tom Zeng and his team for making it happen! >>>> Thank you very much guys! >>>> >>>> Cos >>>> >>>>> On Tue, Mar 24, 2015 at 06:06AM, Konstantin Boudnik wrote: >>>>> Guys, >>>>> >>>>> I want to start a separate thread to track the CI preparations for >> the >>>>> release next month (fingers crossed). Clearly, we can make a >>>>> release without CI, but it'd way easier to test and create binary >>>>> artifacts >> if >>>>> we have a working environment for official validation. Roman has >>>>> done a lot in this direction (many thanks!), but there are still a >>>>> few rough edges, which might be easy to finish of. >>>>> >>>>> I want to figure out a couple of things: >>>>> - what's the state of CI and how much still needs to be done (Rvs? >> Could >>>> you >>>>> share any first hand feedback?) >>>>> - who would be able to help with the CI completion? I can commit >> some of >>>> my >>>>> cycles, but it'd be great to have few more hands on that. >>>>> Clearly, >>>> some >>>>> Jenkins-foo and prior CI skills won't hurt ;) >>>>> >>>>> Please chime in if you can help. Thanks a lot! >>>>> Cos >>
