Also, I wanted to throw in one more bit for those of you that are interested in tinkering along :
http://people.apache.org/~khorgath/releases/0.13.1_RC0/relprep.pl http://people.apache.org/~khorgath/releases/0.13.1_RC0/requested_jiras This is the script and config file I'm using to generate this release. It's very much a hack right now, and I hope to improve it to streamline releases in the future, but how it can be used right now is this way: a) Put it in a hive git repo (and not have any changes that have not been committed - this script will checkout a new branch and commit things to that branch, so you want to make sure to have a clean repo) b) Put the file requested_jiras in that dir as well. c) Run the script from there. It will check the differences between the branch being released ("branch-0.13" is hardcoded currently as a global), and looks at all the commit logs in trunk that correspond to the jiras requested in the requested_jiras file, sorts them in the order they were committed, and then checks out a new branch called relprep-branch-0.13-<timestamp>, and attempts to cherry-pick those commits in. For some patches, this will not work, so there is an override mechanism provided by entries in the requested_jiras file, as can be observed in the file I mention above. At the end of it, you'll have your 0.13.1 repo reproduction to test against if you so desire. Known Bugs : a) I use system("....") or die "...";, which is faulty in that the "die" code will never be reached. I need to fix this, but all the system calls were working for me, and I'd much rather focus on the release now, and improve this script later. This is a TODO b) Some patches (those generated with --no-prefix) don't work with older versions of git. You'll need a 1.8.x git for them, or you have to generate git patches without --no-prefix. On Tue, May 6, 2014 at 6:21 PM, Sushanth Sowmyan <khorg...@gmail.com> wrote: > Hi Folks, > > After a run of the ptest framework across the 0.13.1 codebase, we have > a couple of test failures that I'm trying to track down and resolve. > > If any of you are interested in looking at it on your own in the > meanwhile, the conglomerate patch of all the patches I'm forward > porting into 0.13.1 is over at > http://people.apache.org/~khorgath/releases/0.13.1_RC0/0.13.1.gdiff.patch > > The current tests that are failing are as follows: > > org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullformatCTAS > org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_alter > org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_tblproperties > org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unset_table_view_property > org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_unset_table_property > org.apache.hive.service.cli.TestScratchDir.testLocalScratchDirs > > I'll update and follow up with patch devs as and when I find out the > source for these errors. > > Thanks, > -Sushanth > > On Mon, May 5, 2014 at 6:26 PM, Sushanth Sowmyan <khorg...@gmail.com> wrote: >> Hi Folks, >> >> It's past 6pm PDT on May 5th 2014, so I'm beginning the process to >> generate the 0.13.1 RC0. >> >> I've received backport patches for 0.13 for some of the jiras that did >> not cleanly apply back from 0.14 to 0.13, and I'm thankful for them. >> All requested patches are cleanly applying and compiling. Tests are >> still running, and I expect them to run for a while more still - I >> will make sure to wait for them to complete before actually cutting a >> release candidate. >> >> The very first thing I'm supposed to do per the HowtoRelease wiki, is >> to unlink any open non-blockers from the release, and then send out a >> mail to the dev list confirming that there are no open non-blockers >> for this release version. So, this is that mail: >> >> >> The jira query string I used to verify this is as follows: >> >> project = HIVE AND priority not in (Critical, Major, Minor, Trivial) >> AND fixVersion = 0.13.1 AND status = Open >> >> The good news is that the 0.13.1 branch name was only created a couple >> of hours back, so no issues have been created on this, so there were >> none to unlink. :) >> >> >> On Fri, May 2, 2014 at 6:22 PM, Sushanth Sowmyan <khorg...@gmail.com> wrote: >>> Hi Folks, >>> >>> It is now past 6pm PDT May 2nd 2014, so I have locked down the table >>> of jiras for inclusion into 0.13.1. The jiras included are as listed >>> in : >>> >>> https://cwiki.apache.org/confluence/display/Hive/Hive+0.13.1+Release+tracking >>> >>> The jiras being included are : >>> >>> HIVE-6919, HIVE-6921, HIVE-6957, HIVE-6945, HIVE-6862, >>> HIVE-6883, HIVE-6961, HIVE-4576, HIVE-6952, HIVE-6828 >>> HIVE-6898, HIVE-6783, HIVE-6955, HIVE-6877, HIVE-6985 >>> HIVE-7001 >>> >>> The inclusion requestors for the above are : >>> >>> thejas, ashutoshc, sershe, ekoifman, prasanth_j, hagleitn, szehon, >>> rhbutani and amareshwari. >>> >>> If you are in this list, I might be pinging you in the next couple of >>> days for help if the patch does not cleanly apply, or causes >>> additional failures, and I appreciate any help you can tend. >>> >>> There is one jira on that list, HIVE-6826, which is in the list, and >>> will still be included if it moves from Patch-available to Committed >>> in 0.14 by Monday when I roll out the RC, which I have assurances from >>> vikram that it will, given that it's waiting on a test result. >>> >>> If anyone has any further breaking jiras that they absolutely want in >>> this release, please do still bring it up, I've created a new table >>> for them in the release tracking wiki page. I do, however, raise the >>> bar for inclusion by asking two other committers to also +1 such an >>> inclusion request at this stage. >>> >>> Hopefully, we'll see an RC and a release pretty soon! :) >>> >>> Thanks all! >>> -Sushanth >>> >>> >>> On Thu, May 1, 2014 at 2:34 PM, Sushanth Sowmyan <khorg...@gmail.com> wrote: >>>> Hi All, >>>> >>>> A quick reminder - per the timeline discussed so far, the list of >>>> jiras for inclusion in 0.13.1 RC0 freezes tomorrow (May 2nd) evening >>>> at 6pm PDT. Please make sure you ask for inclusion or edit the wiki >>>> with any further requests so that we can get this release done in a >>>> timely manner. >>>> >>>> Thanks, >>>> -Sushanth >>>> >>>> >>>> On Mon, Apr 28, 2014 at 1:39 PM, Sushanth Sowmyan <khorg...@gmail.com> >>>> wrote: >>>>> Good question - I think that if the number of patches is small ( < ~20 >>>>> which I expect because this is a stabilization release soon after >>>>> 0.13.0), I'd prefer to commit them myself, so that I can retain order >>>>> of patch application and make sure they all apply cleanly. I'm also >>>>> actually using this as an opportunity towards testing out an >>>>> semi-automated script that reads the requested jiras to do this - I >>>>> figure it's useful to test something like that out on a smaller >>>>> release that'll make further releases easier. >>>>> >>>>> On Mon, Apr 28, 2014 at 11:42 AM, Thejas Nair <the...@hortonworks.com> >>>>> wrote: >>>>>> Sushanth, >>>>>> How do you want to go about committing the patches into 0.13 branch ? >>>>>> Will you be able to commit the requested patches into 0.13.1 yourself, or >>>>>> do you want requestors to discuss it with you and then commit the patches >>>>>> themselves (if the requestor is a committer). >>>>>> >>>>>> Thanks, >>>>>> Thejas >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Apr 26, 2014 at 2:19 PM, Sushanth Sowmyan >>>>>> <khorg...@gmail.com>wrote: >>>>>> >>>>>>> Added. >>>>>>> >>>>>>> If others have difficulty editing( I can't figure out how to change >>>>>>> editing >>>>>>> privileges, but it seems to indicate that others can edit) the page, >>>>>>> I'll >>>>>>> accept replies to this thread as well and can add it in. >>>>>>> On Apr 25, 2014 6:25 PM, "Sergey Shelukhin" <ser...@hortonworks.com> >>>>>>> wrote: >>>>>>> >>>>>>> > I don't have access to edit this page (or cannot figure out the UI). >>>>>>> > Username sershe. >>>>>>> > Can you add >>>>>>> > HIVE-6961 : Drop partitions treats partition columns as strings (area >>>>>>> > - >>>>>>> > metastore) >>>>>>> > >>>>>>> > >>>>>>> > On Fri, Apr 25, 2014 at 4:20 PM, Sushanth Sowmyan <khorg...@gmail.com >>>>>>> > >wrote: >>>>>>> > >>>>>>> > > I've created the following wiki link : >>>>>>> > > >>>>>>> > > >>>>>>> > >>>>>>> https://cwiki.apache.org/confluence/display/Hive/Hive+0.13.1+Release+tracking >>>>>>> > > >>>>>>> > > People should be able to request additional jiras by adding it to >>>>>>> > > the >>>>>>> > > list. I think it might make sense to halt addition of requests to >>>>>>> > > the >>>>>>> > > list 3 days before the RC is cut, so as to prevent an endless-tail >>>>>>> > > scenario, unless the bug in question is a breaking severe issue, >>>>>>> > > where, yes, after discussion, we can vote to add it to the list. >>>>>>> > > That >>>>>>> > > also gives us time to run a full suite of tests on a stable build >>>>>>> > > before we cut the RC. >>>>>>> > > >>>>>>> > > I propose that the first RC (RC0) be built on Monday May 5th at 6pm >>>>>>> > > PDT, and the jira list on the wiki be closed to open/easy additions >>>>>>> > > at >>>>>>> > > 6pm PDT on Friday May 2nd. >>>>>>> > > >>>>>>> > > >>>>>>> > > On Fri, Apr 25, 2014 at 2:40 PM, Gunther Hagleitner >>>>>>> > > <ghagleit...@hortonworks.com> wrote: >>>>>>> > > > Sorry - HIVE-6824 isn't needed. Just the other 3. My bad. >>>>>>> > > > >>>>>>> > > > Thanks, >>>>>>> > > > Gunther. >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > On Fri, Apr 25, 2014 at 2:10 PM, Gunther Hagleitner < >>>>>>> > > > ghagleit...@hortonworks.com> wrote: >>>>>>> > > > >>>>>>> > > >> I'd like to request to include these Tez fixes: >>>>>>> > > >> >>>>>>> > > >> HIVE-6824, HIVE-6826, HIVE-6828, HIVE-6898 >>>>>>> > > >> >>>>>>> > > >> Thanks, >>>>>>> > > >> Gunther. >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > > >> On Fri, Apr 25, 2014 at 11:59 AM, Sushanth Sowmyan < >>>>>>> > khorg...@gmail.com >>>>>>> > > >wrote: >>>>>>> > > >> >>>>>>> > > >>> True, I was counting two weeks from today, but 0.13 has already >>>>>>> been >>>>>>> > > >>> out for a week. I'm amenable to having an RC1 out on May 5th. If >>>>>>> any >>>>>>> > > >>> further issues appear that block, then we can deal with them in >>>>>>> > > >>> an >>>>>>> > > >>> RC2/etc modification to that. >>>>>>> > > >>> >>>>>>> > > >>> On Fri, Apr 25, 2014 at 11:45 AM, Thejas Nair < >>>>>>> > the...@hortonworks.com> >>>>>>> > > >>> wrote: >>>>>>> > > >>> > On Fri, Apr 25, 2014 at 11:33 AM, Sushanth Sowmyan < >>>>>>> > > khorg...@gmail.com >>>>>>> > > >>> >wrote: >>>>>>> > > >>> > >>>>>>> > > >>> >> >>>>>>> > > >>> >> I think it's important to get a bugfix/stabilization release >>>>>>> > > >>> >> reasonably quickly, but it's also important to give people a >>>>>>> > little >>>>>>> > > >>> >> time to try out 0.13, discover/report bugs and fix them. So I >>>>>>> > think >>>>>>> > > >>> >> about two weeks is a good point? And instead of releasing an >>>>>>> > > >>> >> RC >>>>>>> > on a >>>>>>> > > >>> >> friday, I'm thinking of pushing it out to Monday - does 12th >>>>>>> > > >>> >> May >>>>>>> > > sound >>>>>>> > > >>> >> good to everyone? >>>>>>> > > >>> >> >>>>>>> > > >>> > >>>>>>> > > >>> > I think we can aim for an earlier date. Most of these issues >>>>>>> > > >>> > seem >>>>>>> > to >>>>>>> > > be >>>>>>> > > >>> > already committed to trunk or have patches available. So the >>>>>>> > > remaining >>>>>>> > > >>> ones >>>>>>> > > >>> > also might get committed to trunk by early next week. How >>>>>>> > > >>> > about >>>>>>> > > shooting >>>>>>> > > >>> > for May 5th (Monday) ? >>>>>>> > > >>> > By then 0.13 would also have been out for 2 weeks. If we have >>>>>>> > > >>> > any >>>>>>> > new >>>>>>> > > >>> > critical bug reported that needs a fix, we can hold off on >>>>>>> > > >>> > the RC >>>>>>> > for >>>>>>> > > >>> few >>>>>>> > > >>> > days. >>>>>>> > > >>> > What do you think ? >>>>>>> > > >>> > >>>>>>> > > >>> > Thanks, >>>>>>> > > >>> > Thejas >>>>>>> > > >>> > >>>>>>> > > >>> > -- >>>>>>> > > >>> > CONFIDENTIALITY NOTICE >>>>>>> > > >>> > NOTICE: This message is intended for the use of the >>>>>>> > > >>> > individual or >>>>>>> > > >>> entity to >>>>>>> > > >>> > which it is addressed and may contain information that is >>>>>>> > > confidential, >>>>>>> > > >>> > privileged and exempt from disclosure under applicable law. If >>>>>>> the >>>>>>> > > >>> reader >>>>>>> > > >>> > of this message is not the intended recipient, you are hereby >>>>>>> > > notified >>>>>>> > > >>> that >>>>>>> > > >>> > any printing, copying, dissemination, distribution, >>>>>>> > > >>> > disclosure or >>>>>>> > > >>> > forwarding of this communication is strictly prohibited. If >>>>>>> > > >>> > you >>>>>>> > have >>>>>>> > > >>> > received this communication in error, please contact the >>>>>>> > > >>> > sender >>>>>>> > > >>> immediately >>>>>>> > > >>> > and delete it from your system. Thank You. >>>>>>> > > >>> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> > > > >>>>>>> > > > -- >>>>>>> > > > CONFIDENTIALITY NOTICE >>>>>>> > > > NOTICE: This message is intended for the use of the individual or >>>>>>> > entity >>>>>>> > > to >>>>>>> > > > which it is addressed and may contain information that is >>>>>>> confidential, >>>>>>> > > > privileged and exempt from disclosure under applicable law. If the >>>>>>> > reader >>>>>>> > > > of this message is not the intended recipient, you are hereby >>>>>>> notified >>>>>>> > > that >>>>>>> > > > any printing, copying, dissemination, distribution, disclosure or >>>>>>> > > > forwarding of this communication is strictly prohibited. If you >>>>>>> > > > have >>>>>>> > > > received this communication in error, please contact the sender >>>>>>> > > immediately >>>>>>> > > > and delete it from your system. Thank You. >>>>>>> > > >>>>>>> > >>>>>>> > -- >>>>>>> > CONFIDENTIALITY NOTICE >>>>>>> > NOTICE: This message is intended for the use of the individual or >>>>>>> > entity >>>>>>> to >>>>>>> > which it is addressed and may contain information that is >>>>>>> > confidential, >>>>>>> > privileged and exempt from disclosure under applicable law. If the >>>>>>> > reader >>>>>>> > of this message is not the intended recipient, you are hereby notified >>>>>>> that >>>>>>> > any printing, copying, dissemination, distribution, disclosure or >>>>>>> > forwarding of this communication is strictly prohibited. If you have >>>>>>> > received this communication in error, please contact the sender >>>>>>> immediately >>>>>>> > and delete it from your system. Thank You. >>>>>>> > >>>>>>> >>>>>> >>>>>> -- >>>>>> CONFIDENTIALITY NOTICE >>>>>> NOTICE: This message is intended for the use of the individual or entity >>>>>> to >>>>>> which it is addressed and may contain information that is confidential, >>>>>> privileged and exempt from disclosure under applicable law. If the reader >>>>>> of this message is not the intended recipient, you are hereby notified >>>>>> that >>>>>> any printing, copying, dissemination, distribution, disclosure or >>>>>> forwarding of this communication is strictly prohibited. If you have >>>>>> received this communication in error, please contact the sender >>>>>> immediately >>>>>> and delete it from your system. Thank You.
