Hi Parth, no problem, I did not start preparing the release since there is one blocker has left:
*DRILL-5669 - Boaz Ben-Zvi (fix will be ready in a couple of days)* Kind regards Arina On Mon, Jul 17, 2017 at 11:26 PM, Parth Chandra <[email protected]> wrote: > Hi Arina, > > I just realized that I did not push the C++ client fix ( DRILL-5659) to > master. I have just done so. > > Sorry if I messed things up for the release, but we should get this fix > in. > > Parth > > On Fri, Jul 14, 2017 at 8:15 AM, Paul Rogers <[email protected]> wrote: > > > I looked into one NPE on CSV. Turns out it is already fixed by previous > > 1.11 PRs, so we’re good. > > > > There is another reported by a user on the user list, but I’ve not heard > > back about getting sample data. Let’s not hold the release for that. > > > > - Paul > > > > > On Jul 14, 2017, at 2:38 AM, Arina Yelchiyeva < > > [email protected]> wrote: > > > > > > *Blockers* > > > DRILL-5659 - Parth (done) > > > DRILL-5669 - Boaz Ben-Zvi (fix will be ready in a couple of days) > > > > > > *Possible blockers* > > > NPE on CSV - Paul can you please provide an update? > > > > > > Kind regards > > > Arina > > > > > > On Thu, Jul 13, 2017 at 2:46 PM, Arina Yelchiyeva < > > > [email protected]> wrote: > > > > > >> *Blockers* > > >> DRILL-5659 - problem is identified, Parth is working on the fix. > > >> > > >> *Not blockers* > > >> DRILL-5660 - Vitalii > > >> DRILL-5468 - Jinfeng > > >> > > >> *Possible blockers* > > >> DRILL-5669 - Boaz Ben-Zvi / Paul > > >> NPE on CSV - Paul > > >> > > >> Kind regards > > >> Arina > > >> > > >> > > >> On Thu, Jul 13, 2017 at 1:28 AM, Paul Rogers <[email protected]> > wrote: > > >> > > >>> I withdraw my suggestion to fix DRILL-5660. Was a big hassle for us > to > > >>> track down the incompatibility the issue internally, but production > > users > > >>> probably don’t attempt to use one build of Drill with metadata files > > >>> created from another. We can worry about the issue later only if a > user > > >>> actually runs into it. > > >>> > > >>> As suggested earlier, let’s just document this obscure case in a > > release > > >>> note. > > >>> > > >>> - Paul > > >>> > > >>>> On Jul 12, 2017, at 3:04 PM, Jinfeng Ni <[email protected]> wrote: > > >>>> > > >>>> Upgrade from 1.10 to 1.11 is irreversible only when parquet metadata > > >>> cache > > >>>> file is involved. > > >>>> > > >>>> I might be wrong, but I thought that if someone is running on > 1.10.0, > > >>> and > > >>>> he wants to go back to 1.0.0, where the first version of parquet > > >>> meatadata > > >>>> cache was introduced, can we guarantee that Drillbit would work > > >>> correctly? > > >>>> I'm trying to understand what DRILL-5660 asks for is a new > > requirement, > > >>> or > > >>>> an requirement being there starting from Drill 1.0.0? > > >>>> > > >>>> Regression failures are normally regarded as release blocker. > > >>>> > > >>>> > > >>>> > > >>>> On Wed, Jul 12, 2017 at 2:54 PM, Paul Rogers <[email protected]> > > wrote: > > >>>> > > >>>>> If we don’t do DRILL-5660 for 1.11 then we don’t need to do it at > > all, > > >>> so > > >>>>> that is a time savings. We could handle this via a release note > item > > >>> that > > >>>>> simply states that an upgrade from 1.10 to 1.11 is irreversible: we > > do > > >>> not > > >>>>> support the ability to fall back to the old version. Everyone OK > with > > >>> that? > > >>>>> > > >>>>> We are seeing some stress test regression failures that, I’m told, > > will > > >>>>> show up as JIRA entries shortly. > > >>>>> > > >>>>> - Paul > > >>>>> > > >>>>>> On Jul 12, 2017, at 11:05 AM, Jinfeng Ni <[email protected]> wrote: > > >>>>>> > > >>>>>> I put some analysis in DRILL-5468. > > >>>>>> > > >>>>>> Also, IMHO, DRILL-5660 is not a release blocker. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> On Tue, Jul 11, 2017 at 11:27 AM, Parth Chandra < > [email protected]> > > >>>>> wrote: > > >>>>>> > > >>>>>>> Re DRILL-5634: > > >>>>>>> From the Apache commons web page [1] - > > >>>>>>> > > >>>>>>> Please note that Apache Commons Crypto doesn't implement the > > >>>>> cryptographic > > >>>>>>> algorithm such as AES directly.* It wraps to Openssl or JCE which > > >>>>> implement > > >>>>>>> the algorithms.* > > >>>>>>> > > >>>>>>> > > >>>>>>> Since we are cleared for using Openssl and JCE, we are OK to > > include > > >>> the > > >>>>>>> AES code in the UDFs. > > >>>>>>> > > >>>>>>> > > >>>>>>> Parth > > >>>>>>> > > >>>>>>> [1] https://commons.apache.org/proper/commons-crypto/ > > >>>>>>> > > >>>>>>> On Tue, Jul 11, 2017 at 10:56 AM, Arina Yelchiyeva < > > >>>>>>> [email protected]> wrote: > > >>>>>>> > > >>>>>>>> During today hangouts we agreed on approximate cut-off date - 14 > > >>> July, > > >>>>>>> 2017 > > >>>>>>>> (date may be shifted due to the blockers). > > >>>>>>>> I'll start preparing test RC on Thursday to solve any release > > >>>>> preparation > > >>>>>>>> issues beforehand. > > >>>>>>>> > > >>>>>>>> *Blockers* > > >>>>>>>> DRILL-5660 - Vitalii > > >>>>>>>> > > >>>>>>>> *Possible blockers* > > >>>>>>>> DRILL-5468 - Jinfeng > > >>>>>>>> NPE on CSV source - Paul > > >>>>>>>> DRILL-5659 - Parth > > >>>>>>>> > > >>>>>>>> *Will be included if done before the cut-off date* > > >>>>>>>> DRILL-5616 / DRILL-5665 - Boaz > > >>>>>>>> DRILL-5634 - Parth will take a look at Apache commons > > implementation > > >>>>> and > > >>>>>>>> update. > > >>>>>>>> > > >>>>>>>> Kind regards > > >>>>>>>> Arina > > >>>>>>>> > > >>>>>>>> On Tue, Jul 11, 2017 at 3:14 PM, Charles Givre < > [email protected]> > > >>>>> wrote: > > >>>>>>>> > > >>>>>>>>> Hi Arina, > > >>>>>>>>> I can’t make the hangout, but we have a few options. > Basically, > > >>> since > > >>>>>>>> all > > >>>>>>>>> the hashing algorithms have no export restrictions, we could: > > >>>>>>>>> 1. Remove the AES encrypt/decrypt until I can figure out what > we > > >>> have > > >>>>>>> to > > >>>>>>>>> do (and do it) > > >>>>>>>>> 2. I can try to research it today or tomorrow to figure out > > >>> exactly > > >>>>>>> what > > >>>>>>>>> we have to do to include AES. > > >>>>>>>>> 3. Remove the entire package > > >>>>>>>>> > > >>>>>>>>> I’d like to see the hashing functions at least included because > > >>> we’ve > > >>>>>>> had > > >>>>>>>>> a lot of requests for that functionality. I also don’t want > > this > > >>> to > > >>>>>>>> hold > > >>>>>>>>> things up, so please let me know your preference. Regardless, > > I’m > > >>>>>>> going > > >>>>>>>> to > > >>>>>>>>> start trying to figure out the restrictions so we’ll know for > the > > >>>>>>> future. > > >>>>>>>>> > > >>>>>>>>> Best, > > >>>>>>>>> — C > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>>> On Jul 11, 2017, at 05:58, Arina Yelchiyeva < > > >>>>>>>> [email protected]> > > >>>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>> @Jinfeng, > > >>>>>>>>>> Let's discuss today on Drill hangout final cut-off date. > > >>>>>>>>>> > > >>>>>>>>>> @Charles > > >>>>>>>>>> Since there were questions about AES in DRILL-5634, should we > > >>> exclude > > >>>>>>>> it > > >>>>>>>>>> from the candidates for 1.11.0 release? > > >>>>>>>>>> > > >>>>>>>>>> @Boaz > > >>>>>>>>>> Do we have any ETA when DRILL-5616 and DRILL-5665 will pass > code > > >>>>>>>> review? > > >>>>>>>>>> > > >>>>>>>>>> Also I believe we should include DRILL-5660 where we need to > > bump > > >>> up > > >>>>>>>>>> parquet metadata version. > > >>>>>>>>>> @Vitalii > > >>>>>>>>>> Do you have any ETA for the fix? > > >>>>>>>>>> > > >>>>>>>>>> Kind regards > > >>>>>>>>>> Arina > > >>>>>>>>>> > > >>>>>>>>>> On Tue, Jul 11, 2017 at 6:02 AM, Boaz Ben-Zvi < > > [email protected]> > > >>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> Hi Arina, > > >>>>>>>>>>> > > >>>>>>>>>>> Two PR have just been submitted, relating to the Hash Aggr > > >>>>>>> Spill: > > >>>>>>>>>>> DRILL-5616 and DRILL-5665 . > > >>>>>>>>>>> Can these be considered for 1.11 as well ? > > >>>>>>>>>>> > > >>>>>>>>>>> Thanks, > > >>>>>>>>>>> > > >>>>>>>>>>> Boaz > > >>>>>>>>>>> > > >>>>>>>>>>> On 7/10/17, 1:49 PM, "Paul Rogers" <[email protected]> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> DRILL-5601 is no longer a blocker for 1.11; we’ll defer that > > >>>>>>> until > > >>>>>>>>>>> after the release to allow others more time to work on other > > >>>>>>> features > > >>>>>>>>>>> rather than doing code reviews. > > >>>>>>>>>>> > > >>>>>>>>>>> - Paul > > >>>>>>>>>>> > > >>>>>>>>>>>> On Jul 10, 2017, at 1:43 PM, Jinfeng Ni <[email protected]> > > wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>> Hi Arina, > > >>>>>>>>>>>> > > >>>>>>>>>>>> It has been a while since we discussed 1.11.0 release. Any > > >>> update > > >>>>>>>>>>> on the > > >>>>>>>>>>>> new release plan? Do we have a cut-off date ? > > >>>>>>>>>>>> > > >>>>>>>>>>>> In the past, I believe Drill release is a time-bounded, not > > >>> feature > > >>>>>>>>>>>> bounded. If certain fixes/features could not make it within > a > > >>>>>>> certain > > >>>>>>>>>>>> time-frame, we had better defer them into future release, > > unless > > >>>>>>>>>>> those > > >>>>>>>>>>>> fixes/features are regarded as blocking issues for a new > > >>> release. > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Tue, Jul 4, 2017 at 5:14 AM, Arina Yelchiyeva < > > >>>>>>>>>>> [email protected] > > >>>>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Latest update: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> We wait for the following Jiras before final cut off. > > >>>>>>>>>>>>> 1. DRILL-5601 - Paul > > >>>>>>>>>>>>> 2. DRILL-5420 - Kunal > > >>>>>>>>>>>>> 3. DRILL-5634 - Charles > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Kind regards > > >>>>>>>>>>>>> Arina > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On Fri, Jun 30, 2017 at 9:29 PM, Kunal Khatua < > > >>> [email protected]> > > >>>>>>>>>>> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> Parth's already +1'd the process with a minor update. So, > > >>> we're > > >>>>>>>>>>> good to > > >>>>>>>>>>>>> go > > >>>>>>>>>>>>>> on DRILL-5420. I'll change the label on that to > > >>> ready-to-commit. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> <http://www.mapr.com/> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> ________________________________ > > >>>>>>>>>>>>>> From: Parth Chandra <[email protected]> > > >>>>>>>>>>>>>> Sent: Thursday, June 29, 2017 9:44:39 AM > > >>>>>>>>>>>>>> To: [email protected] > > >>>>>>>>>>>>>> Subject: Re: [DISCUSS] Drill 1.11.0 release > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> I'm Ok with the pcap format plugin if Pauls comments are > > also > > >>>>>>>>>>> addressed. > > >>>>>>>>>>>>> If > > >>>>>>>>>>>>>> the changes are done before your cutoff deadline,we can > > merge > > >>> it > > >>>>>>>>>>> in. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Thu, Jun 29, 2017 at 12:26 AM, Arina Yelchiyeva < > > >>>>>>>>>>>>>> [email protected]> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Hi all, > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> here is the list of Jira we are waiting before making > final > > >>> cut > > >>>>>>>>>>> off: > > >>>>>>>>>>>>>>> 1. DRILL-5601 - Paul > > >>>>>>>>>>>>>>> 2. DRILL-3640 - Kunal > > >>>>>>>>>>>>>>> 3. DRILL-5420 - Kunal > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Please let me know if there are any others Jiras we > should > > >>> wait > > >>>>>>>>>>> for. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> @Charles, > > >>>>>>>>>>>>>>> you have mentioned that you want to include some UDFs in > > >>> Drill > > >>>>>>>>>>> 1.11, do > > >>>>>>>>>>>>>> you > > >>>>>>>>>>>>>>> have any ETA for the PR? > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> @Paul / Parth > > >>>>>>>>>>>>>>> I see there is active CR for pcap format PR, do we want > to > > >>>>>>>>>>> include it > > >>>>>>>>>>>>> in > > >>>>>>>>>>>>>>> this release as well? > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Kind regards > > >>>>>>>>>>>>>>> Arina > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> On Thu, Jun 29, 2017 at 10:05 AM, Arina Yelchiyeva < > > >>>>>>>>>>>>>>> [email protected]> wrote: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Parth, thanks a lot the instructions. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> On Wed, Jun 28, 2017 at 8:24 PM, Parth Chandra < > > >>>>>>>>>>> [email protected]> > > >>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> FWIW, I put release instructions culled from gists > > written > > >>> by > > >>>>>>>>>>>>> previous > > >>>>>>>>>>>>>>>>> release managers into an updated gist here [1] > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> HTH > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Parth > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> [1] https://github.com/parthchandra/drill/wiki/Drill- > > >>>>>>>>>>>>> Release-Process > > >>>>>>>>>>>>>>>>> <https://github.com/parthchandra/drill/wiki/Drill- > > >>>>>>>>>>> Release-Process> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> On Tue, Jun 27, 2017 at 5:24 PM, Paul Rogers < > > >>>>>>> [email protected]> > > >>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Hi Arina, > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> I have two projects in progress: one is a candidate > for > > >>> 1.11, > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>> other > > >>>>>>>>>>>>>>>>> is > > >>>>>>>>>>>>>>>>>> not. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> The “managed external sort” is the good candidate for > > >>> 1.11: > > >>>>>>>>>>> The PR > > >>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>> DRILL-5601 is being reviewed. Getting that in will > allow > > >>> us > > >>>>>>> to > > >>>>>>>>>>>>>> enable > > >>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>> “new” external sort by default. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> The other project is to address Drill’s memory > > >>> fragmentation > > >>>>>>>>>>> issue > > >>>>>>>>>>>>>>> with > > >>>>>>>>>>>>>>>>>> vectors over 16 MB in size. Two PRs have been open > for a > > >>>>>>> while. > > >>>>>>>>>>>>> They > > >>>>>>>>>>>>>>> are > > >>>>>>>>>>>>>>>>>> the first of several. Each takes a while to review. > So, > > >>> the > > >>>>>>>>>>>>> project > > >>>>>>>>>>>>>>> does > > >>>>>>>>>>>>>>>>>> not fit 1.11. The work can go into the next release > > >>> instead. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Thanks, > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> - Paul > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>> > > >>>>> > > >>> > > >>> > > >> > > > > >
