Hi, I posted a patch to hadoop-10895 couple of days back. Wonder if anyone could help doing a review?
Thanks a lot. --Yongjun On Sun, Oct 19, 2014 at 11:35 AM, Yongjun Zhang <yzh...@cloudera.com> wrote: > Thanks Arun for the info and reminder. > > Hi Alejandro, > > HADOOP-10895 is a blocker in the list Arun sent out and it's currently > owned by you. I wonder whether you will have time to work on. If not, would > you mind reassigning it to me and I will spend time on it? > > Thanks. > > --Yongjun > > > On Tue, Oct 14, 2014 at 11:25 AM, Arun C Murthy <a...@hortonworks.com> > wrote: > >> 2.6.0 is close now. >> >> Here are the remaining blockers, I'm hoping cut an RC in the next week or >> so: >> http://s.apache.org/hadoop-2.6.0-blockers >> >> thanks, >> Arun >> >> On Sep 30, 2014, at 10:42 AM, Arun C Murthy <a...@hortonworks.com> wrote: >> >> > Folks, >> > >> > I've created branch-2.6 to stabilize the release. >> > >> > Committers, please exercise caution henceforth on commits other than >> the ones we've discussed on this thread already. >> > >> > By default new features should now be targeted to the version "2.7" >> henceforth - I've ensure all the projects have that version on jira. >> > >> > thanks, >> > Arun >> > >> > On Sep 26, 2014, at 1:08 AM, Arun Murthy <a...@hortonworks.com> wrote: >> > >> >> Sounds good. I'll branch this weekend and we can merge the jiras we >> >> discussed in this thread as they they get wrapped next week. >> >> >> >> Thanks everyone. >> >> >> >> Arun >> >> >> >> >> >>> On Sep 24, 2014, at 7:39 PM, Vinod Kumar Vavilapalli < >> vino...@apache.org> wrote: >> >>> >> >>> We can branch off in a week or two so that work on branch-2 itself >> can go >> >>> ahead with other features that can't fit in 2.6. Independent of that, >> we >> >>> can then decide on the timeline of the release candidates once >> branch-2.6 >> >>> is close to being done w.r.t the planned features. >> >>> >> >>> Branching it off can let us focus on specific features that we want >> in for >> >>> 2.6 and then eventually blockers for the release, nothing else. There >> is a >> >>> trivial pain of committing to one more branch, but it's worth it in >> this >> >>> case IMO. >> >>> >> >>> A lot of efforts are happening in parallel from the YARN side from >> where I >> >>> see. 2.6 is a little bulky if only on the YARN side and I'm afraid if >> we >> >>> don't branch off and selectively try to get stuff in, it is likely to >> be in >> >>> a perpetual delay. >> >>> >> >>> My 2 cents. >> >>> >> >>> +Vinod >> >>> >> >>> On Wed, Sep 24, 2014 at 3:28 PM, Suresh Srinivas < >> sur...@hortonworks.com> >> >>> wrote: >> >>> >> >>>> Given some of the features are in final stages of stabilization, >> >>>> Arun, we should hold off creating 2.6 branch or building an RC by a >> week? >> >>>> All the features in flux are important ones and worth delaying the >> release >> >>>> by a week. >> >>>> >> >>>> On Wed, Sep 24, 2014 at 11:36 AM, Andrew Wang < >> andrew.w...@cloudera.com> >> >>>> wrote: >> >>>> >> >>>>> Hey Nicholas, >> >>>>> >> >>>>> My concern about Archival Storage isn't related to the code quality >> or >> >>>> the >> >>>>> size of the feature. I think that you and Jing did good work. My >> concern >> >>>> is >> >>>>> that once we ship, we're locked into that set of archival storage >> APIs, >> >>>> and >> >>>>> these APIs are not yet finalized. Simply being able to turn off the >> >>>> feature >> >>>>> does not change the compatibility story. >> >>>>> >> >>>>> I'm willing to devote time to help review these JIRAs and kick the >> tires >> >>>> on >> >>>>> the APIs, but my point above was that I'm not sure it'd all be done >> by >> >>>> the >> >>>>> end of the week. Testing might also reveal additional changes that >> need >> >>>> to >> >>>>> be made, which also might not happen by end-of-week. >> >>>>> >> >>>>> I guess the question before us is if we're comfortable putting >> something >> >>>> in >> >>>>> branch-2.6 and then potentially adding API changes after. I'm okay >> with >> >>>>> that as long as we're all aware that this might happen. >> >>>>> >> >>>>> Arun, as RM is this cool with you? Again, I like this feature and >> I'm >> >>>> fine >> >>>>> with it's inclusion, just a heads up that we might need some extra >> time >> >>>> to >> >>>>> finalize things before an RC can be cut. >> >>>>> >> >>>>> Thanks, >> >>>>> Andrew >> >>>>> >> >>>>> On Tue, Sep 23, 2014 at 7:30 PM, Tsz Wo (Nicholas), Sze < >> >>>>> s29752-hadoop...@yahoo.com.invalid> wrote: >> >>>>> >> >>>>>> Hi, >> >>>>>> >> >>>>>> I am worry about KMS and transparent encryption since there are >> quite >> >>>>> many >> >>>>>> bugs discovered after it got merged to branch-2. It gives us an >> >>>>> impression >> >>>>>> that the feature is not yet well tested. Indeed, transparent >> >>>> encryption >> >>>>> is >> >>>>>> a complicated feature which changes the core part of HDFS. It is >> not >> >>>>> easy >> >>>>>> to get everything right. >> >>>>>> >> >>>>>> >> >>>>>> For HDFS-6584: Archival Storage, it is a relatively simple and low >> risk >> >>>>>> feature. It introduces a new storage type ARCHIVE and the concept >> of >> >>>>> block >> >>>>>> storage policy to HDFS. When a cluster is configured with ARCHIVE >> >>>>> storage, >> >>>>>> the blocks will be stored using the appropriate storage types >> specified >> >>>>> by >> >>>>>> storage policies assigned to the files/directories. Cluster admin >> >>>> could >> >>>>>> disable the feature by simply not configuring any storage type and >> not >> >>>>>> setting any storage policy as before. As Suresh mentioned, >> HDFS-6584 >> >>>> is >> >>>>>> in the final stages to be merged to branch-2. >> >>>>>> >> >>>>>> Regards, >> >>>>>> Tsz-Wo >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Wednesday, September 24, 2014 7:00 AM, Suresh Srinivas < >> >>>>>> sur...@hortonworks.com> wrote: >> >>>>>> >> >>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> I actually would like to see both archival storage and single >> replica >> >>>>>>> memory writes to be in 2.6 release. Archival storage is in the >> final >> >>>>>> stages >> >>>>>>> of getting ready for branch-2 merge as Nicholas has already >> indicated >> >>>> on >> >>>>>>> the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both >> of >> >>>>> these >> >>>>>>> features are being in development for sometime. >> >>>>>>> >> >>>>>>> On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang < >> >>>> andrew.w...@cloudera.com> >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>>> Hey Arun, >> >>>>>>>> >> >>>>>>>> Maybe we could do a quick run through of the Roadmap wiki and >> >>>>>> add/retarget >> >>>>>>>> things accordingly? >> >>>>>>>> >> >>>>>>>> I think the KMS and transparent encryption are ready to go. We've >> >>>> got >> >>>>> a >> >>>>>>>> very few further bug fixes pending, but that's it. >> >>>>>>>> >> >>>>>>>> Two HDFS things that I think probably won't make the end of the >> week >> >>>>> are >> >>>>>>>> archival storage (HDFS-6584) and single replica memory writes >> >>>>>> (HDFS-6581), >> >>>>>>>> which I believe are under the HSM banner. HDFS-6484 was just >> merged >> >>>> to >> >>>>>>>> trunk and I think needs a little more work before it goes into >> >>>>> branch-2. >> >>>>>>>> HDFS-6581 hasn't even been merged to trunk yet, so seems a bit >> >>>> further >> >>>>>> off >> >>>>>>>> yet. >> >>>>>>>> >> >>>>>>>> Just my 2c as I did not work directly on these features. I just >> >>>>>> generally >> >>>>>>>> shy away from shipping bits quite this fresh. >> >>>>>>>> >> >>>>>>>> Thanks, >> >>>>>>>> Andrew >> >>>>>>>> >> >>>>>>>> On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy < >> a...@hortonworks.com> >> >>>>>> wrote: >> >>>>>>>> >> >>>>>>>>> Looks like most of the content is in and hadoop-2.6 is shaping >> up >> >>>>>> nicely. >> >>>>>>>>> >> >>>>>>>>> I'll create branch-2.6 by end of the week and we can go from >> there >> >>>>> to >> >>>>>>>>> stabilize it - hopefully in the next few weeks. >> >>>>>>>>> >> >>>>>>>>> Thoughts? >> >>>>>>>>> >> >>>>>>>>> thanks, >> >>>>>>>>> Arun >> >>>>>>>>> >> >>>>>>>>> On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy < >> >>>> a...@hortonworks.com >> >>>>>> >> >>>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> Folks, >> >>>>>>>>>> >> >>>>>>>>>> With hadoop-2.5 nearly done, it's time to start thinking ahead >> >>>> to >> >>>>>>>>>> hadoop-2.6. >> >>>>>>>>>> >> >>>>>>>>>> Currently, here is the Roadmap per the wiki: >> >>>>>>>>>> >> >>>>>>>>>> • HADOOP >> >>>>>>>>>> • Credential provider HADOOP-10607 >> >>>>>>>>>> • HDFS >> >>>>>>>>>> • Heterogeneous storage (Phase 2) - Support APIs >> >>>>> for >> >>>>>>>>> using >> >>>>>>>>>> storage tiers by the applications HDFS-5682 >> >>>>>>>>>> • Memory as storage tier HDFS-5851 >> >>>>>>>>>> • YARN >> >>>>>>>>>> • Dynamic Resource Configuration YARN-291 >> >>>>>>>>>> • NodeManager Restart YARN-1336 >> >>>>>>>>>> • ResourceManager HA Phase 2 YARN-556 >> >>>>>>>>>> • Support for admin-specified labels in YARN >> >>>>>> YARN-796 >> >>>>>>>>>> • Support for automatic, shared cache for YARN >> >>>>>>>>> application >> >>>>>>>>>> artifacts YARN-1492 >> >>>>>>>>>> • Support NodeGroup layer topology on YARN >> >>>> YARN-18 >> >>>>>>>>>> • Support for Docker containers in YARN >> >>>> YARN-1964 >> >>>>>>>>>> • YARN service registry YARN-913 >> >>>>>>>>>> >> >>>>>>>>>> My suspicion is, as is normal, some will make the cut and some >> >>>>>> won't. >> >>>>>>>>>> Please do add/subtract from the list as appropriate. Ideally, >> it >> >>>>>> would >> >>>>>>>> be >> >>>>>>>>>> good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep >> >>>> up a >> >>>>>>>>> cadence. >> >>>>>>>>>> >> >>>>>>>>>> More importantly, as we discussed previously, we'd like >> >>>>> hadoop-2.6 >> >>>>>> to >> >>>>>>>> be >> >>>>>>>>>> the *last* Apache Hadoop 2.x release which support JDK6. I'll >> >>>>> start >> >>>>>> a >> >>>>>>>>>> discussion with other communities (HBase, Pig, Hive, Oozie >> etc.) >> >>>>> and >> >>>>>>>> see >> >>>>>>>>>> how they feel about this. >> >>>>>>>>>> >> >>>>>>>>>> thanks, >> >>>>>>>>>> Arun >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> Arun C. Murthy >> >>>>>>>>> Hortonworks Inc. >> >>>>>>>>> http://hortonworks.com/ >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> CONFIDENTIALITY NOTICE >> >>>>>>>>> NOTICE: This message is intended for the use of the individual >> or >> >>>>>> entity >> >>>>>>>> to >> >>>>>>>>> which it is addressed and may contain information that is >> >>>>>> confidential, >> >>>>>>>>> privileged and exempt from disclosure under applicable law. If >> the >> >>>>>> reader >> >>>>>>>>> of this message is not the intended recipient, you are hereby >> >>>>> notified >> >>>>>>>> that >> >>>>>>>>> any printing, copying, dissemination, distribution, disclosure >> or >> >>>>>>>>> forwarding of this communication is strictly prohibited. If you >> >>>> have >> >>>>>>>>> received this communication in error, please contact the sender >> >>>>>>>> immediately >> >>>>>>>>> and delete it from your system. Thank You. >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> -- >> >>>>>>> http://hortonworks.com/download/ >> >>>>>>> >> >>>>>>> -- >> >>>>>>> CONFIDENTIALITY NOTICE >> >>>>>>> NOTICE: This message is intended for the use of the individual or >> >>>> entity >> >>>>>> to >> >>>>>>> which it is addressed and may contain information that is >> >>>> confidential, >> >>>>>>> privileged and exempt from disclosure under applicable law. If the >> >>>>> reader >> >>>>>>> of this message is not the intended recipient, you are hereby >> notified >> >>>>>> that >> >>>>>>> any printing, copying, dissemination, distribution, disclosure or >> >>>>>>> forwarding of this communication is strictly prohibited. If you >> have >> >>>>>>> received this communication in error, please contact the sender >> >>>>>> immediately >> >>>>>>> and delete it from your system. Thank You. >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> http://hortonworks.com/download/ >> >>>> >> >>>> -- >> >>>> CONFIDENTIALITY NOTICE >> >>>> NOTICE: This message is intended for the use of the individual or >> entity to >> >>>> which it is addressed and may contain information that is >> confidential, >> >>>> privileged and exempt from disclosure under applicable law. If the >> reader >> >>>> of this message is not the intended recipient, you are hereby >> notified that >> >>>> any printing, copying, dissemination, distribution, disclosure or >> >>>> forwarding of this communication is strictly prohibited. If you have >> >>>> received this communication in error, please contact the sender >> immediately >> >>>> and delete it from your system. Thank You. >> >>> >> >>> -- >> >>> CONFIDENTIALITY NOTICE >> >>> NOTICE: This message is intended for the use of the individual or >> entity to >> >>> which it is addressed and may contain information that is >> confidential, >> >>> privileged and exempt from disclosure under applicable law. If the >> reader >> >>> of this message is not the intended recipient, you are hereby >> notified that >> >>> any printing, copying, dissemination, distribution, disclosure or >> >>> forwarding of this communication is strictly prohibited. If you have >> >>> received this communication in error, please contact the sender >> immediately >> >>> and delete it from your system. Thank You. >> > >> > -- >> > Arun C. Murthy >> > Hortonworks Inc. >> > http://hortonworks.com/hdp/ >> > >> > >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/hdp/ >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified >> that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> immediately >> and delete it from your system. Thank You. >> > >
