+1 on merging. Thanks Brahma for starting the thread.
On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <elgo...@gmail.com> wrote: > Thank you Brahma for pushing this. > > As you mentioned, we have already taken most of the changes into > production. > I want to highlight that the main contribution is the addition of security. > We have been able to test this at a smaller scale (~500 servers and 4 > subclusters) and the performance is great with our current ZooKeeper > deployment. > I would also like to highlight that all the changes are constrained to > hadoop-hdfs-rbf and there is no differences in commons or HDFS. > > +1 on merging > > Inigo > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aajis...@gmail.com> wrote: > > > Thanks Brahma for starting the discussion. > > I'm +1 for merging this. > > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster > > with 2 routers (not in production) and running several tests. > > > > Regards, > > Akira > > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <bra...@apache.org> > > wrote: > > > > > > Dear Hadoop Developers > > > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk. We > have > > > been working on this feature from last several months. > > > This feature work received the contributions from different companies. > > All > > > of the feature development happened smoothly and collaboratively in > > JIRAs. > > > > > > Kindly do take a look at the branch and raise issues/concerns that need > > to > > > be addressed before the merge. > > > > > > *Highlights of HDFS-13891 Branch:* > > > ============================= > > > > > > Adding Security to RBF(1) > > > Adding Missing Client API's(2) > > > Improvements/Bug Fixing > > > Critical - HDFS-13637, HDFS-13834 > > > > > > *Commits:* > > > ======== > > > > > > No of JIRAs Resolved: 72 > > > > > > All this commits are in RBF Module. No changes in hdfs/common. > > > > > > *Tested Cluster:* > > > ============= > > > > > > Most of these changes verified at Uber,Microsoft,Huawei and some other > > > compaines. > > > > > > *Uber*: Most changes are running in production @Uber including the > > critical > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS Routers. > > > Zookeeper as a state store to hold delegation tokens were also stress > > > tested to hold more than 2 Million tokens. --CR Hota > > > > > > *MicroSoft*: Most of these changes are currently running in production > at > > > Microsoft.The security has also been tested in a 500 server cluster > with > > 4 > > > subclsuters. --Inigo Goiri > > > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3 > > > routers.Planning deploy 10K production cluster. > > > > > > *Contributors:* > > > =========== > > > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu Asanuma,Shubham > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He > Xiaoqiao,Sherwood > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving discussions > > and > > > contributing to this. > > > > > > *Future Tasks:* > > > ============ > > > > > > will cleanup the jira's under this umbrella and contiue to work. > > > > > > Reference: > > > 1) https://issues.apache.org/jira/browse/HDFS-13532 > > > 2) https://issues.apache.org/jira/browse/HDFS-13655 > > > > > > > > > > > > > > > --Brahma Reddy Battula > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > >
