Hi Azeez,Paul,All Don't think we have come to a conclusion on this. Are we able to finalize our approach on this as we have come to a juncture where this is required to proceed further with next release.
Regards, Jaminda On Tue, Oct 21, 2014 at 10:02 PM, Prabath Abeysekera <[email protected]> wrote: > Hi All, > > Please do let us know your feedback as to how we should approach this. > > Cheers, > Prabath > > On Wed, Oct 15, 2014 at 5:28 PM, Deependra Ariyadewa <[email protected]> > wrote: > >> >> >> On Wed, Oct 15, 2014 at 4:54 PM, Prabath Abeysekera <[email protected]> >> wrote: >> >>> Hi Paul, >>> >>> We indeed tried using namenode federation as an alternative and IMO, it >>> doesn't really fit in. One of the issues with that approach is, one >>> namenode is only capable of handling just one namespace in it. Therefore, >>> if we are to map tenants with those namespaces, then the deployment would >>> really grow big particularly, when the number of tenants increase, as we >>> might need to deploy at least a namenode each for all the tenants >>> (excluding supportive nodes deployed to achieve HA for each namenode). >>> >> >> MT with multiple name nodes more suitable for system that can >> automatically instantiate name nodes. This model needs to get help form an >> infrastructure like Stratos to instantiate name nodes. Therefore MT with >> mutiple name nodes do not scale well in a public cloud environment. >> >> >>> However, the federation approach would be ideal if we are to deal with a >>> small number of tenants though. >>> >>> Deep and Shani (CC'ed) should be able to share more detailed notes on >>> this as they've been working on this over the last couple of months. >>> >> >> In the second approach we use one HDFS cluster with single name space but >> update the INode file path information based on the tenant. In this >> approach we do not have to update HDFS client tools. Also we can keep >> single cluster and share HDFS resources across tenants. >> >> >>> Cheers, >>> Prabath >>> >>> On Wed, Oct 15, 2014 at 4:20 PM, Paul Fremantle <[email protected]> wrote: >>> >>>> Prabath >>>> >>>> Does HDFS support MT via multiple namenodes? >>>> >>>> Paul >>>> >>>> On 15 October 2014 09:49, Prabath Abeysekera <[email protected]> wrote: >>>> >>>>> Hi Folks, >>>>> >>>>> I've got a 3rd party dependency (i.e Hadoop/HDFS) to be adapted into >>>>> GIT and wondering where exactly I should do the initial developments, etc >>>>> with the newly introduced GIT model? I do realize that I could have simply >>>>> used a private repository or something and do the initial forking and >>>>> stuff, but just checking if there's any recommended space available within >>>>> any WSO2-managed repository where I could simply get the stuff checked in >>>>> and make them available for collaborative developments within the team. >>>>> >>>>> What's the recommended approach? >>>>> >>>>> Let me explain why we need to get this in created as a separate >>>>> repository as well. We'd already done a lot of R&D around getting MT >>>>> implemented for HDFS using the extension points exposed within Hadoop >>>>> without any success. Tried getting ourselves into discussions with the >>>>> Hadoop community also just to see if there's an alternative approach >>>>> available to what we'd suggested for implementing the same but haven't >>>>> received any positive response yet. Therefore, I'm afraid the only >>>>> approach >>>>> we're left with right now is to get the sources forked into our own code >>>>> base and moving forward with the developments. I certainly do realize that >>>>> this would incur maintenance issues and all while trying to upgrade the >>>>> versions, etc. However, I do not see any other feasible alternative option >>>>> that we can make use of, in order to continue developments on this front. >>>>> >>>>> Please do share your feedback as to how we could resolve the >>>>> aforementioned as well. >>>>> >>>>> >>>>> Cheers, >>>>> Prabath >>>>> >>>>> P.S. Pardon if this is something that's already been discussed within >>>>> the list, I'm just trying to get my head around this. :) >>>>> >>>>> -- >>>>> Prabath Abeysekara >>>>> Associate Technical Lead, Data TG. >>>>> WSO2 Inc. >>>>> Email: [email protected] >>>>> Mobile: +94774171471 >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> [email protected] >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> Paul Fremantle >>>> CTO and Co-Founder, WSO2 >>>> OASIS WS-RX TC Co-chair, Apache Member >>>> >>>> UK: +44 207 096 0336 >>>> >>>> blog: http://pzf.fremantle.org >>>> twitter.com/pzfreo >>>> [email protected] >>>> >>>> wso2.com Lean Enterprise Middleware >>>> >>>> Disclaimer: This communication may contain privileged or other >>>> confidential information and is intended exclusively for the addressee/s. >>>> If you are not the intended recipient/s, or believe that you may have >>>> received this communication in error, please reply to the sender indicating >>>> that fact and delete the copy you received and in addition, you should not >>>> print, copy, retransmit, disseminate, or otherwise use the information >>>> contained in this communication. Internet communications cannot be >>>> guaranteed to be timely, secure, error or virus-free. The sender does not >>>> accept liability for any errors or omissions. >>>> >>> >>> >>> >>> -- >>> Prabath Abeysekara >>> Associate Technical Lead, Data TG. >>> WSO2 Inc. >>> Email: [email protected] >>> Mobile: +94774171471 >>> >> >> >> >> -- >> Deependra Ariyadewa >> WSO2, Inc. http://wso2.com/ http://wso2.org >> >> email [email protected]; cell +94 71 403 5996 ; >> Blog http://risenfall.wordpress.com/ >> PGP info: KeyID: 'DC627E6F' >> >> *WSO2 - Lean . Enterprise . Middleware* >> > > > > -- > Prabath Abeysekara > Associate Technical Lead, Data TG. > WSO2 Inc. > Email: [email protected] > Mobile: +94774171471 > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Jaminda Batuwangala, Director of Engineering; WSO2, Inc.; http://wso2.com/ email: [email protected] <[email protected]>; cell: +94 77 308 2174 Lean . Enterprise . Middleware
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
