Hi All, Please do let us know your feedback as to how we should approach this.
Cheers, Prabath On Wed, Oct 15, 2014 at 5:28 PM, Deependra Ariyadewa <[email protected]> wrote: > > > On Wed, Oct 15, 2014 at 4:54 PM, Prabath Abeysekera <[email protected]> > wrote: > >> Hi Paul, >> >> We indeed tried using namenode federation as an alternative and IMO, it >> doesn't really fit in. One of the issues with that approach is, one >> namenode is only capable of handling just one namespace in it. Therefore, >> if we are to map tenants with those namespaces, then the deployment would >> really grow big particularly, when the number of tenants increase, as we >> might need to deploy at least a namenode each for all the tenants >> (excluding supportive nodes deployed to achieve HA for each namenode). >> > > MT with multiple name nodes more suitable for system that can > automatically instantiate name nodes. This model needs to get help form an > infrastructure like Stratos to instantiate name nodes. Therefore MT with > mutiple name nodes do not scale well in a public cloud environment. > > >> However, the federation approach would be ideal if we are to deal with a >> small number of tenants though. >> >> Deep and Shani (CC'ed) should be able to share more detailed notes on >> this as they've been working on this over the last couple of months. >> > > In the second approach we use one HDFS cluster with single name space but > update the INode file path information based on the tenant. In this > approach we do not have to update HDFS client tools. Also we can keep > single cluster and share HDFS resources across tenants. > > >> Cheers, >> Prabath >> >> On Wed, Oct 15, 2014 at 4:20 PM, Paul Fremantle <[email protected]> wrote: >> >>> Prabath >>> >>> Does HDFS support MT via multiple namenodes? >>> >>> Paul >>> >>> On 15 October 2014 09:49, Prabath Abeysekera <[email protected]> wrote: >>> >>>> Hi Folks, >>>> >>>> I've got a 3rd party dependency (i.e Hadoop/HDFS) to be adapted into >>>> GIT and wondering where exactly I should do the initial developments, etc >>>> with the newly introduced GIT model? I do realize that I could have simply >>>> used a private repository or something and do the initial forking and >>>> stuff, but just checking if there's any recommended space available within >>>> any WSO2-managed repository where I could simply get the stuff checked in >>>> and make them available for collaborative developments within the team. >>>> >>>> What's the recommended approach? >>>> >>>> Let me explain why we need to get this in created as a separate >>>> repository as well. We'd already done a lot of R&D around getting MT >>>> implemented for HDFS using the extension points exposed within Hadoop >>>> without any success. Tried getting ourselves into discussions with the >>>> Hadoop community also just to see if there's an alternative approach >>>> available to what we'd suggested for implementing the same but haven't >>>> received any positive response yet. Therefore, I'm afraid the only approach >>>> we're left with right now is to get the sources forked into our own code >>>> base and moving forward with the developments. I certainly do realize that >>>> this would incur maintenance issues and all while trying to upgrade the >>>> versions, etc. However, I do not see any other feasible alternative option >>>> that we can make use of, in order to continue developments on this front. >>>> >>>> Please do share your feedback as to how we could resolve the >>>> aforementioned as well. >>>> >>>> >>>> Cheers, >>>> Prabath >>>> >>>> P.S. Pardon if this is something that's already been discussed within >>>> the list, I'm just trying to get my head around this. :) >>>> >>>> -- >>>> Prabath Abeysekara >>>> Associate Technical Lead, Data TG. >>>> WSO2 Inc. >>>> Email: [email protected] >>>> Mobile: +94774171471 >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> Paul Fremantle >>> CTO and Co-Founder, WSO2 >>> OASIS WS-RX TC Co-chair, Apache Member >>> >>> UK: +44 207 096 0336 >>> >>> blog: http://pzf.fremantle.org >>> twitter.com/pzfreo >>> [email protected] >>> >>> wso2.com Lean Enterprise Middleware >>> >>> Disclaimer: This communication may contain privileged or other >>> confidential information and is intended exclusively for the addressee/s. >>> If you are not the intended recipient/s, or believe that you may have >>> received this communication in error, please reply to the sender indicating >>> that fact and delete the copy you received and in addition, you should not >>> print, copy, retransmit, disseminate, or otherwise use the information >>> contained in this communication. Internet communications cannot be >>> guaranteed to be timely, secure, error or virus-free. The sender does not >>> accept liability for any errors or omissions. >>> >> >> >> >> -- >> Prabath Abeysekara >> Associate Technical Lead, Data TG. >> WSO2 Inc. >> Email: [email protected] >> Mobile: +94774171471 >> > > > > -- > Deependra Ariyadewa > WSO2, Inc. http://wso2.com/ http://wso2.org > > email [email protected]; cell +94 71 403 5996 ; > Blog http://risenfall.wordpress.com/ > PGP info: KeyID: 'DC627E6F' > > *WSO2 - Lean . Enterprise . Middleware* > -- Prabath Abeysekara Associate Technical Lead, Data TG. WSO2 Inc. Email: [email protected] Mobile: +94774171471
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
