Hi Anjana/Gokul/BAM team, I was actually wondering if there's a possibility we could move to 2.6.0 as well, so that there will be not be two different versions and two different bundles, in the platform. I guess, I misunderstood your requirement. The main reason why we need to fork the code is that we did not find any pluggable way to introduce a different authentication mechanism , other than kerberos to HDFS, It is tightly coupled. And for our multi tenancy implementations, we are thinking of introducing it, which would in turn introduce/modify code in hadoop hdfs code base. Hence we needed to fork it.
The HDFS client also seems to have code that makes it tightly coupled to Kerberos & Simple autentication and we might have to work on it as well. However, we would only introduce a new authentication but not change the existing. We do not have any plans in modifying the HDFS Client otherwise. Would it still be necessary for BAM to have a separate orbit bundle? Even with Cassandra we are using a single code base right? IMO we could do the same with HDFS also? On Mon, Jan 26, 2015 at 1:54 PM, Anjana Fernando <[email protected]> wrote: > Hi Shani, > > I guess the main question is, can we skip the Carbonized HDFS client, and > just create an Orbit bundle from the vanilla client, without any > modifications, for us to use. > > Cheers, > Anjana. > > On Mon, Jan 26, 2015 at 12:08 PM, Shani Ranasinghe <[email protected]> wrote: > >> Hi Prabath/Gokul, >> >> According to hadoop documentation there doesn't seem to be any major >> difference between the two releases [1] & [2]. However, 2.5.2 seems to >> have fixed some critical issues of 2.5.1 [3] (which are not directly >> mentioned), which is obviously carried out to 2.6.0. Since the datanode >> carbonization is done on 2.5.1 code, I am not sure how feasible it is to >> change it to 2.6.0, there seems to be improvements done on datanode code in >> this release. Since we have a release in 2 weeks time, and me and Deep are >> both working on tasks for this release, we could get someone from the team >> to check the feasibility of moving to 2.6.0. WDYT? >> >> [1] http://hadoop.apache.org/docs/r2.5.1/ >> [2] http://hadoop.apache.org/docs/stable/ >> [3] http://hadoop.apache.org/docs/r2.5.2/ >> >> On Mon, Jan 26, 2015 at 11:50 AM, Gokul Balakrishnan <[email protected]> >> wrote: >> >>> Hi SS team, >>> >>> For BAM 3.0 Data Layer implementation, we're considering the use of the >>> Hadoop 2.x Java API, to communicate with an external HDFS environment. >>> However, the version currently being used in SS 2.0 work seems to be Hadoop >>> 2.5.1 [1]. Since we won't be needing any multitenancy-related improvements >>> done as part of SS dev work, should we use your version or proceed with the >>> latest 2.6.0 jars for the Java API (as an orbit bundle), since it would be >>> best if we're not tied to SS releases for version upgrades merely for the >>> Java API. What do you suggest? >>> >>> [1] https://github.com/wso2-dev/wso2-hadoop >>> >>> Thanks, >>> >>> -- >>> *Balakrishnan Gokulakrishnan* >>> Software Engineer, >>> WSO2, Inc. http://wso2.com >>> Mob: +94 77 593 5789 | +1 650 272 9927 >>> >> >> >> >> -- >> Thanks and Regards >> *,Shani Ranasinghe* >> Software Engineer >> WSO2 Inc.; http://wso2.com >> lean.enterprise.middleware >> >> mobile: +94 77 2273555 >> linked in: lk.linkedin.com/pub/shani-ranasinghe/34/111/ab >> > > > > -- > *Anjana Fernando* > Senior Technical Lead > WSO2 Inc. | http://wso2.com > lean . enterprise . middleware > -- Thanks and Regards *,Shani Ranasinghe* Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: +94 77 2273555 linked in: lk.linkedin.com/pub/shani-ranasinghe/34/111/ab
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
