[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086433#comment-15086433 ]
liu ming commented on TRAFODION-1729: ------------------------------------- Hi, Dave, As per my understanding, Trafodion cannot support ACID on native HBase tables access. If this is a requirement, yes, this is a problem. However, I feel it is still possible, we can alter the HBase table to add coprocessor when Trafodion need to transactionally access it. I will do an prototype. But I think we first need to confirm: Is now Trafodion support ACID on native HBase tables? thanks, Ming > change the coprocessor deployment method > ---------------------------------------- > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm > Reporter: liu ming > Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its installation will change > nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing > HBase system. Trafodion is considered to be heavy and intrusive in this > manner, and I feel maybe we can change this. > > Should I start this discussion in the dev mail list? > > P.S. a list of changed config items. My proposal will remove the last one, > hope we can get rid of all of them: > hbase.master.distributed.log.splitting false > hbase.snapshot.master.timeoutMillis 600000 > hbase_regionserver_lease_period 600000 > hbase.hregion.impl > org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion > hbase.regionserver.region.split.policy > org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy > hbase.snapshot.enabled true > hbase.bulkload.staging.dir /hbase-staging > hbase.regionserver.region.transactional.tlog true > hbase.snapshot.region.timeout 600000 > hbase_coprocessor_region_classes > org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation > -- This message was sent by Atlassian JIRA (v6.3.4#6332)