[
https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101314#comment-15101314
]
ASF GitHub Bot commented on TRAFODION-1729:
-------------------------------------------
GitHub user mashengchen reopened a pull request:
https://github.com/apache/incubator-trafodion/pull/258
TRAFODION-1729
change the coprocessor deployment method, to avoid intrusive into
underlying HBase system.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mashengchen/incubator-trafodion master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/258.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #258
----
commit 12d7342bb3d63991d31933139f5b3fee216ab8b7
Author: mashengchen <mashengc...@gmail.com>
Date: 2015-12-14T06:16:25Z
Merge pull request #1 from apache/master
update head
commit 3450fb15d4ae86c20c0a31f7e0750709f433af59
Author: mashengchen <mashengc...@gmail.com>
Date: 2015-12-21T05:26:00Z
Merge pull request #2 from apache/master
update head
commit c8fef619f858ff24b8076e4abc2f556d9fb42acb
Author: mashengchen <mashengc...@gmail.com>
Date: 2015-12-23T13:09:41Z
Merge pull request #3 from apache/master
update
commit 0042344cfb67900a1c815da932e5ea4c7ba93a19
Author: mashengchen <mashengc...@gmail.com>
Date: 2015-12-30T10:32:40Z
Merge pull request #4 from apache/master
update head
commit 11f9686b5df87f70b79edac5621da0610c7e70b8
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-11T02:49:44Z
jira1729
commit cac96bc7ea69a3f1293e9ddef5e752f8785fb886
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-11T03:19:05Z
Merge pull request #5 from apache/master
update head
commit 62237083f0db8230e257b6a8a12b5ca21c3831d8
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-11T04:03:24Z
Merge branch 'master' of github.com:apache/incubator-trafodion
commit 5292fdc470db2317f4ab1133d298e98b97992e1e
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-11T04:19:31Z
Merge branch 'master' of github.com:mashengchen/incubator-trafodion
commit 0ea8552120bd12d8c5ed1757325276cca3eac172
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-11T05:39:43Z
Update HBaseClient.java
remove unnecessary log.
commit 6e0193a1837148fe5fecd3653402ddaff61f54ce
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-11T05:41:00Z
Update CoprocessorUtils.java
change log level
commit 0dbaf3b3164069587a854968869211d1b69c3434
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-11T05:52:45Z
jira1729
commit d8b594f2b585de224a007e48ba735f0f5db860be
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-12T04:15:12Z
add copyright
commit e1af6b9b94b237ab98c8e40677ada757a9b76f18
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-12T05:38:26Z
rename & update code
commit 411b703f557bdcff63b971261715d9a2caecb989
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-12T05:50:13Z
Merge branch 'master' of github.com:apache/incubator-trafodion
commit c1b8448c90533a5a2b039cce116ee57563c77299
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-12T05:54:31Z
remove trafcoprocess.properties
commit b8e6e9c920039c2fd7c0a175214b93ed2335d3e9
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-14T07:54:08Z
Update traf_coprocessor.properties
use TrxRegionEndpoint as default
commit b01d8cb00abcfcadf48125649bdb69aac8616ffd
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-15T04:16:02Z
Update CoprocessorUtils.java
commit a005ebf5ccf75a374f034f8b63891a396c651ae9
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-15T06:18:14Z
Update HBaseClient.java
change CoprocessorUtils.addCoprocessor, add isMVCC as param
commit d11e9d4ecf4263b7999d4bf8e6460ba78da04150
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-15T06:22:24Z
Update CoprocessorUtils.java
change addCoprocessor add isMVCC as param.
commit 5751399d9334b77bbaca629b9c93d55355b49f00
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-15T06:27:01Z
Update traf_coprocessor.properties
make mvcc & sscc as a separate ones
commit 379f8aefaf12476b8bc0f7960636a33b1b341449
Author: mashengchen <mashengc...@gmail.com>
Date: 2016-01-15T06:28:45Z
Update CoprocessorUtils.java
----
> change the coprocessor deployment method
> ----------------------------------------
>
> Key: TRAFODION-1729
> URL: https://issues.apache.org/jira/browse/TRAFODION-1729
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: dtm
> Reporter: liu ming
> Assignee: mashengchen
>
> I have a proposal to change our current HBase coprocessor configuration
> method.
> There are three ways to add a coprocessor to a HBase table:
> 1. Via editing hbase-site.xml, which will load coprocessor for ALL
> tables (Trafodion is using this method now)
> 2. Via HBase shell command
> 3. Via HTableDescriptor.addCoprocessor() java API
> Trafodion now is using the first method. I proposed to use method 3, I
> finished a prototype and test seems works well.
>
> Here are the reasons I propose for this change:
> At present, the Trafodion installer needs to modify the hbase-site.xml and
> then restart HBase instance for the configuration to take effect. This step
> not only complicate the installer but also let user think Trafodion is
> intrusive into underlying HBase system. It will be ideal if we can avoid this
> step. Another problem: in CDH, there is a concept called ‘region server
> group’ or something, so the settings will have to carefully handled by
> installer to apply to all groups. As we saw recently in WebRoot deployment,
> Trafodion failed due to this reason. All these are very error prone and
> complicate the Trafodion installer. Once CDH or HDP changed something,
> Trafodion may fail again.
>
> So I spent time to investigate why we need to restart HBase in order to
> install Trafodion.
> As I understand, there are 3 major reasons
> 1. To add hbase-trx coprocessors
> 2. To overload HRegion with TransactionalRegion
> 3. Various configuration settings, need to check one by one.
> The first configuration can be avoided by applying my proposed change.
> The second one, I look through the TransactionalRegion.java, and find out the
> only reason (now) is to overload the getScanner() method to be public so can
> be invoked by the coprocessor. And there are only 1 or 2 places that API is
> invoked in Trafodion code. I checked with Kevin and he proposed by using
> ‘java reflection’ we can also avoid this.
> All other configuration items to some extent look like ‘best to have’, but
> not ‘must to have’. And I also find two config items seems never been used:
> hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm,
> but I search in all code, seems this is never used)
> hbase.regionserver.region.transactional.tlog true (Narendra can
> confirm, this is NEVER used, maybe a legacy config item?)
> Yes, by now, there are still some other config items seems cannot be avoided,
> but I hope we can find some way to remove them in the future. I am not trying
> to solve all issues right now, just want to start the effort to remove
> unnecessary hbase reconfiguration.
> For this example, Coprocessors can be added to a table at run time, no need
> to edit the hbase-site.xml and restart hbase. This is only the first step to
> try to remove the deep impact to the current HBase config and restart HBase.
>
> So I asked for your opinions about this change. If you think this is
> necessary, I will continue to file a JIRA and fix it.
>
> I strongly recommend to get rid of the step of ‘modify hbase-site.xml and
> restart your hbase’ for Trafodion installation, it should be an option , to
> tune the system to best suit Trafodion, but should not be a forced step. To
> be note: Apache Phoenix is also a SQL on HBase, its installation will change
> nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing
> HBase system. Trafodion is considered to be heavy and intrusive in this
> manner, and I feel maybe we can change this.
>
> Should I start this discussion in the dev mail list?
>
> P.S. a list of changed config items. My proposal will remove the last one,
> hope we can get rid of all of them:
> hbase.master.distributed.log.splitting false
> hbase.snapshot.master.timeoutMillis 600000
> hbase_regionserver_lease_period 600000
> hbase.hregion.impl
> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion
> hbase.regionserver.region.split.policy
> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy
> hbase.snapshot.enabled true
> hbase.bulkload.staging.dir /hbase-staging
> hbase.regionserver.region.transactional.tlog true
> hbase.snapshot.region.timeout 600000
> hbase_coprocessor_region_classes
> org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
