Thanks a lot for the help Ming (and thanks for the kind words Pierre!). I will take a look at testing without adding the coprocessor settings (but I won't remove from the installer for now).
Thanks! Amanda On Tue, May 3, 2016 at 12:42 AM, Liu, Ming (Ming) <[email protected]> wrote: > Yes, this is very important topic. > > > > I can confirm to remove two of them : > > Name: hbase.bulkload.staging.dir > > Value: /hbase-staging > > Why: > > Still Needed: NO > > > > Name: hbase.regionserver.region.transactional.tlog > > Value: true > > Why: > > Still Needed: NO > > > > We need more test for : > > > > Name: hbase_coprocessor_region_classes > > Value: > "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" > > Why: Not fully tested to remove it. > > Still needed: Yes > > > > There is already a fix that can get rid of this settings. But we need more > test. That is, remove this from installer and see if any issues. We tested > it briefly and it works well, but never get it full QAed. Trafodion now > should add coprocessor at runtime if installer not doing this. > > > > Another one I have comment: > > > > Name: hbase.hregion.impl > > Value: > org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion > > Why: Trafodion coprocessor need to invoke a private API of HBase Region, > so have to overwrite this class now. When Trafodion move to HBase 1.2, it > is possible to get rid of it, since HBase R1.2 public one required method > for us. See https://issues.apache.org/jira/browse/HBASE-15076?filter=-2 > > Still Needed: Yes. > > > > > > IMHO, all other settings should not be mandatory, but kind of > performance/stability tunings. However, I am not the author of those > settings, so not very sure. It is better others to comment more. > > > > Yes, thanks Amanda to bring this up, and I hope you can keep pushing this > topic. > > I feel some settings there is too old that nobody knows the reason or just > forget, I think testing is a solution: remove it, and test to see if any > issue. But if someone knows that one is MUST-TO-KEEP or DELETE-IT, it is > better to reply. > > > > Thanks, > > Ming > > > > *发件人:* Pierre Smits [mailto:[email protected]] > *发送时间:* 2016年5月3日 15:20 > *收件人:* [email protected] > *主题:* Re: HDFS/HBase/Zookeeper Settings set by Installer > > > > Hi Amanda, > > > > Thanks for bringing this up. Especially the 'why' aspect. This will be > good input for the documentation. > > > > Best regards, > > > Pierre Smits > > > *ORRTIZ.COM <http://www.orrtiz.com>* > > OFBiz based solutions & services > > > *OFBiz Extensions Marketplace* > > http://oem.ofbizci.net/oci-2/ > > > > On Mon, May 2, 2016 at 10:54 PM, Amanda Moran <[email protected]> > wrote: > > Hi there All- > > > > I have been looking over the HDFS/HBase/Zookeeper settings that get set in > the installer and I am wondering if they are all still needed. > > > > If you have requested a setting in the past, could you please add a > description of why it is needed (and if it is still needed)? > > > > Thanks a bunch! > > > > **Note: I know this would have looked much better in a spreadsheet.... but > I want to make sure everyone can see... and email is the best for that! > > > > *HDFS Settings* > > > > Name: namenode_java_heapsize > > Value: 1GB (or 1073741824 bytes) > > Why: > > Still needed: > > > > Name: secondary_namenode_java_heapsize > > Value: 1 GB (or 1073741824 bytes) > > Why: > > Still needed: > > > > Name: dfs_namenode_acls_enabled > > Value: true > > Why: > > Still needed: > > > > *HBase Master Settings* > > > > =HBase Master Config Safety Valve= > > > > Name: hbase_master_distributed_log_splitting > > Value: false > > Why: > > Still needed: > > > > Name: hbase_snapshot_master_timeoutMillis > > Value: 600000 > > Why: > > Still needed: > > > > *HBase Region Server Settings * > > > > Name: hbase_coprocessor_region_classes > > Value: > "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" > > Why: > > Still needed: > > > > Name: hbase_regionserver_lease_period > > Value: 600000 > > Why: > > Still Needed: > > > > =HBase RegionServer Config Safety Valve= > > > > Name: hbase.hregion.impl > > Value: > org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion > > Why: > > Still Needed: > > > > Name: hbase.regionserver.region.split.policy > > Value: org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy > > Why: > > Still Needed: > > > > Name: hbase.snapshot.enabled > > Value: true > > Why: > > Still Needed: > > > > Name: hbase.bulkload.staging.dir > > Value: /hbase-staging > > Why: > > Still Needed: > > > > Name: hbase.regionserver.region.transactional.tlog > > Value: true > > Why: > > Still Needed: > > > > Name: hbase.snapshot.region.timeout > > Value: 600000 > > Why: > > Still Needed: > > > > *Zookeeper Settings* > > > > Name: maxClientCnxns > > Value: 0 > > Why: > > Still Needed: > > > > -- > > Thanks, > > > > Amanda Moran > > > -- Thanks, Amanda Moran
