Thanks for all the suggestions and comments. The resource file traf-site.xml being renamed as trafodion-site.xml is mainly for the client side properties. There is no need for hbase to be restarted for these properties to take effect, but trafodion need to be restarted.
As suggested by Hans/Steve, the resource file can contain the hbase property name prefixed with 'trafodion.'. Trafodion will strip 'trafodion.' and then add to the Configuration. For details on this change, please see https://github.com/apache/incubator-trafodion/pull/777 Selva -----Original Message----- From: Qifan Chen [mailto:[email protected]] Sent: Monday, October 24, 2016 11:14 AM To: [email protected] Subject: Re: [DISCUSS] Introducing Trafodion customization using its own configuration file. This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofing at http://aka.ms/LearnAboutSpoofing We may also clarify (e.g. with a comment in the config file) whether trafodion.hbase.client.scanner.timeout.period applies only to HBase tables used by Trafodion, or to all Hbase tables after the restart. --Qifan ________________________________ From: Dave Birdsall <[email protected]> Sent: Monday, October 24, 2016 12:43:47 PM To: [email protected] Subject: RE: [DISCUSS] Introducing Trafodion customization using its own configuration file. This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofing at http://aka.ms/LearnAboutSpoofing Hi, We should take a look at the Trafodion HBaseClient.java module to see what other properties have been overridden. hbase.client.scanner.timeout.period is one of them; thanks for getting that one. Another one is in method estimateRowCount: HConstants.BUCKET_CACHE_IOENGINE_KEY (You'll have to look up the property name in the HBase HConstants.java file). Dave -----Original Message----- From: Steve Varnau [mailto:[email protected]] Sent: Monday, October 24, 2016 9:59 AM To: [email protected] Subject: RE: [DISCUSS] Introducing Trafodion customization using its own configuration file. This should make install/configuration of trafodion much easier, since it does not need to modify properties in other components. It will certainly help with the cluster-manager integration I'm working on. I agree with Hans that adding trafodion or "traf" to the property names would help clarify. --Steve > -----Original Message----- > From: Hans Zeller [mailto:[email protected]] > Sent: Monday, October 24, 2016 9:06 AM > To: [email protected] > Subject: RE: [DISCUSS] Introducing Trafodion customization using its > own configuration file. > > This sender failed our fraud detection checks and may not be who they > appear to be. Learn about spoofing at http://aka.ms/LearnAboutSpoofing > > Hi, > > Sounds good to me, too. I wonder, however, whether we shouldn't > prepend "trafodion" to the properties in the file. So, instead of > hbase.client.scanner.timeout.period, for example, what about calling > it trafodion.hbase.client.scanner.timeout.period. This should avoid confusion. > > Thanks, > > Hans > > -----Original Message----- > From: Liu, Ming (Ming) [mailto:[email protected]] > Sent: Sunday, October 23, 2016 5:14 AM > To: [email protected] > Subject: RE: [DISCUSS] Introducing Trafodion customization using its > own configuration file. > > I like this proposal very much. +1 > > Ming > > -----Original Message----- > From: Selva Govindarajan [mailto:[email protected]] > Sent: Sunday, October 23, 2016 8:55 AM > To: [email protected] > Subject: RE: [DISCUSS] Introducing Trafodion customization using its > own configuration file. > > With this change, there is no need to restart hbase if 'initialize > trafodion' is done with this revision. You can also avoid restarting > hbase if the table coprocessors are added manually to all Trafodion > tables including the Trafodion meta data tables via hbase shell prior > to installation of Trafodion release containing this change. However, > I haven't tested it out to confirm it for sure. > > Yes. We still need to copy the hbase-trx jar to hbase lib folder so > that the region server processes can find the jar file containing the > table coprocessor classes. > > Currently, Trafodion modifies the ACL of some of the directories in > hdfs. I believe, for these ACL to take effect Hadoop needs to be > restarted and hence hbase also needs to be restarted. > > > Selva > > > -----Original Message----- > From: Zhang, Yi (Eason) [mailto:[email protected]] > Sent: Saturday, October 22, 2016 12:12 AM > To: [email protected] > Subject: Re: [DISCUSS] Introducing Trafodion customization using its > own configuration file. > > If using the new traf-site.ml config file: > > 1. Do we still need to restart HBase during installation? > 2. Do we still need to copy hbase-trx jar to HBase lib folder? > > > Thanks, > Eason > > > > On 22/10/2016, 10:16, "Selva Govindarajan" > <[email protected]> wrote: > > The properties added by the resource traf-site.xml will override > the previously added resources, unless it was marked as final. > > Selva > -----Original Message----- > From: Sean Broeder [mailto:[email protected]] > Sent: Friday, October 21, 2016 7:06 PM > To: [email protected] > Subject: RE: [DISCUSS] Introducing Trafodion customization using > its own configuration file. > > I'm intrigued by this idea. Certainly there have been instances > where the distros vary and that makes for difficult Trafodion installations. > > What would the order of precedence be? How would it work if the > standard hbase-site.xml file had a particular setting that Trafodion > wanted to override? Or would HBase override the Trafodion settings? > Would this create confusion if a user wanted to configure an HBase > setting that Trafodion sets as well? > > Thanks, > Sean > > -----Original Message----- > From: Selva Govindarajan [mailto:[email protected]] > Sent: Friday, October 21, 2016 6:15 PM > To: [email protected] > Subject: [DISCUSS] Introducing Trafodion customization using its > own configuration file. > > Currently Trafodion uses the standard client side Hbase > configuration file hbase-site.xml deployed by the distros. It is found > that there are variations in this configuration file between distros. > At times, the distro manager decides that a given property is not a > client property and it is not added to the deployed hbase client > configuration file. In addition, there are certain properties like > hbase.coprocessor.region.classes need to be configured for Trafodion > tables for the transaction management. Hence, I am planning to > introduce a configuration file traf-site.xml specific to Trafodion > similar to hbase configuration file hbase-site.xml. This configuration file > extends the properties inherited from the standard hbase-site.xml. > > By default, the traf-site.xml comes with the following properties > > <?xml version="1.0"?> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > <!-- Put Trafodion site-specific property overrides in this file. > --> > > <configuration> > <property> > <name>hbase.hregion.impl</name> > > <value>org.apache.hadoop.hbase.regionserver.transactional.Transactiona > lR > egion</value> > </property> > <property> > <name>hbase.coprocessor.region.classes</name> > <value> > > org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver, > > org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint, > org.apache.hadoop.hbase.coprocessor.AggregateImplementation > </value> > </property> > <property> > <name>hbase.client.scanner.timeout.period</name> > <value>3600000</value> > </property> > </configuration> > > This property file will be installed if it doesn't exist at > $MY_SQROOT/etc directory when sqgen is done. Any client side property > can be added to this file and the client connections from Trafodion > client processes will inherit them. > > The hbase.coprocessor.region.classes are added as Table > co-processor to the table descriptor when a Trafodion table is > created. Trafodion installer will not be adding to these properties to hbase > configuration file soon. > > Please feel free to provide your valuable suggestions/comments. > > Thanks > Selva > > > > >
