Amanda: it should be possible and desireable to make all these configuration changes in advance. They're totally valid HBase configuration settings that could very-well exist in advance.
The worse situation is if the Trafodion settings clash with some setting that the customer already has for some other reason. For example, can we really change the staging setting without affecting something else? This is why I am documenting these settings as requirements in the upcoming Provisioning Guide with the reasoning behind the setting. Further, I really recommend that we separate Trafodion installation from configuration changes in the infrastructure to ensure that it's possible to change the configuration changes, test their impact, and THEN install Trafodion. The testing could consist of something as simple as verify the restart worked or as advanced as an application regression test. Very user specific and dependent what else HDFS and HBase is being used for. For example, what's the impact on the rest of the Hadoop environment if turning on HDFS ACLs? Thanks, Gunnar Gunnar On Tue, Feb 9, 2016 at 9:45 AM, Amanda Moran <[email protected]> wrote: > I agree with that :) Thanks Qifan! > > On Tue, Feb 9, 2016 at 8:43 AM, Qifan Chen <[email protected]> wrote: > >> Hi Amanda, >> >> I know the topic that I brought up is a little bit different :-). >> >> But the objectives of both thoughts are pretty much in line with the >> improvement of install experiences. >> >> Thank you. >> >> --Qifan >> >> On Tue, Feb 9, 2016 at 10:36 AM, Amanda Moran <[email protected]> >> wrote: >> >>> Thanks for the response Qifan... but I am not sure we are talking about >>> the same thing. >>> >>> I am talking about the traf_<hadoop distro>_mods98 script that is run as >>> apart of the Trafodion Installation scripts. It is used to setup HBase >>> settings, copy hbase-trx*.jar file, and restart HBase, HDFS, and Zookeeper. >>> >>> I would like to know what are the smallest number of settings that need >>> to be set in advance for Trafodion to install/start. >>> >>> Examples of things set: >>> hbase-site hbase.master.distributed.log.splitting false >>> >>> hbase-site hbase.coprocessor.region.classes >>> "org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation" >>> >>> hbase-site hbase.hregion.impl >>> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion >>> >>> hbase-site hbase.regionserver.region.split.policy >>> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy >>> >>> hbase-site hbase.snapshot.enabled true >>> >>> hbase-site hbase.bulkload.staging.dir /hbase-staging >>> >>> hbase-site hbase.regionserver.region.transactional.tlog true >>> >>> hbase-site hbase.snapshot.master.timeoutMillis 600000 >>> >>> hbase-site hbase.snapshot.region.timeout 600000 >>> >>> hdfs-site dfs.namenode.acls.enabled true >>> >>> hbase-site hbase.client.scanner.timeout.period 600000 >>> >>> Thanks all! >>> >>> On Thu, Feb 4, 2016 at 5:40 AM, Qifan Chen <[email protected]> wrote: >>> >>>> Hi Amanda, >>>> >>>> I like your idea. >>>> >>>> On Trafodion's modification of config files for HBase, HDFS, and >>>> Zookeeper, I wonder if it is possible to re-difine the port range for an >>>> install, without going through all the install steps. This has the >>>> practical value that due to port conflict, an install could not start >>>> successfully, and one can easily switch to a different port range to retry >>>> with this capability. >>>> >>>> In addition, some of the install steps, such as TPCDS data set >>>> generation, and conversion to UTF8, could be made parallel to shorten the >>>> install duration. They are done one table at a time in serial mode now. >>>> >>>> TPCDS table data is generated through the command "dsdgen -scale ..." >>>> which could take the option "-parallel". >>>> Data conversion to UTF8 is done through the iconv utility. >>>> >>>> Regards, --Qifan >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Feb 3, 2016 at 10:34 PM, Amanda Moran <[email protected]> >>>> wrote: >>>> >>>>> HI there All- >>>>> >>>>> The traf_*_mods scripts are used during installation to set HBase, >>>>> HDFS, and Zookeeper settings needed by Trafodion. >>>>> >>>>> The script takes a long time to run (slowing down installation by a >>>>> lot) because it needs to restart all these services for the changes to be >>>>> picked up. >>>>> >>>>> My question is: What if I wanted to set the settings in advance of >>>>> running the installer. What would be the absolute minimum set of settings >>>>> that I need to get installed? I could set other settings later with a >>>>> script or by hand. >>>>> >>>>> Thanks! >>>>> -- >>>>> Thanks, >>>>> >>>>> Amanda Moran >>>>> >>>> >>>> >>>> >>>> -- >>>> Regards, --Qifan >>>> >>>> >>> >>> >>> -- >>> Thanks, >>> >>> Amanda Moran >>> >> >> >> >> -- >> Regards, --Qifan >> >> > > > -- > Thanks, > > Amanda Moran > -- Thanks, Gunnar *If you think you can you can, if you think you can't you're right.*
