zen_discovery_ping_unicast_hosts, if used, also needs square brackets in the parameter string.
Both these should be fixed to provide the square brackets in the template text (as is done by most other params that need them), but I didn’t include that in METRON-634 because it would muck up current users who HAVE included the square brackets in the parameter string, so a db upgrade script is needed. This puts it in the purview of METRON-865 instead of 634. --Matt On 4/26/17, 12:19 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote: Would i have to change that on Centos 7? On April 26, 2017 at 15:12:38, Nick Allen (n...@nickallen.org) wrote: Check on the `network_hosts` setting. I just ran into the same issue. I had to add brackets around it otherwise the file was not valid YAML and it would crash trying to load the config file. network.host: ["_lo:ipv4_","_eth0:ipv4_"] On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > I think it is literally crashing trying to load the config file > > > > On April 26, 2017 at 15:00:25, Michael Miklavcic ( > michael.miklav...@gmail.com) wrote: > > A couple more things you should check, given this setup: > > - 1 master node (not as data node) > - 2 data nodes > > I'd use something like this: > > - gateway_recover_after_data_nodes=1 or 2 > - index_number_of_replicas=0 or 1 (2 data nodes, with 2 total copies of > the index - if you set this value to 2, it would need 1 more data node > to > attain 1 orig index + 2 replicas) > > > > On Wed, Apr 26, 2017 at 12:02 PM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > > > Actually never mind. I am not sure how to read the exception. Is it > > failing to read the configuration? > > > > > > On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwa...@gmail.com) > > wrote: > > > > So the elasticsearch service is disabled. That is the issue. When ES > > calls get status ( service elasticsearch status ) it fails. > > I don’t know systemd too well, but does that mean we didn’t do the equiv > > of chkconfig on? > > > > > > > > On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwa...@gmail.com) > > wrote: > > > > Getting the same error in ES as before from what I can see > > > > > > On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwa...@gmail.com) > > wrote: > > > > I still can’t keep ES running, it dies. > > Also, I’m getting exceptions from enrichment trying to restart when the > > topology wasn’t running. > > > > Is there some coordination in this work? Should I just log jiras? > > If someone is already sussing out ES, we can sync up > > > > > > > > On April 26, 2017 at 11:35:03, zeo...@gmail.com (zeo...@gmail.com) > wrote: > > > > I'm also interested to know why that's important at such a small scale. > > > > Jon > > > > On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ottobackwa...@gmail.com> > > wrote: > > > > > I am following > > > > > > https://community.hortonworks.com/articles/60805/deploying- > > a-fresh-metron-cluster-using-ambari-serv.html > > > I DO have master and data node together. > > > Why is that a problem? > > > > > > I will try again with master and data node separate. > > > > > > > > > On April 26, 2017 at 10:41:59, Michael Miklavcic ( > > > michael.miklav...@gmail.com) wrote: > > > > > > Hey Otto, > > > > > > How do you have the ES nodes configured? For a base install I would > > setup 1 > > > master (NOT as data node) and 2 data nodes (NOT on the same node as the > > > master). This is the install configuration I got working. You can also > > > modify some configuration properties around master node as data node, > > index > > > replicas, and gateway recovery to get it working differently, but this > is > > > what will work OOTB with the default config settings from the mpack. If > > > you've already setup a master node and a data node on the same host, > > we'll > > > need to re-install. > > > > > > Mike > > > > > > > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ottobackwa...@gmail.com> > > > wrote: > > > > > > > File > > > > "/usr/lib/python2.6/site-packages/resource_management/ > core/shell.py", > > > line > > > > 293, in _call > > > > > > > > raise ExecutionFailed(err_msg, code, out, err) > > > > > > > > ExecutionFailed: Execution of 'service elasticsearch status' returned > > 3. > > > ● > > > > elasticsearch.service - Elasticsearch > > > > > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; > > > > disabled; > > > > vendor preset: disabled) > > > > > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT > > > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago > > > > > > > > Docs: http://www.elastic.co > > > > > > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch > > > > -Des.pidfile=${PID_DIR}/elasticsearch.pid > -Des.default.path.home=${ES_ > > > > HOME} > > > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR} > > > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE) > > > > > > > > Process: 16819 > > > > ExecStartPre=/usr/share/elasticsearch/bin/ > > elasticsearch-systemd-pre-exec > > > > (code=exited, status=0/SUCCESS) > > > > > > > > Main PID: 16821 (code=exited, status=1/FAILURE) > > > > > > > > > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: > at > > > > org.elasticsearch.common.settings.Settings$Builder. > > > > loadFromStream(Settings.java:1080) > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: > at > > > > org.elasticsearch.common.settings.Settings$Builder. > > > > loadFromPath(Settings.java:1067) > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: > at > > > > org.elasticsearch.node.internal.InternalSettingsPreparer. > > > > prepareEnvironment(InternalSettingsPreparer.java:88) > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: > at > > > > org.elasticsearch.bootstrap.Bootstrap.initialSettings( > > Bootstrap.java:202) > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: > at > > > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241) > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: > at > > > > org.elasticsearch.bootstrap.Elasticsearch.main( > Elasticsearch.java:35) > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: > > Refer > > > > to > > > > the log for complete error details. > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: > > > > elasticsearch.service: main process exited, code=exited, > > status=1/FAILURE > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit > > > > elasticsearch.service entered failed state. > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT > > > > > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: > > > > elasticsearch.service failed. > > > > > > > > INFO 2017-04-26 08:55:19,962 > > > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr > > > > > > > > > > > > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwa...@gmail.com) > > > > wrote: > > > > > > > > Can you describe what ES issues you are working on? > > > > Fixing the repos got everything installed, but my ES components don’t > > > stay > > > > running. > > > > I need to harvest the errors. > > > > > > > > > > > > > > > > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwa...@gmail.com) > > > > wrote: > > > > > > > > Nm. sorry. I fixed it. > > > > > > > > > > > > > > > > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwa...@gmail.com) > > > > wrote: > > > > > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is > erring > > on > > > > the f://localrepo > > > > even though the folder exists, there is no repodata/repomd.xml. > > > > > > > > What is the command to create a local repo? > > > > > > > > > > > > > > > > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwa...@gmail.com) > > > > wrote: > > > > > > > > I was going by the HW community page. > > > > > > > > Ok, Let me try it > > > > > > > > > > > > > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65...@gmail.com) > > wrote: > > > > > > > > That would do it. It requires 2.4.2+. I would have sworn I put that > in > > > the > > > > README, but I must have only annotated the PR. :( > > > > > > > > I'll get that in asap. > > > > > > > > -D... > > > > > > > > > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler < > ottobackwa...@gmail.com> > > > > wrote: > > > > > > > > > 2.4.1 as a matter of fact > > > > > > > > > > > > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65...@gmail.com) > > > wrote: > > > > > > > > > > Any chance they're running Ambari < 2.4.2? > > > > > > > > > > -D... > > > > > > > > > > > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic < > > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > > > Hey Otto, I don't have wizard screenshots in front of me right > now > > to > > > > > say > > > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run > > > into > > > > > > anything like that, but next time I run through the install I can > > > take > > > > > > another look. > > > > > > > > > > > > https://github.com/apache/incubator-metron/blob/master/ > > > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/ > > > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml > > > > > > > > > > > > > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler < > > > ottobackwa...@gmail.com > > > > > > > > > > > wrote: > > > > > > > > > > > > > stderr: > > > > > > > > > > > > > > Traceback (most recent call last): > > > > > > > > > > > > > > File "/var/lib/ambari-agent/cache/ > common-services/ELASTICSEARCH/ > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module> > > > > > > > > > > > > > > Elasticsearch().execute() > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > libraries/script/script.py", > > > > > > > line 280, in execute > > > > > > > > > > > > > > method(env) > > > > > > > > > > > > > > File "/var/lib/ambari-agent/cache/ > common-services/ELASTICSEARCH/ > > > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install > > > > > > > > > > > > > > self.install_packages(env) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > libraries/script/script.py", > > > > > > > line 567, in install_packages > > > > > > > > > > > > > > retry_count=agent_stack_retry_count) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > core/base.py", > > > > > > > line 155, in __init__ > > > > > > > > > > > > > > self.env.run() > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > core/environment.py", > > > > > > > line 160, in run > > > > > > > > > > > > > > self.run_action(resource, action) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > core/environment.py", > > > > > > > line 124, in run_action > > > > > > > > > > > > > > provider_action() > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > > core/providers/package/__init__.py", line 54, in > action_install > > > > > > > > > > > > > > self.install_package(package_name, self.resource.use_repos, > > > > > > > self.resource.skip_repos) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > > core/providers/package/yumrpm.py", line 49, in install_package > > > > > > > > > > > > > > self.checked_call_with_retries(cmd, sudo=True, > > > > > > > logoutput=self.get_logoutput()) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > > core/providers/package/__init__.py", line 83, in > > > > > > checked_call_with_retries > > > > > > > > > > > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > > core/providers/package/__init__.py", line 91, in > > _call_with_retries > > > > > > > > > > > > > > code, out = func(cmd, **kwargs) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > core/shell.py", > > > > > > > line 71, in inner > > > > > > > > > > > > > > result = function(command, **kwargs) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > core/shell.py", > > > > > > > line 93, in checked_call > > > > > > > > > > > > > > tries=tries, try_sleep=try_sleep) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > core/shell.py", > > > > > > > line 141, in _call_wrapper > > > > > > > > > > > > > > result = _call(command, **kwargs_copy) > > > > > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > > core/shell.py", > > > > > > > line 294, in _call > > > > > > > > > > > > > > raise Fail(err_msg) > > > > > > > > > > > > > > resource_management.core.exceptions.Fail: Execution of > > > '/usr/bin/yum > > > > > -d > > > > > > 0 > > > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing > > to > > > do > > > > > > > > > > > > > > stdout: > > > > > > > > > > > > > > 2017-04-25 14:12:48,669 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using > > > hadoop > > > > > > conf > > > > > > > dir: /usr/hdp/current/hadoop-client/conf > > > > > > > > > > > > > > 2017-04-25 14:12:48,671 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > > Group['metron'] > > > > > > {} > > > > > > > > > > > > > > 2017-04-25 14:12:48,673 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > > > group > > > > > > > Group['metron'] > > > > > > > > > > > > > > 2017-04-25 14:12:48,740 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > Group['livy'] > > > > > {} > > > > > > > > > > > > > > 2017-04-25 14:12:48,741 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > > > group > > > > > > > Group['livy'] > > > > > > > > > > > > > > 2017-04-25 14:12:48,774 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > > > > Group['elasticsearch'] {} > > > > > > > > > > > > > > 2017-04-25 14:12:48,775 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > > > group > > > > > > > Group['elasticsearch'] > > > > > > > > > > > > > > 2017-04-25 14:12:48,812 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > > Group['spark'] > > > > > > {} > > > > > > > > > > > > > > 2017-04-25 14:12:48,812 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > > > group > > > > > > > Group['spark'] > > > > > > > > > > > > > > 2017-04-25 14:12:48,848 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > > > Group['zeppelin'] > > > > > > > {} > > > > > > > > > > > > > > 2017-04-25 14:12:48,849 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > > > group > > > > > > > Group['zeppelin'] > > > > > > > > > > > > > > 2017-04-25 14:12:48,884 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > > Group['hadoop'] > > > > > > {} > > > > > > > > > > > > > > 2017-04-25 14:12:48,885 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > > > group > > > > > > > Group['hadoop'] > > > > > > > > > > > > > > 2017-04-25 14:12:48,921 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > > Group['kibana'] > > > > > > {} > > > > > > > > > > > > > > 2017-04-25 14:12:48,922 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > > > group > > > > > > > Group['kibana'] > > > > > > > > > > > > > > 2017-04-25 14:12:48,959 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > > Group['users'] > > > > > > {} > > > > > > > > > > > > > > 2017-04-25 14:12:48,960 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > User['hive'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:48,961 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > > > user > > > > > > > User['hive'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,227 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > User['storm'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,227 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['storm'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,388 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > > > User['zookeeper'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,388 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['zookeeper'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,518 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': > > True, > > > > > > > 'groups': [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,518 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['infra-solr'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,578 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > User['ams'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,578 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['ams'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,625 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > User['tez'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > [u'users']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,626 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['tez'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,660 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > > User['zeppelin'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,660 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['zeppelin'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,705 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > > User['metron'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,705 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['metron'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,760 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > User['livy'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,760 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['livy'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,803 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > > > > User['elasticsearch'] {'gid': 'hadoop', > 'fetch_nonlocal_groups': > > > > True, > > > > > > > 'groups': [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,804 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['elasticsearch'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,837 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > User['spark'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,837 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['spark'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,885 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > > > User['ambari-qa'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > [u'users']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,885 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['ambari-qa'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,928 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > User['flume'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,928 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['flume'] > > > > > > > > > > > > > > 2017-04-25 14:12:49,972 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > User['kafka'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:49,973 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > > > user > > > > > > > User['kafka'] > > > > > > > > > > > > > > 2017-04-25 14:12:50,025 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > User['hdfs'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:50,026 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > > > user > > > > > > > User['hdfs'] > > > > > > > > > > > > > > 2017-04-25 14:12:50,075 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > User['yarn'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:50,075 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > > > user > > > > > > > User['yarn'] > > > > > > > > > > > > > > 2017-04-25 14:12:50,125 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > User['kibana'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:50,126 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > > > user > > > > > > > User['kibana'] > > > > > > > > > > > > > > 2017-04-25 14:12:50,175 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > User['mapred'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:50,176 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > > > user > > > > > > > User['mapred'] > > > > > > > > > > > > > > 2017-04-25 14:12:50,223 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > User['hbase'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:50,224 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > > > user > > > > > > > User['hbase'] > > > > > > > > > > > > > > 2017-04-25 14:12:50,271 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > User['hcat'] > > > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > > > [u'hadoop']} > > > > > > > > > > > > > > 2017-04-25 14:12:50,272 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > > > user > > > > > > > User['hcat'] > > > > > > > > > > > > > > 2017-04-25 14:12:50,320 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555} > > > > > > > > > > > > > > 2017-04-25 14:12:50,333 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Writing > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it > > doesn't > > > > > exist > > > > > > > > > > > > > > 2017-04-25 14:12:50,334 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Changing > > > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 > > to > > > > 555 > > > > > > > > > > > > > > 2017-04-25 14:12:50,335 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': > > '(test > > > > > $(id > > > > > > -u > > > > > > > ambari-qa) -gt 1000) || (false)'} > > > > > > > > > > > > > > 2017-04-25 14:12:50,364 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Skipping > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa > > > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ > > > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if > > > > > > > > > > > > > > 2017-04-25 14:12:50,365 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', > > 'create_parents': > > > > > True, > > > > > > > 'mode': 0775, 'cd_access': 'a'} > > > > > > > > > > > > > > 2017-04-25 14:12:50,366 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Creating > > > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist. > > > > > > > > > > > > > > 2017-04-25 14:12:50,366 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Changing > > > > > owner > > > > > > > for /tmp/hbase-hbase from 0 to hbase > > > > > > > > > > > > > > 2017-04-25 14:12:50,366 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Changing > > > > > > > permission for /tmp/hbase-hbase from 755 to 775 > > > > > > > > > > > > > > 2017-04-25 14:12:50,368 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': > > > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555} > > > > > > > > > > > > > > 2017-04-25 14:12:50,369 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase > > > > > > > > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/ > > hbase-hbase'] > > > > > > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'} > > > > > > > > > > > > > > 2017-04-25 14:12:50,393 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Skipping > > > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase > > > > > > > > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/ > > hbase-hbase'] > > > > > > > > > > > > due to not_if > > > > > > > > > > > > > > 2017-04-25 14:12:50,394 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > Group['hdfs'] > > > > > {} > > > > > > > > > > > > > > 2017-04-25 14:12:50,395 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > > > group > > > > > > > Group['hdfs'] > > > > > > > > > > > > > > 2017-04-25 14:12:50,430 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > User['hdfs'] > > > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']} > > > > > > > > > > > > > > 2017-04-25 14:12:50,431 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > Modifying > > > > > user > > > > > > > hdfs > > > > > > > > > > > > > > 2017-04-25 14:12:50,474 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS > Type: > > > > > > > > > > > > > > 2017-04-25 14:12:50,474 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > Directory['/etc/hadoop'] {'mode': 0755} > > > > > > > > > > > > > > 2017-04-25 14:12:50,474 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Creating > > > > > > > directory Directory['/etc/hadoop'] since it doesn't exist. > > > > > > > > > > > > > > 2017-04-25 14:12:50,475 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] > > > > > {'owner': > > > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777} > > > > > > > > > > > > > > 2017-04-25 14:12:50,476 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Creating > > > > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_ > > > > tmpdir'] > > > > > > > > > > > > since it doesn't exist. > > > > > > > > > > > > > > 2017-04-25 14:12:50,476 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Changing > > > > > owner > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to > > hdfs > > > > > > > > > > > > > > 2017-04-25 14:12:50,476 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Changing > > > > > group > > > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to > > > hadoop > > > > > > > > > > > > > > 2017-04-25 14:12:50,477 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Changing > > > > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir > > from > > > > > 755 > > > > > > > to 1777 > > > > > > > > > > > > > > 2017-04-25 14:12:50,477 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/'] > > > > > > > {'create_parents': True} > > > > > > > > > > > > > > 2017-04-25 14:12:50,478 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Creating > > > > > > > directory Directory['/var/lib/ambari- > > agent/tmp/AMBARI-artifacts/'] > > > > > since > > > > > > > it doesn't exist. > > > > > > > > > > > > > > 2017-04-25 14:12:50,479 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'] > > > > > {'content': > > > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources// > > > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'} > > > > > > > > > > > > > > 2017-04-25 14:12:50,504 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > Downloading > > > > > the > > > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77- > > > > > > > linux-x64.tar.gz > > > > > > > > > > > > > > 2017-04-25 14:12:53,793 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - > > > > > > > Directory['/usr/jdk64'] {} > > > > > > > > > > > > > > 2017-04-25 14:12:53,793 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - > Creating > > > > > > > directory Directory['/usr/jdk64'] since it doesn't exist. > > > > > > > > > > > > > > 2017-04-25 14:12:53,794 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - > > > > > > Execute[('chmod', > > > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True} > > > > > > > > > > > > > > 2017-04-25 14:12:53,824 > > > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - > > > Execute['cd > > > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf > > > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && > > > > ambari-sudo.sh > > > > > cp > > > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {} > > > > > > > > > > > > > > 2017-04-25 14:13:00,439 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] > {'action': > > > > > > > ['delete']} > > > > > > > > > > > > > > 2017-04-25 14:13:00,440 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > Removing > > > > > > > directory Directory['/var/lib/ambari- > agent/tmp/jdk_tmp_1BRMsS'] > > and > > > > > all > > > > > > > its content > > > > > > > > > > > > > > 2017-04-25 14:13:00,620 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, > > 'cd_access': > > > > > 'a'} > > > > > > > > > > > > > > 2017-04-25 14:13:00,621 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > > Execute[('chmod', > > > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True} > > > > > > > > > > > > > > 2017-04-25 14:13:00,682 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > Initializing > > > > > 2 > > > > > > > repositories > > > > > > > > > > > > > > 2017-04-25 14:13:00,683 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1. > > > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action': > > > > > ['create'], > > > > > > > 'components': [u'HDP', 'main'], 'repo_template': > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% > > endif > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP', > > > > > > > 'mirror_list': None} > > > > > > > > > > > > > > 2017-04-25 14:13:00,709 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content': > > InlineTemplate(...)} > > > > > > > > > > > > > > 2017-04-25 14:13:00,711 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > Writing > > > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist > > > > > > > > > > > > > > 2017-04-25 14:13:00,711 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': ' > > > http://public-repo-1. > > > > > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action': > > > > > ['create'], > > > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template': > > > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list > > > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% > > endif > > > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': > > > 'HDP-UTILS', > > > > > > > 'mirror_list': None} > > > > > > > > > > > > > > 2017-04-25 14:13:00,717 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content': > > > > > InlineTemplate(...)} > > > > > > > > > > > > > > 2017-04-25 14:13:00,717 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > Writing > > > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't > exist > > > > > > > > > > > > > > 2017-04-25 14:13:00,718 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > Package['unzip'] > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5} > > > > > > > > > > > > > > 2017-04-25 14:13:00,846 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > Installing > > > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip') > > > > > > > > > > > > > > 2017-04-25 14:13:03,413 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - > > > > > Package['curl'] > > > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5} > > > > > > > > > > > > > > 2017-04-25 14:13:03,429 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - > Skipping > > > > > > > installation of existing package curl > > > > > > > > > > > > > > 2017-04-25 14:13:03,430 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - > > > > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False, > > > > > > > 'retry_count': 5} > > > > > > > > > > > > > > 2017-04-25 14:13:03,451 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - > > Installing > > > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install > > hdp-select') > > > > > > > > > > > > > > 2017-04-25 14:13:04,948 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - > Install > > ES > > > > > Data > > > > > > > Node > > > > > > > > > > > > > > 2017-04-25 14:13:04,952 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - > > > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_ > unavailability': > > > > > False, > > > > > > > 'retry_count': 5} > > > > > > > > > > > > > > 2017-04-25 14:13:05,086 > > > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - > > Installing > > > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install > > > > > > > elasticsearch-2.3.3') > > > > > > > > > > > > > > > > > > > > > > > > > > > > Command failed after 1 tries > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler ( > > > ottobackwa...@gmail.com) > > > > > > > > > > wrote: > > > > > > > > > > > > > > Yes, I’m going to run through again and get the logs. The basic > > > issue > > > > > > is, > > > > > > > no repo tab -> if I should have one, > > > > > > > and the ES yum command failing with nothing to do, which stops > > > > > deployment > > > > > > > on all the nodes. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic ( > > > > > > > michael.miklav...@gmail.com) wrote: > > > > > > > > > > > > > > I've been doing an internal hwx deployment (HCP) recently for > > > testing > > > > > > > purposes. I have some ES tweaks to work out, but other than > that > > it > > > > > has > > > > > > > been running smoothly. Can you share any log info or > screenshots? > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" < > ottobackwa...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > Should I see the repo tab in the ambari wizard for metron at > > this > > > > > > point? > > > > > > > > Asking for a friend who can’t get the wizard to work :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Jon > > > > >