Getting the same error in ES as before from what I can see
On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwa...@gmail.com) wrote: I still can’t keep ES running, it dies. Also, I’m getting exceptions from enrichment trying to restart when the topology wasn’t running. Is there some coordination in this work? Should I just log jiras? If someone is already sussing out ES, we can sync up On April 26, 2017 at 11:35:03, zeo...@gmail.com (zeo...@gmail.com) wrote: I'm also interested to know why that's important at such a small scale. Jon On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > I am following > > https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html > I DO have master and data node together. > Why is that a problem? > > I will try again with master and data node separate. > > > On April 26, 2017 at 10:41:59, Michael Miklavcic ( > michael.miklav...@gmail.com) wrote: > > Hey Otto, > > How do you have the ES nodes configured? For a base install I would setup 1 > master (NOT as data node) and 2 data nodes (NOT on the same node as the > master). This is the install configuration I got working. You can also > modify some configuration properties around master node as data node, index > replicas, and gateway recovery to get it working differently, but this is > what will work OOTB with the default config settings from the mpack. If > you've already setup a master node and a data node on the same host, we'll > need to re-install. > > Mike > > > On Wed, Apr 26, 2017 at 7:02 AM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > > > File > > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line > > 293, in _call > > > > raise ExecutionFailed(err_msg, code, out, err) > > > > ExecutionFailed: Execution of 'service elasticsearch status' returned 3. > ● > > elasticsearch.service - Elasticsearch > > > > Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; > > disabled; > > vendor preset: disabled) > > > > Active: failed (Result: exit-code) since Tue 2017-04-25 22:14:58 EDT > > <http://airmail.calendar/2017-04-25%2022:14:58%20EDT>; 10h ago > > > > Docs: http://www.elastic.co > > > > Process: 16821 ExecStart=/usr/share/elasticsearch/bin/elasticsearch > > -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_ > > HOME} > > -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR} > > -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE) > > > > Process: 16819 > > ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec > > (code=exited, status=0/SUCCESS) > > > > Main PID: 16821 (code=exited, status=1/FAILURE) > > > > > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at > > org.elasticsearch.common.settings.Settings$Builder. > > loadFromStream(Settings.java:1080) > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at > > org.elasticsearch.common.settings.Settings$Builder. > > loadFromPath(Settings.java:1067) > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at > > org.elasticsearch.node.internal.InternalSettingsPreparer. > > prepareEnvironment(InternalSettingsPreparer.java:88) > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at > > org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202) > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at > > org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241) > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: at > > org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35) > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com elasticsearch[16821]: Refer > > to > > the log for complete error details. > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: > > elasticsearch.service: main process exited, code=exited, status=1/FAILURE > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: Unit > > elasticsearch.service entered failed state. > > > > Apr 25 22:14:58 <http://airmail.calendar/2017-04-25%2022:14:58%20EDT> > > ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: > > elasticsearch.service failed. > > > > INFO 2017-04-26 08:55:19,962 > > <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr > > > > > > On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwa...@gmail.com) > > wrote: > > > > Can you describe what ES issues you are working on? > > Fixing the repos got everything installed, but my ES components don’t > stay > > running. > > I need to harvest the errors. > > > > > > > > On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwa...@gmail.com) > > wrote: > > > > Nm. sorry. I fixed it. > > > > > > > > On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwa...@gmail.com) > > wrote: > > > > Ok, now I see the repos in the ‘pick version’ screen, but it is erring on > > the f://localrepo > > even though the folder exists, there is no repodata/repomd.xml. > > > > What is the command to create a local repo? > > > > > > > > On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwa...@gmail.com) > > wrote: > > > > I was going by the HW community page. > > > > Ok, Let me try it > > > > > > > > On April 25, 2017 at 16:04:07, David Lyle (dlyle65...@gmail.com) wrote: > > > > That would do it. It requires 2.4.2+. I would have sworn I put that in > the > > README, but I must have only annotated the PR. :( > > > > I'll get that in asap. > > > > -D... > > > > > > On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler <ottobackwa...@gmail.com> > > wrote: > > > > > 2.4.1 as a matter of fact > > > > > > > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65...@gmail.com) > wrote: > > > > > > Any chance they're running Ambari < 2.4.2? > > > > > > -D... > > > > > > > > > On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic < > > > michael.miklav...@gmail.com> wrote: > > > > > > > Hey Otto, I don't have wizard screenshots in front of me right now to > > > say > > > > for sure, but I do see a repoinfo.xml in the mpack. I haven't run > into > > > > anything like that, but next time I run through the install I can > take > > > > another look. > > > > > > > > https://github.com/apache/incubator-metron/blob/master/ > > > > metron-deployment/packaging/ambari/metron-mpack/src/main/ > > > > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml > > > > > > > > > > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler < > ottobackwa...@gmail.com > > > > > > > wrote: > > > > > > > > > stderr: > > > > > > > > > > Traceback (most recent call last): > > > > > > > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/ > > > > > 2.3.3/package/scripts/elastic_slave.py", line 71, in <module> > > > > > > > > > > Elasticsearch().execute() > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > libraries/script/script.py", > > > > > line 280, in execute > > > > > > > > > > method(env) > > > > > > > > > > File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/ > > > > > 2.3.3/package/scripts/elastic_slave.py", line 32, in install > > > > > > > > > > self.install_packages(env) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > libraries/script/script.py", > > > > > line 567, in install_packages > > > > > > > > > > retry_count=agent_stack_retry_count) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > core/base.py", > > > > > line 155, in __init__ > > > > > > > > > > self.env.run() > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > core/environment.py", > > > > > line 160, in run > > > > > > > > > > self.run_action(resource, action) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > core/environment.py", > > > > > line 124, in run_action > > > > > > > > > > provider_action() > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > core/providers/package/__init__.py", line 54, in action_install > > > > > > > > > > self.install_package(package_name, self.resource.use_repos, > > > > > self.resource.skip_repos) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > core/providers/package/yumrpm.py", line 49, in install_package > > > > > > > > > > self.checked_call_with_retries(cmd, sudo=True, > > > > > logoutput=self.get_logoutput()) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > core/providers/package/__init__.py", line 83, in > > > > checked_call_with_retries > > > > > > > > > > return self._call_with_retries(cmd, is_checked=True, **kwargs) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > > core/providers/package/__init__.py", line 91, in _call_with_retries > > > > > > > > > > code, out = func(cmd, **kwargs) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > core/shell.py", > > > > > line 71, in inner > > > > > > > > > > result = function(command, **kwargs) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > core/shell.py", > > > > > line 93, in checked_call > > > > > > > > > > tries=tries, try_sleep=try_sleep) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > core/shell.py", > > > > > line 141, in _call_wrapper > > > > > > > > > > result = _call(command, **kwargs_copy) > > > > > > > > > > File "/usr/lib/python2.6/site-packages/resource_management/ > > > > core/shell.py", > > > > > line 294, in _call > > > > > > > > > > raise Fail(err_msg) > > > > > > > > > > resource_management.core.exceptions.Fail: Execution of > '/usr/bin/yum > > > -d > > > > 0 > > > > > -e 0 -y install elasticsearch-2.3.3' returned 1. Error: Nothing to > do > > > > > > > > > > stdout: > > > > > > > > > > 2017-04-25 14:12:48,669 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Using > hadoop > > > > conf > > > > > dir: /usr/hdp/current/hadoop-client/conf > > > > > > > > > > 2017-04-25 14:12:48,671 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > Group['metron'] > > > > {} > > > > > > > > > > 2017-04-25 14:12:48,673 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > group > > > > > Group['metron'] > > > > > > > > > > 2017-04-25 14:12:48,740 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > Group['livy'] > > > {} > > > > > > > > > > 2017-04-25 14:12:48,741 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > group > > > > > Group['livy'] > > > > > > > > > > 2017-04-25 14:12:48,774 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > > Group['elasticsearch'] {} > > > > > > > > > > 2017-04-25 14:12:48,775 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > group > > > > > Group['elasticsearch'] > > > > > > > > > > 2017-04-25 14:12:48,812 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > Group['spark'] > > > > {} > > > > > > > > > > 2017-04-25 14:12:48,812 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > group > > > > > Group['spark'] > > > > > > > > > > 2017-04-25 14:12:48,848 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > > Group['zeppelin'] > > > > > {} > > > > > > > > > > 2017-04-25 14:12:48,849 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > group > > > > > Group['zeppelin'] > > > > > > > > > > 2017-04-25 14:12:48,884 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > Group['hadoop'] > > > > {} > > > > > > > > > > 2017-04-25 14:12:48,885 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > group > > > > > Group['hadoop'] > > > > > > > > > > 2017-04-25 14:12:48,921 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > Group['kibana'] > > > > {} > > > > > > > > > > 2017-04-25 14:12:48,922 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > group > > > > > Group['kibana'] > > > > > > > > > > 2017-04-25 14:12:48,959 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > > > Group['users'] > > > > {} > > > > > > > > > > 2017-04-25 14:12:48,960 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - > User['hive'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:48,961 > > > > > <http://airmail.calendar/2017-04-25%2014:12:48%20EDT> - Adding > user > > > > > User['hive'] > > > > > > > > > > 2017-04-25 14:12:49,227 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > User['storm'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,227 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['storm'] > > > > > > > > > > 2017-04-25 14:12:49,388 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > User['zookeeper'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,388 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['zookeeper'] > > > > > > > > > > 2017-04-25 14:12:49,518 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > > User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, > > > > > 'groups': [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,518 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['infra-solr'] > > > > > > > > > > 2017-04-25 14:12:49,578 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > User['ams'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,578 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['ams'] > > > > > > > > > > 2017-04-25 14:12:49,625 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > User['tez'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > [u'users']} > > > > > > > > > > 2017-04-25 14:12:49,626 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['tez'] > > > > > > > > > > 2017-04-25 14:12:49,660 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > User['zeppelin'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,660 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['zeppelin'] > > > > > > > > > > 2017-04-25 14:12:49,705 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > User['metron'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,705 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['metron'] > > > > > > > > > > 2017-04-25 14:12:49,760 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > User['livy'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,760 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['livy'] > > > > > > > > > > 2017-04-25 14:12:49,803 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > > User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': > > True, > > > > > 'groups': [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,804 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['elasticsearch'] > > > > > > > > > > 2017-04-25 14:12:49,837 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > User['spark'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,837 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['spark'] > > > > > > > > > > 2017-04-25 14:12:49,885 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > > > User['ambari-qa'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > [u'users']} > > > > > > > > > > 2017-04-25 14:12:49,885 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['ambari-qa'] > > > > > > > > > > 2017-04-25 14:12:49,928 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > User['flume'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,928 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['flume'] > > > > > > > > > > 2017-04-25 14:12:49,972 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - > > User['kafka'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:49,973 > > > > > <http://airmail.calendar/2017-04-25%2014:12:49%20EDT> - Adding > user > > > > > User['kafka'] > > > > > > > > > > 2017-04-25 14:12:50,025 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > User['hdfs'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:50,026 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > user > > > > > User['hdfs'] > > > > > > > > > > 2017-04-25 14:12:50,075 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > User['yarn'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:50,075 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > user > > > > > User['yarn'] > > > > > > > > > > 2017-04-25 14:12:50,125 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > User['kibana'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:50,126 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > user > > > > > User['kibana'] > > > > > > > > > > 2017-04-25 14:12:50,175 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > User['mapred'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:50,176 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > user > > > > > User['mapred'] > > > > > > > > > > 2017-04-25 14:12:50,223 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > User['hbase'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:50,224 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > user > > > > > User['hbase'] > > > > > > > > > > 2017-04-25 14:12:50,271 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > User['hcat'] > > > > > {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': > > > [u'hadoop']} > > > > > > > > > > 2017-04-25 14:12:50,272 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > user > > > > > User['hcat'] > > > > > > > > > > 2017-04-25 14:12:50,320 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555} > > > > > > > > > > 2017-04-25 14:12:50,333 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Writing > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't > > > exist > > > > > > > > > > 2017-04-25 14:12:50,334 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing > > > > > permission for /var/lib/ambari-agent/tmp/changeUid.sh from 644 to > > 555 > > > > > > > > > > 2017-04-25 14:12:50,335 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test > > > $(id > > > > -u > > > > > ambari-qa) -gt 1000) || (false)'} > > > > > > > > > > 2017-04-25 14:12:50,364 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa > > > > > /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ > > > > > ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if > > > > > > > > > > 2017-04-25 14:12:50,365 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': > > > True, > > > > > 'mode': 0775, 'cd_access': 'a'} > > > > > > > > > > 2017-04-25 14:12:50,366 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating > > > > > directory Directory['/tmp/hbase-hbase'] since it doesn't exist. > > > > > > > > > > 2017-04-25 14:12:50,366 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing > > > owner > > > > > for /tmp/hbase-hbase from 0 to hbase > > > > > > > > > > 2017-04-25 14:12:50,366 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing > > > > > permission for /tmp/hbase-hbase from 755 to 775 > > > > > > > > > > 2017-04-25 14:12:50,368 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': > > > > > StaticFile('changeToSecureUid.sh'), 'mode': 0555} > > > > > > > > > > 2017-04-25 14:12:50,369 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase > > > > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] > > > > > > > > {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'} > > > > > > > > > > 2017-04-25 14:12:50,393 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Skipping > > > > > Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase > > > > > > > /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] > > > > > > > > due to not_if > > > > > > > > > > 2017-04-25 14:12:50,394 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > Group['hdfs'] > > > {} > > > > > > > > > > 2017-04-25 14:12:50,395 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Adding > group > > > > > Group['hdfs'] > > > > > > > > > > 2017-04-25 14:12:50,430 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > User['hdfs'] > > > > > {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']} > > > > > > > > > > 2017-04-25 14:12:50,431 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Modifying > > > user > > > > > hdfs > > > > > > > > > > 2017-04-25 14:12:50,474 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - FS Type: > > > > > > > > > > 2017-04-25 14:12:50,474 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > Directory['/etc/hadoop'] {'mode': 0755} > > > > > > > > > > 2017-04-25 14:12:50,474 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating > > > > > directory Directory['/etc/hadoop'] since it doesn't exist. > > > > > > > > > > 2017-04-25 14:12:50,475 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] > > > {'owner': > > > > > 'hdfs', 'group': 'hadoop', 'mode': 01777} > > > > > > > > > > 2017-04-25 14:12:50,476 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating > > > > > directory Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_ > > tmpdir'] > > > > > > > > since it doesn't exist. > > > > > > > > > > 2017-04-25 14:12:50,476 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing > > > owner > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to hdfs > > > > > > > > > > 2017-04-25 14:12:50,476 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing > > > group > > > > > for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from 0 to > hadoop > > > > > > > > > > 2017-04-25 14:12:50,477 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Changing > > > > > permission for /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir from > > > 755 > > > > > to 1777 > > > > > > > > > > 2017-04-25 14:12:50,477 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/'] > > > > > {'create_parents': True} > > > > > > > > > > 2017-04-25 14:12:50,478 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - Creating > > > > > directory Directory['/var/lib/ambari-agent/tmp/AMBARI-artifacts/'] > > > since > > > > > it doesn't exist. > > > > > > > > > > 2017-04-25 14:12:50,479 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > > > > > File['/var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'] > > > {'content': > > > > > DownloadSource('http://css-fx-met-poc-82:8080/resources// > > > > > jdk-8u77-linux-x64.tar.gz'), 'not_if': 'test -f > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz'} > > > > > > > > > > 2017-04-25 14:12:50,504 > > > > > <http://airmail.calendar/2017-04-25%2014:12:50%20EDT> - > Downloading > > > the > > > > > file from http://css-fx-met-poc-82:8080/resources//jdk-8u77- > > > > > linux-x64.tar.gz > > > > > > > > > > 2017-04-25 14:12:53,793 > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - > > > > > Directory['/usr/jdk64'] {} > > > > > > > > > > 2017-04-25 14:12:53,793 > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - Creating > > > > > directory Directory['/usr/jdk64'] since it doesn't exist. > > > > > > > > > > 2017-04-25 14:12:53,794 > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - > > > > Execute[('chmod', > > > > > 'a+x', u'/usr/jdk64')] {'sudo': True} > > > > > > > > > > 2017-04-25 14:12:53,824 > > > > > <http://airmail.calendar/2017-04-25%2014:12:53%20EDT> - > Execute['cd > > > > > /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS && tar -xf > > > > > /var/lib/ambari-agent/tmp/jdk-8u77-linux-x64.tar.gz && > > ambari-sudo.sh > > > cp > > > > > -rp /var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS/* /usr/jdk64'] {} > > > > > > > > > > 2017-04-25 14:13:00,439 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] {'action': > > > > > ['delete']} > > > > > > > > > > 2017-04-25 14:13:00,440 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Removing > > > > > directory Directory['/var/lib/ambari-agent/tmp/jdk_tmp_1BRMsS'] and > > > all > > > > > its content > > > > > > > > > > 2017-04-25 14:13:00,620 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > File['/usr/jdk64/jdk1.8.0_77/bin/java'] {'mode': 0755, 'cd_access': > > > 'a'} > > > > > > > > > > 2017-04-25 14:13:00,621 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > Execute[('chmod', > > > > > '-R', '755', u'/usr/jdk64/jdk1.8.0_77')] {'sudo': True} > > > > > > > > > > 2017-04-25 14:13:00,682 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > Initializing > > > 2 > > > > > repositories > > > > > > > > > > 2017-04-25 14:13:00,683 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > Repository['HDP-2.5'] {'base_url': 'http://public-repo-1. > > > > > hortonworks.com/HDP/centos7/2.x/updates/2.5.5.0', 'action': > > > ['create'], > > > > > 'components': [u'HDP', 'main'], 'repo_template': > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP', > > > > > 'mirror_list': None} > > > > > > > > > > 2017-04-25 14:13:00,709 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)} > > > > > > > > > > 2017-04-25 14:13:00,711 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing > > > > > File['/etc/yum.repos.d/HDP.repo'] because it doesn't exist > > > > > > > > > > 2017-04-25 14:13:00,711 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > Repository['HDP-UTILS-1.1.0.21'] {'base_url': ' > http://public-repo-1. > > > > > > hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action': > > > ['create'], > > > > > 'components': [u'HDP-UTILS', 'main'], 'repo_template': > > > > > '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list > > > > > %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif > > > > > %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': > 'HDP-UTILS', > > > > > 'mirror_list': None} > > > > > > > > > > 2017-04-25 14:13:00,717 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content': > > > InlineTemplate(...)} > > > > > > > > > > 2017-04-25 14:13:00,717 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Writing > > > > > File['/etc/yum.repos.d/HDP-UTILS.repo'] because it doesn't exist > > > > > > > > > > 2017-04-25 14:13:00,718 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - > > > Package['unzip'] > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5} > > > > > > > > > > 2017-04-25 14:13:00,846 > > > > > <http://airmail.calendar/2017-04-25%2014:13:00%20EDT> - Installing > > > > > package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip') > > > > > > > > > > 2017-04-25 14:13:03,413 > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - > > > Package['curl'] > > > > > {'retry_on_repo_unavailability': False, 'retry_count': 5} > > > > > > > > > > 2017-04-25 14:13:03,429 > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping > > > > > installation of existing package curl > > > > > > > > > > 2017-04-25 14:13:03,430 > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - > > > > > Package['hdp-select'] {'retry_on_repo_unavailability': False, > > > > > 'retry_count': 5} > > > > > > > > > > 2017-04-25 14:13:03,451 > > > > > <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Installing > > > > > package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install hdp-select') > > > > > > > > > > 2017-04-25 14:13:04,948 > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - Install ES > > > Data > > > > > Node > > > > > > > > > > 2017-04-25 14:13:04,952 > > > > > <http://airmail.calendar/2017-04-25%2014:13:04%20EDT> - > > > > > Package['elasticsearch-2.3.3'] {'retry_on_repo_unavailability': > > > False, > > > > > 'retry_count': 5} > > > > > > > > > > 2017-04-25 14:13:05,086 > > > > > <http://airmail.calendar/2017-04-25%2014:13:05%20EDT> - Installing > > > > > package elasticsearch-2.3.3 ('/usr/bin/yum -d 0 -e 0 -y install > > > > > elasticsearch-2.3.3') > > > > > > > > > > > > > > > > > > > > Command failed after 1 tries > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On April 25, 2017 at 13:47:17, Otto Fowler ( > ottobackwa...@gmail.com) > > > > > > wrote: > > > > > > > > > > Yes, I’m going to run through again and get the logs. The basic > issue > > > > is, > > > > > no repo tab -> if I should have one, > > > > > and the ES yum command failing with nothing to do, which stops > > > deployment > > > > > on all the nodes. > > > > > > > > > > > > > > > > > > > > On April 25, 2017 at 13:36:25, Michael Miklavcic ( > > > > > michael.miklav...@gmail.com) wrote: > > > > > > > > > > I've been doing an internal hwx deployment (HCP) recently for > testing > > > > > purposes. I have some ES tweaks to work out, but other than that it > > > has > > > > > been running smoothly. Can you share any log info or screenshots? > > > > > > > > > > Mike > > > > > > > > > > On Apr 25, 2017 11:30 AM, "Otto Fowler" <ottobackwa...@gmail.com> > > > wrote: > > > > > > > > > > > Hi, > > > > > > Should I see the repo tab in the ambari wizard for metron at this > > > > point? > > > > > > Asking for a friend who can’t get the wizard to work :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Jon
