Hi Dima,

I think it would make a lot of sense to make 642 a sub-task of 634.  Certainly 
it’s a needed improvement and there’s no need to wait for the laundry list of 
other stuff in 634.  In fact, shorter is better.  But gathering the pieces as 
subtasks will make it easier to track and avoid duplicating work.

Regarding 641, as I commented in the PR, I think the proposed change is only 
needed if the user is using Python 2.6.  But there’s no harm in making the 
change anyway.  Since it isn’t mentioned in 634, just go ahead and pursue it 
separately.

Thanks for asking.

Regarding elasticsearch_config_path in 
metron-deployment/roles/metron_streaming/defaults/main.yml , I don’t really 
know.  I think it is probably a reference to the location of elastic-env.sh, 
which is itself not actually used for anything as far as I can tell.  In 
METRON-634 I actually propose removing elastic-env.sh and its progenitor file, 
elastic-env.xml.  But other more knowledgeable people haven’t had the chance to 
chime in on that yet :-)

--Matt


On 1/2/17, 11:01 PM, "Dima Kovalyov" <dima.koval...@sstech.us> wrote:

    Thank you Matt,
    
    I haven't seen 634 before. Should I merge my tickets as sub-tasks
    addressing some of the points your bring up there?
    
    Also, I have a note about elasticsearch_config_path which is set in
    metron-deployment/roles/metron_streaming/defaults/main.yml. It seems
    like it is not used anywhere in the code base.
    
    Please let me know how I should proceed with two tickets I have created
    that are relevant to yours.
    
    - Dima
    
    On 01/01/2017 04:43 AM, Matt Foley wrote:
    > Hi Dima,
    > Great to have the how-to doc in the wiki where it belongs.  Now we have a 
doc to edit as we improve the install process :-)
    >
    > Did you look at https://issues.apache.org/jira/browse/METRON-634 before 
opening METRON-642?  Please see my comments in the Jira for METRON-642.
    > Thanks,
    > --Matt
    >
    >
    >
    > On 12/30/16, 11:44 PM, "Dima Kovalyov" <dima.koval...@sstech.us> wrote:
    >
    >     Hey,
    >     
    >     I wanted to finish what I've started with document for Metron with HDP
    >     2.5, so I have migrated document (with minor text fixes and
    >     clarifications) to here:
    >     
https://cwiki.apache.org/confluence/display/METRON/Metron+with+HDP+2.5+bare-metal+install
    >     Old google doc was replaced with the link to this article.
    >     
    >     I also, created number of pull requests to fix minor bugs here and 
there
    >     and created these two tickets: METRON-641 and METRON-642.
    >     Please let me know if I did something out of proper procedure.
    >     
    >     Also, I agree that we should eventually strip HDP related steps from 
the
    >     document, so in the end it will be like:
    >     1. Build Mpack
    >     2. Add to Ambari
    >     3. Assigned Masters and Slave
    >     4. PROFIT
    >     But since we are where we are, let's leave it like that and fix all 
the
    >     bugs first.
    >     
    >     p.s. have a happy holidays everyone
    >     
    >     - Dima
    >     
    >     On 12/16/2016 04:21 AM, Matt Foley wrote:
    >     > I seem to have found the difficulty.  It will NOT show up on any 
system that has /bin/java defined, which may account for why other folks with 
Centos7 test systems aren’t seeing the behavior.
    >     >
    >     > On my Centos7 test system, it so happens that /bin/java is not 
defined, even though $JAVA_HOME is correctly defined, and “$JAVA_HOME/bin” is 
in the PATH.  In Centos7, when services launch through the (new in 7) systemctl 
process, it drops all inherited environment variables and starts over fresh.  
Although the systemd launch script 
/usr/lib/systemd/system/elasticsearch.service does read in the 
/etc/sysconfig/elasticsearch as an “EnvironmentFile”, it does not include 
JAVA_HOME.
    >     >
    >     > When, eventually, the user-level launcher script at 
/usr/share/elasticsearch/bin/elasticsearch gets invoked, JAVA_HOME is still 
undefined.  But it looks for $JAVA_HOME/bin/java, so if “/bin/java” is linked 
in the file system, then it’s good!  But if not, the launcher script dies.  
Regrettably that launcher script, even though it is fairly complex, does not 
write to any log file, and its stdout was closed long ago by the service-level 
launcher.  So I had to hack it to see what it was doing.
    >     >
    >     > The solution is to simply write JAVA_HOME={{java64_home}} into the 
elastic-sysconfig template.
    >     >
    >     > BTW, while munging thru code I reached the conclusion that 
elastic-env.sh is basically orphaned.  Does anyone know of scripts that source 
it? (Of course elastic-env.xml is still important, I’m only asking about the 
elastic-env.sh file templated from it.)
    >     >
    >     > Thanks,
    >     > --Matt
    >     >
    >     >
    >     > On 12/14/16, 2:41 PM, "Matt Foley" <mfo...@hortonworks.com> wrote:
    >     >
    >     > No, node.data and node.master are both correctly set to true (with 
Ambari’s agreement/participation) in the elasticsearch.yml file in CONF_DIR, 
and this is being correctly picked up by ES when launched interactively.  I 
really think this is in the service management stuff in 
/etc/init.d/elasticsearch and /usr/share/elasticsearch/bin/elasticsearch .  
Remains to be proven, of course…
    >     >
    >     > The reason I think ES isn’t even being successfully launched by 
systemd, is there is zero logging anywhere, except in ambari where it shows 
nothing but a successful service launch.  No files created in 
/var/log/elasticsearch, which all scripts agree is the value of LOG_DIR, 
despite permissions set to “drwxr-xr-x. elasticsearch elasticsearch”
    >     >
    >     > Thanks,
    >     > --Matt
    >     >
    >     > On 12/14/16, 2:07 PM, "David Lyle" <dlyle65...@gmail.com> wrote:
    >     >
    >     >     Aha! There's your problem. :)
    >     >     
    >     >     Kidding aside, that is weird. I would expect the ES instance to 
come up and
    >     >     go status red right away, not up and die.
    >     >     
    >     >     I did have a horrible, horrible hack that made all that work, 
it involved
    >     >     modifying the stored es templates to both have node.master and 
node.data
    >     >     set to true in
    >     >     /var/lib/ambari-server/resources/common-services/ELASTICSEARCH 
(from
    >     >     memory, path may be a bit off). It occurred to me that an easy 
incremental
    >     >     step toward to full METRON-608 would be to simply expose the 
templates to
    >     >     the config pages...
    >     >     
    >     >     -D...
    >     >     
    >     >     
    >     >     On Wed, Dec 14, 2016 at 4:59 PM, Matt Foley 
<mfo...@hortonworks.com> wrote:
    >     >     
    >     >     > David,
    >     >     > No, it’s in my METRON-608 single-node deployment  :-)
    >     >     >
    >     >     > On 12/14/16, 12:46 PM, "David Lyle" <dlyle65...@gmail.com> 
wrote:
    >     >     >
    >     >     >     Hi Matt and Jon,
    >     >     >
    >     >     >     FWIW, Metron with the MPack has been tested extensively 
on CentOS 7.
    >     >     > Works
    >     >     >     like a champ. The issue right now is that sensor install 
is a CentOS
    >     >     >     6/Ansible proposition.
    >     >     >
    >     >     >     I haven't seen the issue with ES that you're 
experiencing, Matt. Is
    >     >     > that in
    >     >     >     a 1 Master 3 Data node ES config or something else?
    >     >     >
    >     >     >     -D...
    >     >     >
    >     >     >
    >     >     >     On Wed, Dec 14, 2016 at 3:29 PM, Matt Foley 
<mfo...@hortonworks.com>
    >     >     > wrote:
    >     >     >
    >     >     >     > I hope we will try to support Centos7. Many of my 
company’s
    >     >     > customers are
    >     >     >     > requiring new installs to use RHEL 7 or Centos 7 rather 
than 6.  In
    >     >     >     > addition, we have the benefit that Centos 6 RPMs 
basically always
    >     >     > run fine
    >     >     >     > in Centos7 (barring heavy-duty low-level system 
manipulations that
    >     >     > Metron,
    >     >     >     > Kibana, and ES just don’t use), ES doesn’t distinguish 
between them,
    >     >     > and
    >     >     >     > Kibana install support for Centos7 is already in place 
in the
    >     >     > Mpack.  And
    >     >     >     > of course the Hadoop Stack runs fine in 7.
    >     >     >     >
    >     >     >     > For what it’s worth, I’ve been working with the Mpack 
installation on
    >     >     >     > Centos7 for the last few weeks, and it works fine 
except for a
    >     >     > persistent
    >     >     >     > issue where the Elasticsearch service works fine if 
launched
    >     >     > interactively
    >     >     >     > but terminates immediately if launched with same 
arguments as a
    >     >     > service.
    >     >     >     > Hope to find the cause in the next couple days.  (If 
anyone knows
    >     >     > why,
    >     >     >     > would love to hear.  The daemonized launch scripts come 
from ES, not
    >     >     > from
    >     >     >     > Metron, so “should just work”.)
    >     >     >     >
    >     >     >     > Thanks,
    >     >     >     > --Matt
    >     >     >     >
    >     >     >     > On 12/13/16, 6:46 PM, "zeo...@gmail.com" 
<zeo...@gmail.com> wrote:
    >     >     >     >
    >     >     >     >     For now, I'm not sure what the solution is, but I 
would think
    >     >     > choosing
    >     >     >     > one
    >     >     >     >     specific list of required/supported software 
(including OS) and
    >     >     >     > documenting
    >     >     >     >     that thoroughly is the right start.  Something like 
what
    >     >     > currently
    >     >     >     > exists
    >     >     >     >     for the vagrant side of things.
    >     >     >     >
    >     >     >     >     That said, long term I would love to see broader 
support and
    >     >     >     > automatically
    >     >     >     >     generating documentation derived from the code.  
Isn't CentOS 6
    >     >     > still
    >     >     >     >     preferred over 7?  That's what I've been working on 
solely for
    >     >     > that
    >     >     >     >     reason.
    >     >     >     >
    >     >     >     >     Jon
    >     >     >     >
    >     >     >     >     On Tue, Dec 13, 2016, 21:07 Matt Foley 
<ma...@apache.org> wrote:
    >     >     >     >
    >     >     >     >     > In my work on METRON-608, I’ve found a lot of 
small but
    >     >     > significant
    >     >     >     > bugs
    >     >     >     >     > in the existing Mpack (version 0.3.0).  These 
bugs integrate
    >     >     > with a
    >     >     >     > lot of
    >     >     >     >     > the oddball tweaks specified in the draft Install 
docs I saw.
    >     >     > I
    >     >     >     > would like
    >     >     >     >     > to submit a PR for these bugs, but they must be 
accompanied by
    >     >     >     > changes in
    >     >     >     >     > the Install doc.
    >     >     >     >     >
    >     >     >     >     > How do we want to manage this?  Should we version 
the Install
    >     >     > doc?
    >     >     >     > Or
    >     >     >     >     > have multiple sections for the different 
versions?  Or have
    >     >     >     > footnotes or
    >     >     >     >     > sidebar comments about which version certain 
paragraphs do and
    >     >     > do
    >     >     >     > not apply
    >     >     >     >     > to?
    >     >     >     >     >
    >     >     >     >     > I’m inclined to use sidebars within the document, 
because the
    >     >     >     > dependencies
    >     >     >     >     > aren’t just on which versions of HDP, Ambari, and 
Metron you
    >     >     > use.
    >     >     >     > They
    >     >     >     >     > also depend on Python 2.6 vs 2.7, and Centos 6 vs 
7.
    >     >     >     >     >
    >     >     >     >     > Thanks,
    >     >     >     >     > --Matt
    >     >     >     >     >
    >     >     >     >     >
    >     >     >     >     > On 12/13/16, 4:13 PM, "zeo...@gmail.com" 
<zeo...@gmail.com>
    >     >     > wrote:
    >     >     >     >     >
    >     >     >     >     >     Sorry about the delay.  To be honest I 
delayed once I saw
    >     >     > the
    >     >     >     >     > management UI
    >     >     >     >     >     PR because I was going to wait for that to be 
merged into
    >     >     > master
    >     >     >     >     > before I
    >     >     >     >     >     did my mpack install.  My initial thought is 
that there
    >     >     > may need
    >     >     >     > to be
    >     >     >     >     > some
    >     >     >     >     >     sort of a merger between what you provided 
and the
    >     >     > Hortonworks
    >     >     >     > blog
    >     >     >     >     > post,
    >     >     >     >     >     as I would prefer a single, comprehensive 
post over
    >     >     >     > fragmentation.
    >     >     >     >     >
    >     >     >     >     >     Jon
    >     >     >     >     >
    >     >     >     >     >     On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <
    >     >     >     > dima.koval...@sstech.us>
    >     >     >     >     > wrote:
    >     >     >     >     >
    >     >     >     >     >     > Thank you Jon,
    >     >     >     >     >     >
    >     >     >     >     >     > Just wondering if you have finished 
polishing the
    >     >     > document? I
    >     >     >     > will
    >     >     >     >     > have
    >     >     >     >     >     > some time this week to polish and publish 
it if there is
    >     >     >     > anything
    >     >     >     >     > left
    >     >     >     >     >     > to format.
    >     >     >     >     >     >
    >     >     >     >     >     > - Dima
    >     >     >     >     >     >
    >     >     >     >     >     > On 12/01/2016 04:41 PM, zeo...@gmail.com 
wrote:
    >     >     >     >     >     > > While I can't delegate access to the 
wiki, I am
    >     >     > working on
    >     >     >     >     > migrating the
    >     >     >     >     >     > > documentation over.  I have been 
modifying the
    >     >     > document for
    >     >     >     >     > clarity and
    >     >     >     >     >     > > updating up some of the formatting, so 
please give me
    >     >     > some
    >     >     >     > time to
    >     >     >     >     > do so.
    >     >     >     >     >     > >
    >     >     >     >     >     > > Jon
    >     >     >     >     >     > >
    >     >     >     >     >     > > On Thu, Dec 1, 2016 at 9:33 AM Dima 
Kovalyov <
    >     >     >     >     > dima.koval...@sstech.us>
    >     >     >     >     >     > > wrote:
    >     >     >     >     >     > >
    >     >     >     >     >     > >> Hello,
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Who can provide me with an access to add 
new page in
    >     >     > this
    >     >     >     >     > Installation
    >     >     >     >     >     > >> space below?
    >     >     >     >     >     > >> 
https://cwiki.apache.org/confluence/display/METRON/
    >     >     >     > Installation
    >     >     >     >     >     > >> I want to add up-to-date article on 
how-to install
    >     >     > Metron
    >     >     >     > using
    >     >     >     >     > Ambari
    >     >     >     >     >     > >> mpack.
    >     >     >     >     >     > >> Please advise.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> My username is "dima" registered with 
electronic mail
    >     >     >     > dimdroll
    >     >     >     >     > hosted by
    >     >     >     >     >     > >> gmail.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Right now I have it posted here:
    >     >     >     >     >     > >> https://goo.gl/HWGwpj
    >     >     >     >     >     > >> But instead of maintaining it there it 
will be
    >     >     > convenient
    >     >     >     > for me
    >     >     >     >     > to have
    >     >     >     >     >     > >> it in confluence instead of gdocs as I 
am using
    >     >     > confluence
    >     >     >     > in our
    >     >     >     >     >     > internal
    >     >     >     >     >     > >> project as well, so I can basically copy 
formatted doc
    >     >     >     > between
    >     >     >     >     > each
    >     >     >     >     >     > other.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Thank you.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> - Dima
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> On 11/30/2016 04:57 PM, 
zeo...@gmail.com<mailto:
    >     >     > Zeolla
    >     >     >     > @GMail.com>
    >     >     >     >     >     > wrote:
    >     >     >     >     >     > >> Probably the best long term place for 
this would be
    >     >     > here<
    >     >     >     >     >     > >> 
https://cwiki.apache.org/confluence/display/METRON/
    >     >     >     > Installation>,
    >     >     >     >     >     > >> potentially with a new mpack article (to
    >     >     > differentiate from
    >     >     >     > the
    >     >     >     >     > ambari
    >     >     >     >     >     > >> install which is already there, but 
dated).  I can
    >     >     > help with
    >     >     >     >     > migrating
    >     >     >     >     >     > it
    >     >     >     >     >     > >> to the wiki - I will run through your 
documentation
    >     >     > later
    >     >     >     > today
    >     >     >     >     > and
    >     >     >     >     >     > request
    >     >     >     >     >     > >> edit permissions if I find anything 
worth updating.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Would you be OK if I put your doc in the 
wiki at some
    >     >     > point?
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Jon
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> On Wed, Nov 30, 2016 at 9:51 AM Dima 
Kovalyov <
    >     >     >     >     > dima.koval...@sstech.us
    >     >     >     >     >     > >> <mailto:dima.koval...@sstech.us>> wrote:
    >     >     >     >     >     > >> As multiple users asked I have published 
document for
    >     >     >     > installing
    >     >     >     >     > Metron
    >     >     >     >     >     > on
    >     >     >     >     >     > >> Centos 6 using Ambari 2.4 with HDP 2.5:
    >     >     >     >     >     > >> https://goo.gl/HWGwpj
    >     >     >     >     >     > >> It may look ugly as this is direct 
export from
    >     >     > internal
    >     >     >     >     > confluence with
    >     >     >     >     >     > >> cosmetic changes and our internal 
details removed.
    >     >     > Let me
    >     >     >     > know if
    >     >     >     >     > there
    >     >     >     >     >     > is
    >     >     >     >     >     > >> a better place to publish it.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Feel free to request edit permissions, I 
would like
    >     >     > the
    >     >     >     > document
    >     >     >     >     > to be
    >     >     >     >     >     > >> up-to-date, simplified and corrected if 
there are any
    >     >     >     > errors. The
    >     >     >     >     > goal
    >     >     >     >     >     > for
    >     >     >     >     >     > >> this document is to provide installation 
guidance in a
    >     >     >     > manner
    >     >     >     >     > that Linux
    >     >     >     >     >     > >> administrator with basic knowledge could 
perform
    >     >     >     > installation
    >     >     >     >     > without
    >     >     >     >     >     > >> opening any other links or searching for 
commands,
    >     >     > step by
    >     >     >     > step.
    >     >     >     >     >     > >> Thank you.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> p.s. feel free to let me know if you 
will encounter
    >     >     > any
    >     >     >     > errors as
    >     >     >     >     > I've
    >     >     >     >     >     > >> seen them bunch.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> - Dima
    >     >     >     >     >     > >>
    >     >     >     >     >     > >>
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> On 11/30/2016 04:45 PM, 
zeo...@gmail.com<mailto:
    >     >     > Zeolla
    >     >     >     > @GMail.com>
    >     >     >     >     >     > wrote:
    >     >     >     >     >     > >> I also have written up a method to 
retrieve and setup
    >     >     > the
    >     >     >     > required
    >     >     >     >     >     > >> packages in my script<
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
https://github.com/JonZeolla/Development/blob/master/Bash/
    >     >     >     > setupMetron.sh>
    >     >     >     >     >     > >> which could probably be cleaned up and 
incorporated.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Jon
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> On Wed, Nov 30, 2016 at 9:24 AM Nick 
Allen <
    >     >     >     > n...@nickallen.org
    >     >     >     >     > <mailto:
    >     >     >     >     >     > >> n...@nickallen.org>> wrote:
    >     >     >     >     >     > >> In regards to your comment about a "full 
package
    >     >     > version
    >     >     >     > dump"
    >     >     >     >     > there is
    >     >     >     >     >     > at
    >     >     >     >     >     > >> least a start at 
`metron-deployment/scripts/
    >     >     >     > platform-info.sh`.
    >     >     >     >     > If we
    >     >     >     >     >     > can
    >     >     >     >     >     > >> enhance that, let's do it.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <
    >     >     > sho...@gmail.com
    >     >     >     > <mailto:
    >     >     >     >     >     > >> sho...@gmail.com>> wrote:
    >     >     >     >     >     > >> Hi
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Just to wrap this up. I've tried to 
deploy it on HDP
    >     >     > 2.5
    >     >     >     > but I
    >     >     >     >     > got the
    >     >     >     >     >     > >> exact same problem. What I did was 
removing all
    >     >     > components
    >     >     >     > for
    >     >     >     >     > 2.4.3.0
    >     >     >     >     >     > and
    >     >     >     >     >     > >> then created a new cluster based on 2.5. 
I left it at
    >     >     > that
    >     >     >     > as the
    >     >     >     >     > most
    >     >     >     >     >     > >> likely issue could be with my base 
install. May that
    >     >     > be
    >     >     >     > wrong
    >     >     >     >     > version of
    >     >     >     >     >     > >> something or even a missing dev package.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> As my work also coincided with the 
announcement by
    >     >     > Casey I
    >     >     >     > went
    >     >     >     >     > ahead
    >     >     >     >     >     > and
    >     >     >     >     >     > >> installed 0.3.0 as a dev installation 
instead. That
    >     >     > worked
    >     >     >     > great
    >     >     >     >     > as the
    >     >     >     >     >     > >> dev-full-platform guide is very complete.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> As a general feedback, in order to get 
more people
    >     >     >     > interested and
    >     >     >     >     > not
    >     >     >     >     >     > >> having noobs like me come up with 
redundant questions
    >     >     > - At
    >     >     >     > this
    >     >     >     >     > stage
    >     >     >     >     >     > also
    >     >     >     >     >     > >> sorry for not providing a meaningful 
subject, I
    >     >     > simply was
    >     >     >     > too
    >     >     >     >     > excited
    >     >     >     >     >     > :)
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> There is much to win with a small 
extension of the
    >     >     >     > installation
    >     >     >     >     > guide. I
    >     >     >     >     >     > >> don't think it needs much more prose 
around but a full
    >     >     >     > package
    >     >     >     >     > version
    >     >     >     >     >     > dump
    >     >     >     >     >     > >> with every new release would help 
greatly to get
    >     >     > things
    >     >     >     > going.
    >     >     >     >     > For the
    >     >     >     >     >     > >> things which need to be compiled in, the 
versioning
    >     >     >     > information
    >     >     >     >     > should
    >     >     >     >     >     > be
    >     >     >     >     >     > >> enough. I am saying this, as everyone 
starts with a
    >     >     >     > different base
    >     >     >     >     >     > system.
    >     >     >     >     >     > >> Some have the chance to start from 
scratch, some have
    >     >     > a
    >     >     >     > system
    >     >     >     >     > running
    >     >     >     >     >     > and
    >     >     >     >     >     > >> build something on top and then there 
are the
    >     >     > experienced
    >     >     >     > Metron
    >     >     >     >     > users,
    >     >     >     >     >     > who
    >     >     >     >     >     > >> have many base requirements.. they tend 
to forget.
    >     >     > And I
    >     >     >     > believe
    >     >     >     >     > at the
    >     >     >     >     >     > end
    >     >     >     >     >     > >> it always fails just because of some 
very tiny small
    >     >     >     > difference.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> If there is a space somewhere available, 
this could
    >     >     > even be
    >     >     >     > a
    >     >     >     >     > spot,
    >     >     >     >     >     > where
    >     >     >     >     >     > >> people can say, "hey, I got this running 
on XYZ and
    >     >     > here is
    >     >     >     > the
    >     >     >     >     > package
    >     >     >     >     >     > >> version dump". A simple table with 
'Metron version',
    >     >     > 'OS
    >     >     >     > release',
    >     >     >     >     >     > 'Package
    >     >     >     >     >     > >> list file', 'Version Details of every 
other needed
    >     >     >     > to-be-compiled
    >     >     >     >     >     > package',
    >     >     >     >     >     > >> is already a big help. This really could 
supplement
    >     >     > the
    >     >     >     > existing
    >     >     >     >     > guide
    >     >     >     >     >     > >> found here:
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
https://cwiki.apache.org/confluence/display/METRON/
    >     >     >     > Metron+Installation+on+an+Ambari-Managed+Cluster
    >     >     >     >     >     > >> .
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> But then, I could be alone and others 
did not
    >     >     > experience
    >     >     >     > the same
    >     >     >     >     > and
    >     >     >     >     >     > >> simply got it to run, I don't know. 
Anyway, I would be
    >     >     >     > happy to
    >     >     >     >     > help if
    >     >     >     >     >     > >> help is needed.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Cheers
    >     >     >     >     >     > >> Shoggi
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <
    >     >     > sho...@gmail.com
    >     >     >     > <mailto:
    >     >     >     >     >     > >> sho...@gmail.com>> wrote:
    >     >     >     >     >     > >> Hi Simon
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Thank you for your very quick response. 
I installed
    >     >     > it on
    >     >     >     > HDP
    >     >     >     >     > 2.4.3.0.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> I had a 0.2.0 instance running and was 
very conscious
    >     >     > about
    >     >     >     >     > versioning
    >     >     >     >     >     > the
    >     >     >     >     >     > >> various components. Somehow I had the 
understanding
    >     >     > of not
    >     >     >     >     > touching 2.5
    >     >     >     >     >     > as
    >     >     >     >     >     > >> of yet. I give that a try.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Thanks heaps !
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Shoggi
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> On Sun, Nov 27, 2016 at 2:52 PM, Simon 
Elliston Ball <
    >     >     >     >     >     > >> si...@simonellistonball.com<mailto:simon@
    >     >     >     > simonellistonball.com>>
    >     >     >     >     > wrote:
    >     >     >     >     >     > >> Hi Shoggi,
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Can I ask which version of HDP you used 
as a base. The
    >     >     >     > article is
    >     >     >     >     > a
    >     >     >     >     >     > little
    >     >     >     >     >     > >> out of date in that is calls for 2.4, 
which the 0.3.0
    >     >     >     > release is
    >     >     >     >     > based
    >     >     >     >     >     > on
    >     >     >     >     >     > >> 2.5. There was a major version 
difference in storm
    >     >     > between
    >     >     >     > the
    >     >     >     >     > two,
    >     >     >     >     >     > which
    >     >     >     >     >     > >> seems likely to have caused the problem 
you're seeing.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Simon
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Sent from my iPad
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> On 27 Nov 2016, at 13:48, shoggi 
<sho...@gmail.com
    >     >     > <mailto:
    >     >     >     >     >     > sho...@gmail.com>>
    >     >     >     >     >     > >> wrote:
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Hi all
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> I followed the installation guide to 
deploy Metron via
    >     >     >     > Ambari
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
https://community.hortonworks.com/articles/60805/deploying-
    >     >     >     > a-fresh-metron-cluster-using-ambari-serv.html
    >     >     >     >     >     > >> .
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> I ended up with all but the Metron 
Components
    >     >     > (Enrichment,
    >     >     >     >     > Parsing,
    >     >     >     >     >     > >> Parsers) to start up. The closest to 
pinpoint the
    >     >     > issue is,
    >     >     >     > when
    >     >     >     >     > I try
    >     >     >     >     >     > to
    >     >     >     >     >     > >> start any new parser topology such as 
with bro:
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> 
/usr/metron/0.3.0/bin/start_parser_topology.sh -k
    >     >     >     >     > x.localdomain:6667 -z
    >     >     >     >     >     > >> x.localdomain:2181 -s bro
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> The error I get is:
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Exception in thread "main" java.lang.
    >     >     > BootstrapMethodError:
    >     >     >     >     >     > >> java.lang.NoClassDefFoundError:
    >     >     > org/apache/storm/Config
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Or in more details:
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Running: /usr/jdk64/jdk1.8.0_77/bin/java 
-client
    >     >     >     > -Ddaemon.name=
    >     >     >     >     >     > >> -Dstorm.options= 
-Dstorm.home=/usr/hdp/2.4.3.0-
    >     >     > 227/storm
    >     >     >     >     >     > >> -Dstorm.log.dir=/var/log/storm
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/
    >     >     >     > lib:/usr/hdp/current/storm-client/lib
    >     >     >     >     >     > >> -Dstorm.conf.file= -cp
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
/usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.
    >     >     >     > 
3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > 
storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > clojure-1.6.0.jar:/usr/hdp/
    >     >     >     >     >     > >> 2.4.3.<http://2.4.3
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
.>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > 
storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/
    >     >     >     > 
lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > 
gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-
    >     >     >     > auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
    >     >     >     >     >     > >> 2.4.3.<http://2.4.3
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
.>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
    >     >     >     >     >     > >> 2.4.3.<http://2.4.3
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
.>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/
    >     >     >     > 
usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/
    >     >     >     > hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.
    >     >     >     > 
v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.
    >     >     >     > 
94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
    >     >     >     >     >     > >> 2.4.3.<http://2.4.3
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
.>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > 
storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > 
log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > minlog-1.2.jar:/usr/hdp/
    >     >     >     >     >     > >> 2.4.3.<http://2.4.3
    >     >     >     >     >     >
    >     >     >     >     > 
.>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-
    >     >     >     > 227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-
    >     >     >     > 227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-
    >     >     >     > 227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > 
storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/
    >     >     >     > lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/
    >     >     >     > 
storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > 
tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/
    >     >     >     > 
zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-
    >     >     >     > uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/
    >     >     >     > hdp/2.4.3.0-227/storm/bin
    >     >     >     >     >     > >> -Dstorm.jar=/usr/metron/0.3.0/
    >     >     >     > lib/metron-parsers-0.3.0-uber.jar
    >     >     >     >     >     > >> 
org.apache.metron.parsers.topology.ParserTopologyCLI
    >     >     > -k
    >     >     >     >     >     > x.localdomain:6667
    >     >     >     >     >     > >> -z x.localdomain:2181 -s bro
    >     >     >     >     >     > >> Exception in thread "main" java.lang.
    >     >     > BootstrapMethodError:
    >     >     >     >     >     > >> java.lang.NoClassDefFoundError:
    >     >     > org/apache/storm/Config
    >     >     >     >     >     > >> at
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
org.apache.metron.parsers.topology.ParserTopologyCLI$
    >     >     >     > ParserOptions.<init>(ParserTopologyCLI.java:199)
    >     >     >     >     >     > >> at
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
org.apache.metron.parsers.topology.ParserTopologyCLI$
    >     >     >     > ParserOptions.<clinit>(ParserTopologyCLI.java:47)
    >     >     >     >     >     > >> at
    >     >     >     >     >     > >>
    >     >     >     >     >     >
    >     >     >     >     > 
org.apache.metron.parsers.topology.ParserTopologyCLI.
    >     >     >     > main(ParserTopologyCLI.java:266)
    >     >     >     >     >     > >> Caused by: 
java.lang.NoClassDefFoundError:
    >     >     >     > org/apache/storm/Config
    >     >     >     >     >     > >> ... 3 more
    >     >     >     >     >     > >> Caused by: 
java.lang.ClassNotFoundException:
    >     >     >     >     > org.apache.storm.Config
    >     >     >     >     >     > >> at java.net.URLClassLoader.
    >     >     > findClass(URLClassLoader.java:
    >     >     >     > 381)
    >     >     >     >     >     > >> at 
java.lang.ClassLoader.loadClass(ClassLoader.java:
    >     >     > 424)
    >     >     >     >     >     > >> at 
sun.misc.Launcher$AppClassLoader.loadClass(
    >     >     >     > Launcher.java:331)
    >     >     >     >     >     > >> at 
java.lang.ClassLoader.loadClass(ClassLoader.java:
    >     >     > 357)
    >     >     >     >     >     > >> ... 3 more
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Must be something wrong with my config, 
possibly I
    >     >     > have
    >     >     >     > overlooked
    >     >     >     >     >     > >> something but can't put my finger on it. 
Any help is
    >     >     >     > appreciated.
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Many thanks
    >     >     >     >     >     > >> Shoggi
    >     >     >     >     >     > >>
    >     >     >     >     >     > >>
    >     >     >     >     >     > >>
    >     >     >     >     >     > >>
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> --
    >     >     >     >     >     > >> Nick Allen <n...@nickallen.org<mailto:nic
    >     >     > k...@nickallen.org>>
    >     >     >     >     >     > >> --
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Jon
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Sent from my mobile device
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> --
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Jon
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> Sent from my mobile device
    >     >     >     >     >     > >>
    >     >     >     >     >     > >> --
    >     >     >     >     >     > > Jon
    >     >     >     >     >     > >
    >     >     >     >     >     > > Sent from my mobile device
    >     >     >     >     >     > >
    >     >     >     >     >     >
    >     >     >     >     >     > --
    >     >     >     >     >
    >     >     >     >     >     Jon
    >     >     >     >     >
    >     >     >     >     >     Sent from my mobile device
    >     >     >     >     >
    >     >     >     >     >
    >     >     >     >     >
    >     >     >     >     > --
    >     >     >     >
    >     >     >     >     Jon
    >     >     >     >
    >     >     >     >     Sent from my mobile device
    >     >     >     >
    >     >     >     >
    >     >     >     >
    >     >     >
    >     >     >
    >     >     >
    >     >     
    >     >
    >     >
    >     >
    >     >
    >     >
    >     
    >     
    >     
    >
    >
    >
    
    
    
    



Reply via email to