Ya slackpublisher has some issue and we have fixed it and also changed the
implementation to use rest api instad of sdk.
We will push the patch soon.

Thanks
Jayesh

On Fri, Feb 2, 2018 at 9:02 AM Colm O hEigeartaigh <cohei...@apache.org>
wrote:

> Thanks. I've submitted an initial PR to fix the Slack
> ClassNotFoundException issue here:
>
> https://issues.apache.org/jira/browse/EAGLE-879
> https://github.com/apache/eagle/pull/984
>
> However, no actual Slack messages are sent. The problem is that
> AlertSlackPublisher only sends the message if the event contains a
> "severity" that matches the configured "severity" on the publisher. However
> the event contains neither "severity" (or "message") so the message never
> gets sent. This is for an HDFS audit log - I'm not sure if there are other
> scenarios where there is a "severity" column in the event? Either that or
> it looks like the SlackPublisher was written before the event column
> headers were changed.
>
> Colm.
>
> On Fri, Feb 2, 2018 at 12:54 AM, Jayesh Senjaliya <jay...@apache.org>
> wrote:
>
>> resolved those tickets now.
>>
>> I have asked the developer to rebase the PR #941, if he doesnt get to it
>> by
>> this week, i will take care of, its long pending one.
>> Thanks for verifying though.
>>
>> - Jayesh
>>
>> On Thu, Feb 1, 2018 at 8:56 AM, Colm O hEigeartaigh <cohei...@apache.org>
>> wrote:
>>
>> > Thanks Jayesh. I have two more PRs awaiting review:
>> >
>> > https://github.com/apache/eagle/pull/981
>> > https://github.com/apache/eagle/pull/982
>> >
>> > Thanks for the JIRA privileges, I can now assign issues to me + change
>> the
>> > versions. However, I can't "resolve" JIRAs that weren't reported by me
>> > which is annoying. These 3 JIRAs should be resolved as they are already
>> > merged:
>> >
>> > https://issues.apache.org/jira/browse/EAGLE-445
>> > https://issues.apache.org/jira/browse/EAGLE-476
>> > https://issues.apache.org/jira/browse/EAGLE-331
>> >
>> > In addition, I tested the fix for the Email issue and it works
>> correctly.
>> > The PR (https://github.com/apache/eagle/pull/941) just needs to have
>> the
>> > extra commits stripped away - I attached a version of the patch on the
>> > JIRA.
>> >
>> > Colm.
>> >
>> > On Wed, Jan 31, 2018 at 10:08 PM, Jayesh Senjaliya <jay...@apache.org>
>> > wrote:
>> >
>> > > Thanks for the PRs. I have merged them.
>> > >
>> > > welcome to the developer community Colm. I have also added you to jira
>> > > project so can assign the tasks to yourself.
>> > >
>> > > lets create ticket to fix the dedup functionality, I m actually
>> surprised
>> > > we havent hit this issue yet. we do use multiple publishers but
>> someone
>> > can
>> > > verify this.
>> > >
>> > > Thanks
>> > > Jayesh
>> > >
>> > >
>> > >
>> > > On Wed, Jan 31, 2018 at 9:25 AM, Colm O hEigeartaigh <
>> > cohei...@apache.org>
>> > > wrote:
>> > >
>> > >> Thanks Jayesh. I've created a JIRA here for some admin work for some
>> > >> issues
>> > >> that were incorrectly flagged as "fix for" 0.5.1/0.6.0:
>> > >>
>> > >> https://issues.apache.org/jira/browse/EAGLE-1076
>> > >>
>> > >> I've submitted the following (fairly trivial) pull requests. Could I
>> ask
>> > >> that you or one of the other committers review?
>> > >>
>> > >> https://github.com/apache/eagle/pull/978
>> > >> https://github.com/apache/eagle/pull/979
>> > >> https://github.com/apache/eagle/pull/980
>> > >>
>> > >> It would be good to try to inject some energy into the project. We
>> need
>> > >> more than one active committer though.
>> > >>
>> > >> Just in terms of the Alert Deduplication issue. The
>> DefaultDeDuplicator
>> > >> works per "output" in the policy rule. So if you have more than one
>> > >> AlertPublisher, I think it is guaranteed to only publish to one of
>> them.
>> > >> Instead, surely it would make more sense to work per publisher?
>> > >>
>> > >> Colm.
>> > >>
>> > >> On Tue, Jan 30, 2018 at 10:39 PM, Jayesh Senjaliya <
>> jay...@apache.org>
>> > >> wrote:
>> > >>
>> > >> > Hi Colm,
>> > >> >
>> > >> > appreciate your suggestions/ efforts in looking into this project,
>> > >> > putting my comments inline...
>> > >> >
>> > >> > a) There is already a JIRA to bump the version here, although the
>> PR
>> > >> does
>> > >> > not apply as it is too old: https://issues.apache.org/
>> > >> > jira/browse/EAGLE-1025
>> > >> > .
>> > >> > I can submit a new PR, but should the version be 0.6.0 or 0.5.1?
>> > >> >
>> > >> > *since there are still minor issues, i would say, we put up 0.5.1
>> as
>> > >> next
>> > >> > version. I've updated/rebased the PR (
>> > >> > https://github.com/apache/eagle/pull/936
>> > >> > <https://github.com/apache/eagle/pull/936> )*
>> > >> >
>> > >> >
>> > >> > b) The issues that are "resolved" for the 0.5.1 release in JIRA are
>> > >> > actually already fixed in 0.5.0, so they should be updated (
>> > >> > https://issues.apache.org/jira/projects/EAGLE/versions/12341128).
>> > >> However,
>> > >> > the following two issues are resolved even though they are not
>> merged
>> > to
>> > >> > master?
>> > >> >   https://issues.apache.org/jira/browse/EAGLE-1051 .  - * this was
>> > >> pending
>> > >> > from developer;s response but i think this is reviewed, so I have
>> > merged
>> > >> > it.*
>> > >> >   https://issues.apache.org/jira/browse/EAGLE-1068 .  - * this is
>> > >> reopened
>> > >> > now. I dont think this is done yet. Also this is big change.*
>> > >> >
>> > >> >
>> > >> > Like I said I can submit PRs but I'm not convinced there is any
>> > >> activity on
>> > >> > the project. Where are the rest of the committers?
>> > >> >
>> > >> > *let me give you some ocontext on this. so there were lot of
>> > development
>> > >> > happened during last releases, and most of applications that were
>> > added
>> > >> are
>> > >> > being used in production at multiple enterprise companies, but we
>> are
>> > >> out
>> > >> > of ideas on new apps, so at this point we are only focusing on bug
>> > fixes
>> > >> > and tech upgrades until we get some new ideas to brainstorm and
>> add.*
>> > >> >
>> > >> > *I think current community's thinking is based on their own
>> industries
>> > >> > use-cases, but there is definitely room for new features and
>> > integration
>> > >> > with other monitoring and security components like grafana and
>> > rangers.*
>> > >> >
>> > >> >
>> > >> > *Thanks,*
>> > >> > *Jayesh*
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Tue, Jan 30, 2018 at 8:11 AM, Colm O hEigeartaigh <
>> > >> cohei...@apache.org>
>> > >> > wrote:
>> > >> >
>> > >> > > Hi Jayesh,
>> > >> > >
>> > >> > > Dev suggestions:
>> > >> > >
>> > >> > > a) There is already a JIRA to bump the version here, although
>> the PR
>> > >> does
>> > >> > > not apply as it is too old: https://issues.apache.org/
>> > >> > > jira/browse/EAGLE-1025.
>> > >> > > I can submit a new PR, but should the version be 0.6.0 or 0.5.1?
>> > >> > > b) The issues that are "resolved" for the 0.5.1 release in JIRA
>> are
>> > >> > > actually already fixed in 0.5.0, so they should be updated (
>> > >> > > https://issues.apache.org/jira/projects/EAGLE/versions/12341128
>> ).
>> > >> > However,
>> > >> > > the following two issues are resolved even though they are not
>> > merged
>> > >> to
>> > >> > > master?
>> > >> > >   https://issues.apache.org/jira/browse/EAGLE-1051
>> > >> > >   https://issues.apache.org/jira/browse/EAGLE-1068
>> > >> > >
>> > >> > > Like I said I can submit PRs but I'm not convinced there is any
>> > >> activity
>> > >> > on
>> > >> > > the project. Where are the rest of the committers?
>> > >> > >
>> > >> > > Multiple Publisher issue:
>> > >> > >
>> > >> > > If I assign two publishers for one policy, the alert only goes to
>> > the
>> > >> > first
>> > >> > > policy. In the logs I see:
>> > >> > >
>> > >> > > 2018-01-30T15:52:45.835+0000 o.a.e.a.e.p.d.DefaultDeduplicator
>> > [INFO]
>> > >> > > Alert
>> > >> > > event is skipped because it's duplicated: Alert {site=sandbox,
>> > >> > > stream=eagle_output,timestamp=2018-01-30
>> > >> > > 00:00:11,300,data={securityZone=NA, dst=null, sensitivityType=NA,
>> > >> > > src=/apps/hbase/data/archive/data/default/ambarismoketest,
>> > >> allowed=true,
>> > >> > > host=172.22.7.129, cmd=listStatus, user=SOMETHING7.COM,
>> > >> > > timestamp=1517270411300}, policyId=test,
>> > >> > > createdBy=alertBolt3-evaluator_stage1, metaVersion=null}
>> > >> > >
>> > >> > > It looks like this deduplicator is not working properly, as I'm
>> > >> guessing
>> > >> > it
>> > >> > > should only be used to de-duplicate events for a single
>> publisher?
>> > >> > >
>> > >> > > Incognito mode: Already tried it but with the same result. Could
>> I
>> > ask
>> > >> > you
>> > >> > > to try the docker image to see if the UI is working correctly for
>> > you
>> > >> > > there?
>> > >> > >
>> > >> > > Colm.
>> > >> > >
>> > >> > > On Mon, Jan 29, 2018 at 6:46 PM, Jayesh Senjaliya <
>> > jay...@apache.org>
>> > >> > > wrote:
>> > >> > >
>> > >> > > > Hi Colm,
>> > >> > > >
>> > >> > > > Thanks for the list of dev suggestions, I think we should take
>> > care
>> > >> of
>> > >> > > > those. even better if you can provide PR with the changes or at
>> > >> keast
>> > >> > can
>> > >> > > > you please create a ticket so we can track it?
>> > >> > > >
>> > >> > > > for other issues.
>> > >> > > >
>> > >> > > > - I dont have any issue with multiple publisher, but if there
>> is
>> > any
>> > >> > > error
>> > >> > > > updating the publisher info in storm topology, i might try
>> > >> restarting
>> > >> > the
>> > >> > > > topology and see if that works.
>> > >> > > > - for us, chrome works as fine as firefox.  can u try incognito
>> > >> mode?
>> > >> > > just
>> > >> > > > to be sure to have clean cache?
>> > >> > > >
>> > >> > > > Thanks
>> > >> > > > Jayesh
>> > >> > > >
>> > >> > > >
>> > >> > > > On Thu, Jan 25, 2018 at 4:19 AM, Colm O hEigeartaigh <
>> > >> > > cohei...@apache.org>
>> > >> > > > wrote:
>> > >> > > >
>> > >> > > > > Thanks again for your feedback. Jayesh, adding
>> > >> AlertEagleStorePlugin
>> > >> > > did
>> > >> > > > > the trick, I can now see alerts in the UI, thanks! By the
>> way, I
>> > >> > can't
>> > >> > > > > configure two Alert Publishers, or else the Alert
>> DeDuplicator
>> > >> bins
>> > >> > the
>> > >> > > > > alert. Is this a known issue?
>> > >> > > > >
>> > >> > > > > Could I ask which browser people are using with the UI? There
>> > >> appears
>> > >> > > to
>> > >> > > > be
>> > >> > > > > a  bug with Chromium where it doesn't list the pages under
>> > >> > Auth.isAdmin
>> > >> > > > > even though I am logged on as an administrator. It works OK
>> in
>> > >> > Firefox.
>> > >> > > > > Even with Firefox though, I only see a limited number of
>> links
>> > in
>> > >> the
>> > >> > > > > left-hand column - I can't get back to the "integration"
>> page.
>> > Can
>> > >> > > > someone
>> > >> > > > > else confirm this please?
>> > >> > > > >
>> > >> > > > > Could I suggest the devs do some basic house-keeping tasks:
>> > >> > > > >
>> > >> > > > > a) "Release" version 0.5.0 in JIRA (it's still listed as
>> > >> > "unreleased").
>> > >> > > > > b) Figure out whether the next version will be 0.5.1 or 0.6.0
>> > and
>> > >> > > update
>> > >> > > > > the versions on Master accordingly with 0.5.1-SNAPSHOT or
>> > >> > > 0.6.0-SNAPSHOT.
>> > >> > > > > There are some issues marked here as resolved for 0.5.1 -
>> > >> > > > >
>> https://issues.apache.org/jira/projects/EAGLE/versions/12341128
>> > ),
>> > >> > > > however
>> > >> > > > > I
>> > >> > > > > don't see a branch for 0.5.x?
>> > >> > > > >
>> > >> > > > > Colm.
>> > >> > > > >
>> > >> > > > > On Thu, Jan 25, 2018 at 8:16 AM, Jayesh Senjaliya <
>> > >> jay...@apache.org
>> > >> > >
>> > >> > > > > wrote:
>> > >> > > > >
>> > >> > > > > > Hi,
>> > >> > > > > >
>> > >> > > > > > we do use eagle 0.5 in production although we dont use all
>> the
>> > >> > > > available
>> > >> > > > > > hadoop applications.
>> > >> > > > > >
>> > >> > > > > > EAGLE-968 <https://issues.apache.org/jira/browse/EAGLE-968
>> >
>> > is
>> > >> a
>> > >> > fix
>> > >> > > > for
>> > >> > > > > > email issue we found while our testing. should be merged
>> soon
>> > >> > after a
>> > >> > > > > > rebase.
>> > >> > > > > >
>> > >> > > > > > @Colm, did you tried adding storage publisher
>> > >> > > (AlertEagleStorePlugin)?
>> > >> > > > to
>> > >> > > > > > see alerts on UI ?
>> > >> > > > > >
>> > >> > > > > > Thanks
>> > >> > > > > > Jayesh
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > > > On Wed, Jan 24, 2018 at 7:08 PM, Edward Zhang <
>> > >> > > yonzhang2...@gmail.com>
>> > >> > > > > > wrote:
>> > >> > > > > >
>> > >> > > > > >> Eagle 0.5 was deployed in production as far as I know,
>> but it
>> > >> may
>> > >> > > not
>> > >> > > > be
>> > >> > > > > >> exact the current version in master branch.
>> > >> > > > > >>
>> > >> > > > > >> Thanks for your investigation, seems there is still some
>> bug
>> > in
>> > >> > 0.5,
>> > >> > > > but
>> > >> > > > > >> this particular issue seems is due to dependent components
>> > >> version
>> > >> > > > > conflict.
>> > >> > > > > >>
>> > >> > > > > >> @Jayesh is this Jira ready for merge to master?
>> > >> > > https://issues.apache
>> > >> > > > .
>> > >> > > > > >> org/jira/browse/EAGLE-968
>> > >> > > > > >>
>> > >> > > > > >>
>> > >> > > > > >> Thanks
>> > >> > > > > >> Edward
>> > >> > > > > >>
>> > >> > > > > >> On Tue, Jan 23, 2018 at 5:10 AM, Colm O hEigeartaigh <
>> > >> > > > > cohei...@apache.org
>> > >> > > > > >> > wrote:
>> > >> > > > > >>
>> > >> > > > > >>> OK I've made some more progress. I wasn't seeing any
>> email
>> > >> alerts
>> > >> > > due
>> > >> > > > > to
>> > >> > > > > >>> https://issues.apache.org/jira/browse/EAGLE-968. Once I
>> > >> > configure
>> > >> > > a
>> > >> > > > > >>> Kafka
>> > >> > > > > >>> alert, I can see the alerts flowing into my topic. It's
>> > still
>> > >> not
>> > >> > > > clear
>> > >> > > > > >>> to
>> > >> > > > > >>> me however where the policy "output" is going. I also
>> don't
>> > >> see
>> > >> > any
>> > >> > > > > >>> alerts
>> > >> > > > > >>> in the UI window.
>> > >> > > > > >>>
>> > >> > > > > >>> Could I ask what the status of the project is in general?
>> > >> There
>> > >> > > have
>> > >> > > > > been
>> > >> > > > > >>> no commits to master since November, so I'm not sure if
>> > there
>> > >> is
>> > >> > > any
>> > >> > > > > >>> point
>> > >> > > > > >>> in submitting Pull Requests for outstanding bugs? Are
>> recent
>> > >> > > versions
>> > >> > > > > of
>> > >> > > > > >>> Apache Eagle used in production?
>> > >> > > > > >>>
>> > >> > > > > >>> Colm.
>> > >> > > > > >>>
>> > >> > > > > >>> On Mon, Jan 22, 2018 at 1:07 PM, Colm O hEigeartaigh <
>> > >> > > > > >>> cohei...@apache.org>
>> > >> > > > > >>> wrote:
>> > >> > > > > >>>
>> > >> > > > > >>> >
>> > >> > > > > >>> > I've done that but I'm not seeing any alerts, which is
>> > why I
>> > >> > want
>> > >> > > > to
>> > >> > > > > >>> find
>> > >> > > > > >>> > out what the "output" of a policy is and where I can
>> check
>> > >> > this.
>> > >> > > > > >>> >
>> > >> > > > > >>> > Colm.
>> > >> > > > > >>> >
>> > >> > > > > >>> > On Mon, Jan 22, 2018 at 1:05 PM, SUDHA JENSLIN <
>> > >> > > sjens...@gmail.com
>> > >> > > > >
>> > >> > > > > >>> wrote:
>> > >> > > > > >>> >
>> > >> > > > > >>> >> Create and add a publisher to see the output.
>> > >> > > > > >>> >>
>> > >> > > > > >>> >>
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> Regards,
>> > >> > > > > >>> >> Sudha jenslin
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> On Jan 22, 2018 6:31 PM, "Colm O hEigeartaigh" <
>> > >> > > > cohei...@apache.org
>> > >> > > > > >
>> > >> > > > > >>> >> wrote:
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> Thanks - the error was due to a problem running Storm
>> > with
>> > >> > Java
>> > >> > > > 1.8.
>> > >> > > > > >>> I've
>> > >> > > > > >>> >> abandoned the docker image for now, and I'm trying to
>> get
>> > >> it
>> > >> > > > working
>> > >> > > > > >>> >> locally.
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> There are two things I'm not clear on currently, if
>> > someone
>> > >> > > could
>> > >> > > > > >>> fill me
>> > >> > > > > >>> >> in:
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> a) For the  'Hdfs Audit Log Monitor' application, the
>> > Kafka
>> > >> > > > Consumer
>> > >> > > > > >>> Topic
>> > >> > > > > >>> >> is 'hdfs_audit_log_sandbox'. Under 'Kafka Topic for
>> > >> Auditlog
>> > >> > > Event
>> > >> > > > > >>> Sink'
>> > >> > > > > >>> >> it
>> > >> > > > > >>> >> also specifies 'hdfs_audit_event_sandbox'. However the
>> > >> > > > documentation
>> > >> > > > > >>> for
>> > >> > > > > >>> >> the application mentions
>> 'hdfs_audit_log_enriched_sandb
>> > >> ox'?
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> When I click on "STREAMS", the
>> > >> "HDFS_AUDIT_LOG_ENRICHED_STREA
>> > >> > > > > >>> M_SANDBOX"
>> > >> > > > > >>> >> uses the topic "hdfs_audit_event_sandbox". And indeed
>> > when
>> > >> I
>> > >> > run
>> > >> > > > the
>> > >> > > > > >>> >> application, I can see cleansed log data appearing in
>> > >> > > > > >>> >> "hdfs_audit_event_sandbox". So I'm thinking here that
>> > >> > > > > >>> >> 'hdfs_audit_log_enriched_sandbox' is not correct or
>> > >> > necessary?
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> b) It's unclear to me where the output data goes when
>> you
>> > >> > > create a
>> > >> > > > > >>> policy.
>> > >> > > > > >>> >> E.g. say I have:
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> from HDFS_AUDIT_LOG_ENRICHED_
>> > STREAM_SANDBOX[str:contains(
>> > >> > > src,'/hb
>> > >> > > > > >>> ase')]
>> > >> > > > > >>> >> select * group by user insert into
>> > hdfs_audit_log_enriched_
>> > >> > > > > stream_out
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> Where is "hdfs_audit_log_enriched_stream_out" defined
>> > (is
>> > >> it
>> > >> > a
>> > >> > > > > Kafka
>> > >> > > > > >>> >> topic?). How can I check the output to make sure the
>> > >> policy is
>> > >> > > > > working
>> > >> > > > > >>> >> correctly?
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> Thanks,
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> Colm.
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> On Wed, Jan 17, 2018 at 10:32 PM, Edward Zhang <
>> > >> > > > > >>> yonzhang2...@gmail.com>
>> > >> > > > > >>> >> wrote:
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> > There is a data preparation stage between data
>> > >> source(HDFS
>> > >> > > audit
>> > >> > > > > >>> log)
>> > >> > > > > >>> >> and
>> > >> > > > > >>> >> > Alert Engine. This stage is running in Storm and
>> > >> transform
>> > >> > the
>> > >> > > > raw
>> > >> > > > > >>> HDFS
>> > >> > > > > >>> >> log
>> > >> > > > > >>> >> > into something which can be alerted.
>> > >> > > > > >>> >> >
>> > >> > > > > >>> >> > The input for data preparation is
>> > hdfs_audit_log_sandbox
>> > >> > topic
>> > >> > > > and
>> > >> > > > > >>> >> output
>> > >> > > > > >>> >> > is
>> > >> > > > > >>> >> >  hdfs_audit_log_enriched_sandbox.
>> > >> > > > > >>> >> > The input for Alert Engine is
>> hdfs_audit_log_enriched_
>> > >> > sandbox
>> > >> > > > and
>> > >> > > > > >>> >> output
>> > >> > > > > >>> >> > is
>> > >> > > > > >>> >> > hdfs_audit_log_alert_sandbox.
>> > >> > > > > >>> >> >
>> > >> > > > > >>> >> > Seems in your case, the data preparation staging is
>> not
>> > >> > > working.
>> > >> > > > > We
>> > >> > > > > >>> >> > probably need look at Storm console and figure out
>> if
>> > >> that
>> > >> > > part
>> > >> > > > is
>> > >> > > > > >>> >> working.
>> > >> > > > > >>> >> >
>> > >> > > > > >>> >> > Thanks
>> > >> > > > > >>> >> > Edward
>> > >> > > > > >>> >> >
>> > >> > > > > >>> >> > On Wed, Jan 17, 2018 at 7:19 AM, Colm O
>> hEigeartaigh <
>> > >> > > > > >>> >> cohei...@apache.org>
>> > >> > > > > >>> >> > wrote:
>> > >> > > > > >>> >> >
>> > >> > > > > >>> >> > > Hi Jayesh,
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > Many thanks for your feedback! I was able to make
>> a
>> > >> little
>> > >> > > > > further
>> > >> > > > > >>> >> > headway.
>> > >> > > > > >>> >> > > There are two configuration problems with the
>> > official
>> > >> > > docker
>> > >> > > > > >>> image:
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > a) A mix of "sandbox.eagle.apache.org" and "
>> > >> > > > > >>> server.eagle.apache.org"
>> > >> > > > > >>> >> > (this
>> > >> > > > > >>> >> > > only occurs in the instructions for running the
>> > docker
>> > >> > > image.
>> > >> > > > > The
>> > >> > > > > >>> >> version
>> > >> > > > > >>> >> > > that can be started via the script in the eagle
>> > source
>> > >> is
>> > >> > > OK).
>> > >> > > > > >>> I'll
>> > >> > > > > >>> >> > submit
>> > >> > > > > >>> >> > > a PR to fix this once I get a basic use-case
>> working.
>> > >> > > > > >>> >> > > b) For the audit case, it automatically logs HDFS
>> > audit
>> > >> > logs
>> > >> > > > to
>> > >> > > > > >>> the
>> > >> > > > > >>> >> KAFKA
>> > >> > > > > >>> >> > > topic sandbox_hdfs_audit_log instead of the
>> expected
>> > >> > > > > >>> >> > hdfs_audit_log_sandbox
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > I've fixed these things locally and I can verify
>> that
>> > >> > > > everything
>> > >> > > > > >>> is
>> > >> > > > > >>> >> > started
>> > >> > > > > >>> >> > > correctly in Ambari. I log into the docker
>> container
>> > >> and
>> > >> > > > create
>> > >> > > > > >>> >> > > hdfs_audit_log_sandbox and
>> hdfs_audit_log_enriched_
>> > >> > sandbox
>> > >> > > > > >>> topics,
>> > >> > > > > >>> >> and
>> > >> > > > > >>> >> > > verify that the HDFS audit logs are flowing into
>> the
>> > >> first
>> > >> > > > > topic.
>> > >> > > > > >>> >> Then in
>> > >> > > > > >>> >> > > the UI I start the Alert Engine and then the HDFS
>> > Audit
>> > >> > Log
>> > >> > > > > >>> Monitor
>> > >> > > > > >>> >> > > application (changing localhost:6667 to
>> > >> > > > > >>> server.eagle.apache.org:6667
>> > >> > > > > >>> >> ).
>> > >> > > > > >>> >> > > Both
>> > >> > > > > >>> >> > > applications start up correctly and show
>> "running".
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > I then create a policy with an email alert along
>> the
>> > >> lines
>> > >> > > of
>> > >> > > > > from
>> > >> > > > > >>> >> > > "HDFS_AUDIT_LOG_ENRICHED_
>> > STREAM_SANDBOX[str:contains(
>> > >> > > src,'/h
>> > >> > > > > >>> base')]
>> > >> > > > > >>> >> > select
>> > >> > > > > >>> >> > > * group by user insert into
>> hdfs_audit_log_enriched_
>> > >> > > > > stream_out".
>> > >> > > > > >>> >> However
>> > >> > > > > >>> >> > > at
>> > >> > > > > >>> >> > > this point I'm stuck - nothing appears in the
>> alert
>> > >> > window.
>> > >> > > Is
>> > >> > > > > >>> there
>> > >> > > > > >>> >> > > anything obvious I'm doing wrong, or how can I get
>> > >> access
>> > >> > to
>> > >> > > > > logs
>> > >> > > > > >>> to
>> > >> > > > > >>> >> > figure
>> > >> > > > > >>> >> > > out what the problem is? Other topics such as
>> > >> > > > > >>> >> "hdfs_audit_event_sandbox"
>> > >> > > > > >>> >> > > are mentioned in the streams window, but the
>> > >> documentation
>> > >> > > > > doesn't
>> > >> > > > > >>> >> say to
>> > >> > > > > >>> >> > > create them.
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > The UI is buggy though on both Firefox and
>> Chromium
>> > on
>> > >> > > Linux.
>> > >> > > > > What
>> > >> > > > > >>> >> > > browser/platform are people using with the UI?
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > Colm.
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > On Wed, Jan 17, 2018 at 12:27 AM, Jayesh
>> Senjaliya <
>> > >> > > > > >>> jay...@apache.org
>> > >> > > > > >>> >> >
>> > >> > > > > >>> >> > > wrote:
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > > Hi Colm,
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > > Please find my comments inline.
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > > a) The official docker image uses 0.5.0-SNAPSHOT
>> > and
>> > >> not
>> > >> > > the
>> > >> > > > > >>> >> released
>> > >> > > > > >>> >> > > > version.
>> > >> > > > > >>> >> > > > - this is because we uploaded docker image
>> before
>> > >> apache
>> > >> > > > > >>> release.
>> > >> > > > > >>> >> > > actually
>> > >> > > > > >>> >> > > > this is same codebase apache-eagle-0.5, and it
>> can
>> > be
>> > >> > > fixed
>> > >> > > > > >>> easily
>> > >> > > > > >>> >> by
>> > >> > > > > >>> >> > > just
>> > >> > > > > >>> >> > > > rebuilding docker image. there should not be any
>> > >> > mismatch
>> > >> > > > due
>> > >> > > > > to
>> > >> > > > > >>> >> this.
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > > b) Aside from the above, the official docker
>> image
>> > >> uses
>> > >> > a
>> > >> > > > mix
>> > >> > > > > >>> of "
>> > >> > > > > >>> >> > > > server.eagle.apache.org" and "
>> > >> sandbox.eagle.apache.org"
>> > >> > > as
>> > >> > > > > the
>> > >> > > > > >>> host
>> > >> > > > > >>> >> > > name.
>> > >> > > > > >>> >> > > > The HBase service doesn't start by default in
>> > Ambari
>> > >> as
>> > >> > a
>> > >> > > > > >>> result.
>> > >> > > > > >>> >> > > > - the only places it uses sandbox is in example
>> > >> script
>> > >> > > which
>> > >> > > > > you
>> > >> > > > > >>> >> will
>> > >> > > > > >>> >> > > have
>> > >> > > > > >>> >> > > > to update anyway, which i agree that it would be
>> > >> good to
>> > >> > > > keep
>> > >> > > > > it
>> > >> > > > > >>> >> > > > consistent.
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > > c) The UI seems quite buggy. On both chromium
>> and
>> > >> > > firefox, I
>> > >> > > > > >>> only
>> > >> > > > > >>> >> see
>> > >> > > > > >>> >> > > > links to "Sandbox" and "Alert" on the left
>> > hand-side.
>> > >> > > Once I
>> > >> > > > > >>> click
>> > >> > > > > >>> >> on
>> > >> > > > > >>> >> > > > "Alert" I have no way of going back to see the
>> > >> > > > applications. I
>> > >> > > > > >>> don't
>> > >> > > > > >>> >> > see
>> > >> > > > > >>> >> > > > the links to "integration" or "sites" as in the
>> > >> picture
>> > >> > > > here:
>> > >> > > > > >>> >> > > > http://eagle.apache.org/docs/l
>> > >> atest/applications/#jmx-
>> > >> > > > monito
>> > >> > > > > >>> ring
>> > >> > > > > >>> >> > > > - when hbase is as deep storage is used, and if
>> > eagle
>> > >> > app
>> > >> > > > has
>> > >> > > > > >>> issue
>> > >> > > > > >>> >> > > > connecting to hbase, the UI becomes
>> unresponsive.
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > > d) In chromium, the button to create a new
>> policy
>> > >> does
>> > >> > not
>> > >> > > > > >>> exist - I
>> > >> > > > > >>> >> > can
>> > >> > > > > >>> >> > > > only see it on Firefox.
>> > >> > > > > >>> >> > > > - i have seen when you logged in, you will see
>> > admin
>> > >> > > > actions.
>> > >> > > > > >>> but if
>> > >> > > > > >>> >> > this
>> > >> > > > > >>> >> > > > still an issue, can you please file UI bug?
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > > e) I'm trying to get the "Hdfs Audit Log
>> Monitor"
>> > >> > use-case
>> > >> > > > > >>> working,
>> > >> > > > > >>> >> but
>> > >> > > > > >>> >> > > it
>> > >> > > > > >>> >> > > > seems to be stuck in "Initialized".
>> > >> > > > > >>> >> > > > this eagle docs has example on how to setup the
>> > app.
>> > >> pls
>> > >> > > let
>> > >> > > > > us
>> > >> > > > > >>> >> know if
>> > >> > > > > >>> >> > > > you find any gaps.
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > > Thanks for trying out, and sharing your
>> findings,
>> > >> > > > > >>> >> > > > Jayesh
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > > On Tue, Jan 16, 2018 at 3:34 AM, Colm O
>> > hEigeartaigh
>> > >> <
>> > >> > > > > >>> >> > > cohei...@apache.org>
>> > >> > > > > >>> >> > > > wrote:
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > >> Hi all,
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> I'm trying to play around a bit with Apache
>> Eagle
>> > >> 0.5.0
>> > >> > > to
>> > >> > > > no
>> > >> > > > > >>> >> avail.
>> > >> > > > > >>> >> > > Here
>> > >> > > > > >>> >> > > >> are the problems I've run into so far:
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> a) The official docker image uses
>> 0.5.0-SNAPSHOT
>> > and
>> > >> > not
>> > >> > > > the
>> > >> > > > > >>> >> released
>> > >> > > > > >>> >> > > >> version.
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> b) Aside from the above, the official docker
>> image
>> > >> > uses a
>> > >> > > > mix
>> > >> > > > > >>> of "
>> > >> > > > > >>> >> > > >> server.eagle.apache.org" and "
>> > >> sandbox.eagle.apache.org
>> > >> > "
>> > >> > > as
>> > >> > > > > the
>> > >> > > > > >>> >> host
>> > >> > > > > >>> >> > > >> name. The HBase service doesn't start by
>> default
>> > in
>> > >> > > Ambari
>> > >> > > > > as a
>> > >> > > > > >>> >> > result.
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> c) The UI seems quite buggy. On both chromium
>> and
>> > >> > > firefox,
>> > >> > > > I
>> > >> > > > > >>> only
>> > >> > > > > >>> >> see
>> > >> > > > > >>> >> > > >> links to "Sandbox" and "Alert" on the left
>> > >> hand-side.
>> > >> > > Once
>> > >> > > > I
>> > >> > > > > >>> click
>> > >> > > > > >>> >> on
>> > >> > > > > >>> >> > > >> "Alert" I have no way of going back to see the
>> > >> > > > applications.
>> > >> > > > > I
>> > >> > > > > >>> >> don't
>> > >> > > > > >>> >> > see
>> > >> > > > > >>> >> > > >> the links to "integration" or "sites" as in the
>> > >> picture
>> > >> > > > here:
>> > >> > > > > >>> >> > > >> http://eagle.apache.org/docs/l
>> > >> atest/applications/#jmx-
>> > >> > > > monito
>> > >> > > > > >>> ring
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> d) In chromium, the button to create a new
>> policy
>> > >> does
>> > >> > > not
>> > >> > > > > >>> exist -
>> > >> > > > > >>> >> I
>> > >> > > > > >>> >> > can
>> > >> > > > > >>> >> > > >> only see it on Firefox.
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> e) I'm trying to get the "Hdfs Audit Log
>> Monitor"
>> > >> > > use-case
>> > >> > > > > >>> working,
>> > >> > > > > >>> >> > but
>> > >> > > > > >>> >> > > >> it seems to be stuck in "Initialized".
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> Could someone fill me in on what the
>> "recommended"
>> > >> way
>> > >> > is
>> > >> > > > to
>> > >> > > > > >>> start
>> > >> > > > > >>> >> > > Apache
>> > >> > > > > >>> >> > > >> Eagle so that I can play around with the
>> > >> functionality
>> > >> > > that
>> > >> > > > > it
>> > >> > > > > >>> >> offers?
>> > >> > > > > >>> >> > > >> Clearly the docker approach is buggy. Also,
>> what
>> > >> > browser
>> > >> > > > > >>> should be
>> > >> > > > > >>> >> > used?
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> Thanks,
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> Colm.
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> --
>> > >> > > > > >>> >> > > >> Colm O hEigeartaigh
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >> Talend Community Coder
>> > >> > > > > >>> >> > > >> http://coders.talend.com
>> > >> > > > > >>> >> > > >>
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > > >
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > --
>> > >> > > > > >>> >> > > Colm O hEigeartaigh
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> > > Talend Community Coder
>> > >> > > > > >>> >> > > http://coders.talend.com
>> > >> > > > > >>> >> > >
>> > >> > > > > >>> >> >
>> > >> > > > > >>> >>
>> > >> > > > > >>> >>
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> --
>> > >> > > > > >>> >> Colm O hEigeartaigh
>> > >> > > > > >>> >>
>> > >> > > > > >>> >> Talend Community Coder
>> > >> > > > > >>> >> http://coders.talend.com
>> > >> > > > > >>> >>
>> > >> > > > > >>> >>
>> > >> > > > > >>> >>
>> > >> > > > > >>> >
>> > >> > > > > >>> >
>> > >> > > > > >>> > --
>> > >> > > > > >>> > Colm O hEigeartaigh
>> > >> > > > > >>> >
>> > >> > > > > >>> > Talend Community Coder
>> > >> > > > > >>> > http://coders.talend.com
>> > >> > > > > >>> >
>> > >> > > > > >>>
>> > >> > > > > >>>
>> > >> > > > > >>>
>> > >> > > > > >>> --
>> > >> > > > > >>> Colm O hEigeartaigh
>> > >> > > > > >>>
>> > >> > > > > >>> Talend Community Coder
>> > >> > > > > >>> http://coders.talend.com
>> > >> > > > > >>>
>> > >> > > > > >>
>> > >> > > > > >>
>> > >> > > > > >
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > --
>> > >> > > > > Colm O hEigeartaigh
>> > >> > > > >
>> > >> > > > > Talend Community Coder
>> > >> > > > > http://coders.talend.com
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > --
>> > >> > > Colm O hEigeartaigh
>> > >> > >
>> > >> > > Talend Community Coder
>> > >> > > http://coders.talend.com
>> > >> > >
>> > >> >
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Colm O hEigeartaigh
>> > >>
>> > >> Talend Community Coder
>> > >> http://coders.talend.com
>> > >>
>> > >
>> > >
>> >
>> >
>> > --
>> > Colm O hEigeartaigh
>> >
>> > Talend Community Coder
>> > http://coders.talend.com
>> >
>>
>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>

Reply via email to