Re: [Discuss] How do you think rename "develop" branch to "trunk"branch so that able to close pull request automatically?

[Don Bosco Durai]

I also feel that moving development to “trunk” will be a good thing. Right now, 
synchronizing the final release branch to the “trunk” seems to be a redundant 
activity. In the release notes, we can always ask the users to use the release 
branch and also when we create sub releases, they would be off the previous 
release branch, so it would just work naturally.

Bosco



On 8/7/16, 7:12 PM, "Michael Wu"  wrote:

Sounds good to me, as long as it benefits the project.

On Fri, Aug 5, 2016 at 3:22 PM, Hao Chen  wrote:

> Currently we are faced a problem that we can't close the pull request from
> contributors automatically.
>
> When a pull request is merged in develop branch, github could not close 
the
> pull request automatically and committers don't have permission to close
> the pull request manually on github page, so that we have to ask the
> contributor to close the branch manually otherwise there would be lots of
> "OPEN" pull requests listed though most are merged.
>
> Learning from github service:
>
> "
> To close this pull request, make a commit to your *master/trunk *branch
> with (at least) the following in the commit message:
>
> This closes #305
> "
>
> Commits with *"Closes #PULL-REQUEST-ID"* will only work on master/trunk, 
in
> fact *trunk* branch should be our *develo *in purpose and would be better
> for management.
>
> Any comments are appreciated.
>
> - Hao
>

[jira] [Resolved] (EAGLE-415) Compilation failure in develop branch - Copyright missing

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/EAGLE-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai resolved EAGLE-415.
---
Resolution: Fixed

Pull request created https://github.com/apache/incubator-eagle/pull/302

> Compilation failure in develop branch - Copyright missing
> -
>
> Key: EAGLE-415
> URL: https://issues.apache.org/jira/browse/EAGLE-415
> Project: Eagle
>  Issue Type: Bug
>    Reporter: Don Bosco Durai
>    Assignee: Don Bosco Durai
>
> Apache copyright is missing in file 
> eagle-examples/eagle-app-example/src/main/java/org/apache/eagle/app/example/ExampleStormConfig.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-415) Compilation failure in develop branch - Copyright missing

[Don Bosco Durai (JIRA)]

Don Bosco Durai created EAGLE-415:
-

 Summary: Compilation failure in develop branch - Copyright missing
 Key: EAGLE-415
 URL: https://issues.apache.org/jira/browse/EAGLE-415
 Project: Eagle
  Issue Type: Bug
Reporter: Don Bosco Durai
Assignee: Don Bosco Durai


Apache copyright is missing in file 
eagle-examples/eagle-app-example/src/main/java/org/apache/eagle/app/example/ExampleStormConfig.java




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (EAGLE-400) Compile errors in develop branch

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397313#comment-15397313
 ] 

Don Bosco Durai commented on EAGLE-400:
---

Created pull request to fix this issue: 
https://github.com/apache/incubator-eagle/pull/282


> Compile errors in develop branch
> 
>
> Key: EAGLE-400
> URL: https://issues.apache.org/jira/browse/EAGLE-400
> Project: Eagle
>  Issue Type: Bug
>    Reporter: Don Bosco Durai
>    Assignee: Don Bosco Durai
>
> Facing below 2 compile errors in develop branch
> [INFO] -
> [ERROR] COMPILATION ERROR : 
> [INFO] -
> [ERROR] 
> /incubator-eagle/eagle-core/eagle-metadata/eagle-metadata-base/src/main/java/org/apache/eagle/metadata/resource/RestResponse.java:[29,7]
>  error: class RESTResponse is public, should be declared in a file named 
> RESTResponse.java
> [INFO] 1 error
> [ERROR] COMPILATION ERROR : 
> [INFO] -
> [ERROR] 
> /incubator-eagle/eagle-core/eagle-app/eagle-app-base/src/main/java/org/apache/eagle/app/spi/AbstractApplicationProvider.java:[50,1]
>  error:  expected
> [INFO] 1 error



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (EAGLE-400) Compile errors in develop branch

[Don Bosco Durai (JIRA)]

Don Bosco Durai created EAGLE-400:
-

 Summary: Compile errors in develop branch
 Key: EAGLE-400
 URL: https://issues.apache.org/jira/browse/EAGLE-400
 Project: Eagle
  Issue Type: Bug
Reporter: Don Bosco Durai
Assignee: Don Bosco Durai


Facing below 2 compile errors in develop branch

[INFO] -
[ERROR] COMPILATION ERROR : 
[INFO] -
[ERROR] 
/incubator-eagle/eagle-core/eagle-metadata/eagle-metadata-base/src/main/java/org/apache/eagle/metadata/resource/RestResponse.java:[29,7]
 error: class RESTResponse is public, should be declared in a file named 
RESTResponse.java
[INFO] 1 error


[ERROR] COMPILATION ERROR : 
[INFO] -
[ERROR] 
/incubator-eagle/eagle-core/eagle-app/eagle-app-base/src/main/java/org/apache/eagle/app/spi/AbstractApplicationProvider.java:[50,1]
 error:  expected
[INFO] 1 error




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Eagle build broken in github

[Don Bosco Durai]

I had the same issue with OpenJDK 1.8. But it works with Sun JDK 1.8.

Bosco


On 7/25/16, 5:37 PM, "Edward Zhang"  wrote:

Seems it is JDK issue?

[INFO] Compiling 16 source files to

/home/jenkins/jenkins-slave/workspace/incubator-eagle-main/eagle-core/eagle-data-process/eagle-stream-process-base/target/classes
at 1469439473203[ERROR] error: error while loading CharSequence, class
file 
'/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/jre/lib/rt.jar(java/lang/CharSequence.class)'
is broken[INFO] (class java.lang.RuntimeException/bad constant pool
tag 18 at byte 10)[ERROR] error: error while loading AnnotatedElement,
class file 
'/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/jre/lib/rt.jar(java/lang/reflect/AnnotatedElement.class)'
is broken[INFO] (class java.lang.RuntimeException/bad constant pool
tag 18 at byte 76)[ERROR] error: error while loading ConcurrentMap,
class file 
'/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/jre/lib/rt.jar(java/util/concurrent/ConcurrentMap.class)'
is broken[INFO] (class java.lang.RuntimeException/bad constant pool
tag 18 at byte 61)[ERROR] three errors found

Re: Build failure in branch "develop"

[Don Bosco Durai]

I was mostly referring to a Docker container just for building. I know you have 
Docker for runtime. I have not tried that yet.

Thanks

Bosco


On 7/25/16, 12:25 PM, "Edward Zhang" <yonzhang2...@gmail.com> wrote:

Thanks. Are you saying that you will build docker file for community to use?
As far as I know current docker image is not fully tested and sometimes may
not be stable as it contains HBase which is often down unexpectedly.

Thanks
Edward

On Mon, Jul 25, 2016 at 12:21 PM, Don Bosco Durai <bo...@apache.org> wrote:

> Thanks. This worked. Additionally, I had to install node.js. Let me see if
> I can make the Dockerfile available in some form for others to use it (if
> needed).
>
> Bosco
>
>
> On 7/25/16, 1:28 AM, "Michael Wu" <mchl@gmail.com> wrote:
>
> This issue has been fixed in develop branch.
>
> Michael
>
> On Mon, Jul 25, 2016 at 3:07 PM, Zhao, Qingwen <qingwz...@ebay.com>
> wrote:
>
> > OK.
> >
> > Seems there are some problems in this new branch, which is in the
> > development and unstable right now. Sorry about that.
> >
> > If you are trying Eagle, please use the newly released version
> branch 0.4.
> > If you want to contribute new features, develop branch is ok.
> >
> > For this issue, we are fixing :)
> >
> > Regards,
> > Qingwen Zhao | Apache Eagle
> > Desk: 20998167 | Cell: +86 18818215025
> >
> >
> >
> >
> >
> >
> > On 7/25/16, 2:31 PM, "Don Bosco Durai" <bo...@apache.org> wrote:
> >
> > >Yes, I am running mvn from the root folder of the project.
> > >
> > >Bosco
> > >
> > >
> > >On 7/24/16, 11:29 PM, "Michael Wu" <mchl@gmail.com> wrote:
> > >
> > >Just to make sure, are you executing from the project root
> folder? If
> > >    you're executing the mvn command from within a sub folder, it
> may
> > >require
> > >that you've installed depended artifacts to your local maven
> > >repository in
> > >previous builds.
> > >
> > >Michael
> > >
> > >On Mon, Jul 25, 2016 at 1:46 PM, Don Bosco Durai <
> bo...@apache.org>
> > >wrote:
> > >
> > >> Still same issue on my mac and fresh docker image with Sun
> JDK 1.8
> > >>
> > >> [root@500777b9b778 incubator-eagle]# git branch
> > >> * develop
> > >>
> > >> [root@500777b9b778 incubator-eagle]# $JAVA_HOME/bin/java
> -version
> > >> java version "1.8.0_101"
> > >> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> > >> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed
> mode)
> > >>
> > >> [root@500777b9b778 incubator-eagle]# mvn clean
> > >> [INFO] Scanning for projects...
> > >> [ERROR] [ERROR] Some problems were encountered while
> processing the
> > >POMs:
> > >> [WARNING] 'parent.relativePath' of POM
> > >> org.apache.eagle:eagle-common:[unknown-version]
> > >> (/work/incubator-eagle/eagle-core/eagle-common/pom.xml)
> points at
> > >> org.apache.eagle:eagle-core instead of
> > >org.apache.eagle:eagle-query-parent,
> > >> please verify your project structure @ line 22, column 10
> > >> [FATAL] Non-resolvable parent POM for
> > >> org.apache.eagle:eagle-common:[unknown-version]: Could not
> find
> > >artifact
> > >>
> org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNAPSHOT
> > >and
> > >> 'parent.relativePath' points at wrong local POM @ line 22,
> column 10
> > >> [WARNING]
> > >'dependencies.dependency.(groupId:artifactId:type:classifier)'
> > >> must be unique: joda-time:joda-time:jar -> duplicate

Re: Build failure in branch "develop"

[Don Bosco Durai]

Thanks. This worked. Additionally, I had to install node.js. Let me see if I 
can make the Dockerfile available in some form for others to use it (if needed).

Bosco


On 7/25/16, 1:28 AM, "Michael Wu" <mchl@gmail.com> wrote:

This issue has been fixed in develop branch.

Michael

On Mon, Jul 25, 2016 at 3:07 PM, Zhao, Qingwen <qingwz...@ebay.com> wrote:

> OK.
>
> Seems there are some problems in this new branch, which is in the
> development and unstable right now. Sorry about that.
>
> If you are trying Eagle, please use the newly released version branch 0.4.
> If you want to contribute new features, develop branch is ok.
>
> For this issue, we are fixing :)
>
> Regards,
> Qingwen Zhao | Apache Eagle
> Desk: 20998167 | Cell: +86 18818215025
    >
    >
>
>
>
>
> On 7/25/16, 2:31 PM, "Don Bosco Durai" <bo...@apache.org> wrote:
>
> >Yes, I am running mvn from the root folder of the project.
> >
> >Bosco
> >
> >
> >On 7/24/16, 11:29 PM, "Michael Wu" <mchl@gmail.com> wrote:
> >
> >Just to make sure, are you executing from the project root folder? If
> >you're executing the mvn command from within a sub folder, it may
> >require
> >that you've installed depended artifacts to your local maven
> >repository in
> >previous builds.
> >
> >Michael
> >
> >On Mon, Jul 25, 2016 at 1:46 PM, Don Bosco Durai <bo...@apache.org>
> >wrote:
> >
> >> Still same issue on my mac and fresh docker image with Sun JDK 1.8
> >>
> >> [root@500777b9b778 incubator-eagle]# git branch
> >> * develop
> >>
> >> [root@500777b9b778 incubator-eagle]# $JAVA_HOME/bin/java -version
> >> java version "1.8.0_101"
> >> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> >> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
> >>
> >> [root@500777b9b778 incubator-eagle]# mvn clean
> >> [INFO] Scanning for projects...
> >> [ERROR] [ERROR] Some problems were encountered while processing the
> >POMs:
> >> [WARNING] 'parent.relativePath' of POM
> >> org.apache.eagle:eagle-common:[unknown-version]
> >> (/work/incubator-eagle/eagle-core/eagle-common/pom.xml) points at
> >> org.apache.eagle:eagle-core instead of
> >org.apache.eagle:eagle-query-parent,
> >> please verify your project structure @ line 22, column 10
> >> [FATAL] Non-resolvable parent POM for
> >> org.apache.eagle:eagle-common:[unknown-version]: Could not find
> >artifact
> >> org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNAPSHOT
> >and
> >> 'parent.relativePath' points at wrong local POM @ line 22, column 
10
> >> [WARNING]
> >'dependencies.dependency.(groupId:artifactId:type:classifier)'
> >> must be unique: joda-time:joda-time:jar -> duplicate declaration of
> >version
> >> (?) @ org.apache.eagle:alert-engine:[unknown-version],
> >>
> 
>/work/incubator-eagle/eagle-core/eagle-alert-parent/eagle-alert/alert-engi
> >ne/pom.xml,
> >> line 115, column 15
> >> [WARNING]
> >'dependencies.dependency.(groupId:artifactId:type:classifier)'
> >> must be unique: org.apache.kafka:kafka-clients:jar -> duplicate
> >declaration
> >> of version (?) @ org.apache.eagle:alert-engine:[unknown-version],
> >>
> 
>/work/incubator-eagle/eagle-core/eagle-alert-parent/eagle-alert/alert-engi
> >ne/pom.xml,
> >> line 127, column 15
> >> [WARNING]
> >'dependencies.dependency.(groupId:artifactId:type:classifier)'
> >> must be unique: org.apache.eagle:eagle-metadata-base:jar -> version
> >> ${project.version} vs 0.5.0-incubating-SNAPSHOT @
> >> org.apache.eagle:eagle-app-base:[unknown-version],
> >> /work/incubator-eagle/eagle-core/eagle-app/eagle-app-base/pom.xml,
> >line 95,
> >> column 21
> >> [WARNING]
> >'dependencies.dependency.(groupId:artifactId:type:classifier)'
> >> must be unique: junit:junit:jar -> dup

Re: Build failure in branch "develop"

[Don Bosco Durai]

Yes, I am running mvn from the root folder of the project.

Bosco


On 7/24/16, 11:29 PM, "Michael Wu" <mchl@gmail.com> wrote:

Just to make sure, are you executing from the project root folder? If
you're executing the mvn command from within a sub folder, it may require
that you've installed depended artifacts to your local maven repository in
previous builds.

Michael

On Mon, Jul 25, 2016 at 1:46 PM, Don Bosco Durai <bo...@apache.org> wrote:

> Still same issue on my mac and fresh docker image with Sun JDK 1.8
>
> [root@500777b9b778 incubator-eagle]# git branch
> * develop
>
> [root@500777b9b778 incubator-eagle]# $JAVA_HOME/bin/java -version
> java version "1.8.0_101"
> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>
> [root@500777b9b778 incubator-eagle]# mvn clean
> [INFO] Scanning for projects...
> [ERROR] [ERROR] Some problems were encountered while processing the POMs:
> [WARNING] 'parent.relativePath' of POM
> org.apache.eagle:eagle-common:[unknown-version]
> (/work/incubator-eagle/eagle-core/eagle-common/pom.xml) points at
> org.apache.eagle:eagle-core instead of 
org.apache.eagle:eagle-query-parent,
> please verify your project structure @ line 22, column 10
> [FATAL] Non-resolvable parent POM for
> org.apache.eagle:eagle-common:[unknown-version]: Could not find artifact
> org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNAPSHOT and
> 'parent.relativePath' points at wrong local POM @ line 22, column 10
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)'
> must be unique: joda-time:joda-time:jar -> duplicate declaration of 
version
> (?) @ org.apache.eagle:alert-engine:[unknown-version],
> 
/work/incubator-eagle/eagle-core/eagle-alert-parent/eagle-alert/alert-engine/pom.xml,
> line 115, column 15
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)'
> must be unique: org.apache.kafka:kafka-clients:jar -> duplicate 
declaration
> of version (?) @ org.apache.eagle:alert-engine:[unknown-version],
> 
/work/incubator-eagle/eagle-core/eagle-alert-parent/eagle-alert/alert-engine/pom.xml,
> line 127, column 15
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)'
> must be unique: org.apache.eagle:eagle-metadata-base:jar -> version
> ${project.version} vs 0.5.0-incubating-SNAPSHOT @
> org.apache.eagle:eagle-app-base:[unknown-version],
> /work/incubator-eagle/eagle-core/eagle-app/eagle-app-base/pom.xml, line 
95,
> column 21
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)'
> must be unique: junit:junit:jar -> duplicate declaration of version (?) @
> org.apache.eagle:eagle-app-base:[unknown-version],
> /work/incubator-eagle/eagle-core/eagle-app/eagle-app-base/pom.xml, line
> 100, column 21
> [WARNING] 'build.plugins.plugin.version' for
> org.codehaus.mojo:exec-maven-plugin is missing. @
> org.apache.eagle:eagle-webservice:[unknown-version],
> /work/incubator-eagle/eagle-webservice/pom.xml, line 400, column 12
>  @
> [ERROR] The build could not read 1 project -> [Help 1]
> [ERROR]
> [ERROR]   The project org.apache.eagle:eagle-common:[unknown-version]
> (/work/incubator-eagle/eagle-core/eagle-common/pom.xml) has 1 error
> [ERROR] Non-resolvable parent POM for
> org.apache.eagle:eagle-common:[unknown-version]: Could not find artifact
> org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNAPSHOT and
> 'parent.relativePath' points at wrong local POM @ line 22, column 10 ->
> [Help 2]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
> [ERROR] [Help 2]
> 
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
>
> Bosco
>
>
> On 7/24/16, 7:29 PM, "Zhao, Qingwen" <qingwz...@ebay.com> wrote:
>
> Just JDK8 is required in the develop branch.
>
> I built this branch in my local env (OS X), and it went through well.
>
> Regards,
> Qingwen Zhao | Apache Eagle
>
>
>
> On 7/25/16, 9:

Re: Build failure in branch "develop"

[Don Bosco Durai]

Still same issue on my mac and fresh docker image with Sun JDK 1.8

[root@500777b9b778 incubator-eagle]# git branch
* develop

[root@500777b9b778 incubator-eagle]# $JAVA_HOME/bin/java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

[root@500777b9b778 incubator-eagle]# mvn clean
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[WARNING] 'parent.relativePath' of POM 
org.apache.eagle:eagle-common:[unknown-version] 
(/work/incubator-eagle/eagle-core/eagle-common/pom.xml) points at 
org.apache.eagle:eagle-core instead of org.apache.eagle:eagle-query-parent, 
please verify your project structure @ line 22, column 10
[FATAL] Non-resolvable parent POM for 
org.apache.eagle:eagle-common:[unknown-version]: Could not find artifact 
org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNAPSHOT and 
'parent.relativePath' points at wrong local POM @ line 22, column 10
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: joda-time:joda-time:jar -> duplicate declaration of version (?) @ 
org.apache.eagle:alert-engine:[unknown-version], 
/work/incubator-eagle/eagle-core/eagle-alert-parent/eagle-alert/alert-engine/pom.xml,
 line 115, column 15
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: org.apache.kafka:kafka-clients:jar -> duplicate declaration of 
version (?) @ org.apache.eagle:alert-engine:[unknown-version], 
/work/incubator-eagle/eagle-core/eagle-alert-parent/eagle-alert/alert-engine/pom.xml,
 line 127, column 15
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: org.apache.eagle:eagle-metadata-base:jar -> version 
${project.version} vs 0.5.0-incubating-SNAPSHOT @ 
org.apache.eagle:eagle-app-base:[unknown-version], 
/work/incubator-eagle/eagle-core/eagle-app/eagle-app-base/pom.xml, line 95, 
column 21
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: junit:junit:jar -> duplicate declaration of version (?) @ 
org.apache.eagle:eagle-app-base:[unknown-version], 
/work/incubator-eagle/eagle-core/eagle-app/eagle-app-base/pom.xml, line 100, 
column 21
[WARNING] 'build.plugins.plugin.version' for 
org.codehaus.mojo:exec-maven-plugin is missing. @ 
org.apache.eagle:eagle-webservice:[unknown-version], 
/work/incubator-eagle/eagle-webservice/pom.xml, line 400, column 12
 @
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project org.apache.eagle:eagle-common:[unknown-version] 
(/work/incubator-eagle/eagle-core/eagle-common/pom.xml) has 1 error
[ERROR] Non-resolvable parent POM for 
org.apache.eagle:eagle-common:[unknown-version]: Could not find artifact 
org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNAPSHOT and 
'parent.relativePath' points at wrong local POM @ line 22, column 10 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] 
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

Bosco


On 7/24/16, 7:29 PM, "Zhao, Qingwen" <qingwz...@ebay.com> wrote:

Just JDK8 is required in the develop branch.

I built this branch in my local env (OS X), and it went through well.

Regards,
Qingwen Zhao | Apache Eagle
    
    
    
On 7/25/16, 9:35 AM, "Don Bosco Durai" <bo...@apache.org> wrote:

>I am getting the below error when I build on ³develop² branch. However,
>on the same machine and env, the build on the release bits went through.
>
> 
>
>Is there any other additional steps to build off ³develop² branch?
>
> 
>
>Thanks
>
> 
>
>Bosco
>
> 
>
> 
>
>$ mvn -DskipTests clean package
>
>[INFO] Scanning for projects...
>
>[ERROR] [ERROR] Some problems were encountered while processing the POMs:
>
>[WARNING] 'parent.relativePath' of POM
>org.apache.eagle:eagle-common:[unknown-version]
>(/home/build/git/incubator-eagle/eagle-core/eagle-common/pom.xml) points
>at org.apache.eagle:eagle-core instead of
>org.apache.eagle:eagle-query-parent, please verify your project structure
>@ line 22, column 10
>
>[FATAL] Non-resolvable parent POM for
>org.apache.eagle:eagle-common:[unknown-version]: Could not find artifact
>org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNA

Build failure in branch "develop"

[Don Bosco Durai]

I am getting the below error when I build on “develop” branch. However, on the 
same machine and env, the build on the release bits went through.

 

Is there any other additional steps to build off “develop” branch?

 

Thanks

 

Bosco

 

 

$ mvn -DskipTests clean package

[INFO] Scanning for projects...

[ERROR] [ERROR] Some problems were encountered while processing the POMs:

[WARNING] 'parent.relativePath' of POM 
org.apache.eagle:eagle-common:[unknown-version] 
(/home/build/git/incubator-eagle/eagle-core/eagle-common/pom.xml) points at 
org.apache.eagle:eagle-core instead of org.apache.eagle:eagle-query-parent, 
please verify your project structure @ line 22, column 10

[FATAL] Non-resolvable parent POM for 
org.apache.eagle:eagle-common:[unknown-version]: Could not find artifact 
org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNAPSHOT and 
'parent.relativePath' points at wrong local POM @ line 22, column 10

[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: joda-time:joda-time:jar -> duplicate declaration of version (?) @ 
org.apache.eagle:alert-engine:[unknown-version], 
/home/build/git/incubator-eagle/eagle-core/eagle-alert-parent/eagle-alert/alert-engine/pom.xml,
 line 115, column 15

[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: org.apache.kafka:kafka-clients:jar -> duplicate declaration of 
version (?) @ org.apache.eagle:alert-engine:[unknown-version], 
/home/build/git/incubator-eagle/eagle-core/eagle-alert-parent/eagle-alert/alert-engine/pom.xml,
 line 127, column 15

[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: org.apache.eagle:eagle-metadata-base:jar -> version 
${project.version} vs 0.5.0-incubating-SNAPSHOT @ 
org.apache.eagle:eagle-app-base:[unknown-version], 
/home/build/git/incubator-eagle/eagle-core/eagle-app/eagle-app-base/pom.xml, 
line 95, column 21

[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: junit:junit:jar -> duplicate declaration of version (?) @ 
org.apache.eagle:eagle-app-base:[unknown-version], 
/home/build/git/incubator-eagle/eagle-core/eagle-app/eagle-app-base/pom.xml, 
line 100, column 21

[WARNING] 'build.plugins.plugin.version' for 
org.codehaus.mojo:exec-maven-plugin is missing. @ 
org.apache.eagle:eagle-webservice:[unknown-version], 
/home/build/git/incubator-eagle/eagle-webservice/pom.xml, line 400, column 12

 @ 

[ERROR] The build could not read 1 project -> [Help 1]

[ERROR]   

[ERROR]   The project org.apache.eagle:eagle-common:[unknown-version] 
(/home/build/git/incubator-eagle/eagle-core/eagle-common/pom.xml) has 1 error

[ERROR] Non-resolvable parent POM for 
org.apache.eagle:eagle-common:[unknown-version]: Could not find artifact 
org.apache.eagle:eagle-query-parent:pom:0.5.0-incubating-SNAPSHOT and 
'parent.relativePath' points at wrong local POM @ line 22, column 10 -> [Help 2]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please 
read the following articles:

[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException

[ERROR] [Help 2] 
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

 

Re: Incubating Project Info Page of eagle has been updated

[Don Bosco Durai]

Seems, the Apache Eagle download page also needs to be updated with the latest 
source bits links:

https://eagle.incubator.apache.org/docs/download.html

Thanks

Bosco


On 7/24/16, 5:29 PM, "Liangfei.Su"  wrote:

Looks better. Thanks, Michael :)

On Sat, Jul 23, 2016 at 11:47 AM, Michael Wu  wrote:

> Hi all,
>
> Just an update for you to be aware of:
>
> The page at http://incubator.apache.org/projects/eagle.html has not been
> updated for long since  0.3.0-incubating release, now that we already have
> 0.4.0-incubating released, I just made an update with the news happened
> recently since the last update.
>
> Thanks.
> Michael
>

Re: Apache Ranger integration for Audit Logs...

[Don Bosco Durai]

I have some spare time and was planning to work on this. If no one currently 
looking into this JIRA, then can you assign it to me? 

https://issues.apache.org/jira/browse/EAGLE-59


Thanks

Bosco


On 11/29/15, 8:43 PM, "Don Bosco Durai" <bo...@apache.org> wrote:

Edward

Thanks. I will look into HdfsAuditLogProcessorMain class.

I will upload the sample files in the JIRA. 



Thanks

Bosco


On 11/29/15, 7:56 PM, "Zhang, Edward (GDI Hadoop)" <yonzh...@ebay.com> 
wrote:

>One more thing, Bosco, could you please copy some sample hdfs audit log,
>hbase log and hive log to here?
>
>I realize with Ranger data source, we probably still need some minor code
>development as follows
>1. Substitute existing eagle data source(raw hdfs audit log) with Ranger
>data source, for example, in HdfsAuditLogProcessorMain, modify the code to
>use different log deserializer
>2. Ensure output of Ranger log deserializer is compatible to existing
>eagle data source.
>
>With the above code change, we can automatically get all capabilities like
>sensitivity data join, user hadoop command reassembly, hive query
>semantics parsing etc.
>
>Thanks
>Edward Zhang
>
>On 11/29/15, 18:52, "Zhang, Edward (GDI Hadoop)" <yonzh...@ebay.com> wrote:
>
>>Hi Bosco,
>>
>>Thanks for creating this ticket. It is very helpful if EAGLE can use
>>Ranger as data source and automatically get monitoring capability in 9
>>Hadoop components.
>>
>>If a datasource is not from Kafka, and needs a lot of pre-processing, it
>>is not trivial to integrate that data source.
>>
>>Ranger¹s data source should be uniform in syntax and the integration
>>should be straightforward, if we have a uniform deserializer.
>>
>>I think we can document the steps of integrating a new datasource.
>>
>>Thanks
>>Edward Zhang
>>
>>On 11/29/15, 12:00, "Don Bosco Durai" <bo...@apache.org> wrote:
>>
>>>Hi Eagle team
>>>
>>>I am excited to see all the activities on this project. I have created a
>>>JIRA (https://issues.apache.org/jira/browse/EAGLE-59) to track the
>>>integration with Apache Ranger.
>>>
>>>One way to integrate is for Ranger to send the audit logs in the same way
>>>as native log format to Kafka. However, Ranger already is doing the
>>>normalization of the audit format for all the components. So
>>>reconstructing might not be a good way to go.
>>>
>>>I am still getting familiar with the internals of Apache Eagle, but if
>>>someone can help me or document how a 3rd party source can be integrated
>>>with Apache Eagle, then it will be great. Also, what is the change
>>>required on the analytics side to support new data sources? E.g. If we
>>>integrate with Ranger Audit Logs, we would get audit logs from around 9
>>>components right away. How can we use it?
>>>
>>>If you are okay, I am willing to work on this JIRA.
>>>
>>>Thanks
>>>
>>>Bosco
>>> 
>>>
>>
>

Re: Build error due to Scala version...

[Don Bosco Durai]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>at java.lang.reflect.Method.invoke(Method.java:606)
>at 
> com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
>Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution 
>failed.
>
>From: Edward Zhang [mailto:yonzhang2...@apache.org]
>Sent: Wednesday, December 02, 2015 4:13 PM
>To: dev@eagle.incubator.apache.org
>Subject: Re: Build error due to Scala version...
>
>I also met another issue after I deployed the latest build into sandbox.
>
>The http://localhost:9099/eagle-service is redirected to 
>http://localhost:9099/eagle-service/ui and then it has HTTP 404 issue.
>
>[Inline image 1]
>
>Thanks
>Edward
>
>On Sun, Nov 29, 2015 at 11:57 PM, Don Bosco Durai 
><bo...@apache.org<mailto:bo...@apache.org>> wrote:
>I am using mac.
>
>Bosco
>
>
>
>
>On 11/29/15, 11:10 PM, "Chen, Hao" 
><hao.ch...@ebay.com<mailto:hao.ch...@ebay.com>> wrote:
>
>>Please install https://www.npmjs.com/ before compiling. And what’s your 
>>environment?
>>
>>Thanks,
>>
>>
>>Hao
>>
>>On 11/30/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)" 
>><yonzh...@ebay.com<mailto:yonzh...@ebay.com>> wrote:
>>
>>>Looks it is related to recent changes by Jilin on NPM support.
>>>
>>>Thanks
>>>Edward
>>>
>>>On 11/29/15, 20:55, "Don Bosco Durai" 
>>><bo...@apache.org<mailto:bo...@apache.org>> wrote:
>>>
>>>>Thanks for the suggestion. With JDK 1.7, I went further, but stuck at
>>>>other place:
>>>>
>>>>[INFO] eagle-security-userprofile-common .. SUCCESS [
>>>>11.817 s]
>>>>[INFO] eagle-webservice ... FAILURE [
>>>>0.267 s]
>>>>[INFO] eagle-security-hdfs-auditlog ... SKIPPED
>>>>[INFO] eagle-security-userprofile-detection ... SKIPPED
>>>>[INFO] eagle-security-userprofile-training  SKIPPED
>>>>[INFO] eagle-security-hive  SKIPPED
>>>>[INFO] eagle-security-hdfs-securitylog  SKIPPED
>>>>[INFO] eagle-security-hbase-securitylog ... SKIPPED
>>>>[INFO] eagle-external-parent .. SKIPPED
>>>>[INFO] eagle-log4jkafka ... SKIPPED
>>>>[INFO] eagle-topology-assembly  SKIPPED
>>>>[INFO] eagle-assembly . SKIPPED
>>>>[INFO]
>>>>
>>>>[INFO] BUILD FAILURE
>>>>[INFO]
>>>>
>>>>[INFO] Total time: 01:48 min
>>>>[INFO] Finished at: 2015-11-29T20:43:40-08:00
>>>>[INFO] Final Memory: 64M/619M
>>>>[INFO]
>>>>
>>>>[ERROR] Failed to execute goal
>>>>org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (exec-ui-install) on
>>>>project eagle-webservice: Command execution failed. Process exited with
>>>>an error: 1 (Exit value: 1) -> [Help 1]
>>>>[ERROR]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>On 11/29/15, 6:59 PM, "Liangfei.Su" 
>>>><suliang...@gmail.com<mailto:suliang...@gmail.com>> wrote:
>>>>
>>>>>Saw this before, the solution is to use java 1.7, it builds ok when i
>>>>>switch to JDK 1.7.
>>>>>
>>>>>
>>>>>Thanks,
>>>>>Ralph
>>>>>
>>>>>On Mon, Nov 30, 2015 at 3:08 AM, Don Bosco Durai 
>>>>><bo...@apache.org<mailto:bo...@apache.org>>
>>>>>wrote:
>>>>>
>>>>>> I am getting a build error while building Apache Eagle. It seems to be
>>>>>> because the Scala version. I have scala-2.11.5 installed on my
>>>>>>machine. I
>>>>>> can plan downgrading my Scala, but that would break my Kafka 0.9

Build error due to Scala version...

[Don Bosco Durai]

I am getting a build error while building Apache Eagle. It seems to be because 
the Scala version. I have scala-2.11.5 installed on my machine. I can plan 
downgrading my Scala, but that would break my Kafka 0.9 builds. Any suggestions?

[INFO] --- maven-scala-plugin:2.15.0:compile (scala-compile-first) @ 
eagle-stream-process-base ---
[INFO] Checking for multiple versions of scala
[WARNING]  Expected all dependencies to require Scala version: 2.10.0
[WARNING]  org.apache.kafka:kafka_2.10:0.8.1.2.2.0.0-2041 requires scala 
version: 2.10.4
[WARNING] Multiple versions of scala libraries detected!
[INFO] includes = [**/*.java,**/*.scala,]
[INFO] excludes = []
[INFO] 
/Users/bdurai/git/incubator-eagle/eagle-core/eagle-data-process/eagle-stream-process-base/src/main/java:-1:
 info: compiling
[INFO] 
/Users/bdurai/git/incubator-eagle/eagle-core/eagle-data-process/eagle-stream-process-base/src/main/scala:-1:
 info: compiling
[INFO] Compiling 10 source files to 
/Users/bdurai/git/incubator-eagle/eagle-core/eagle-data-process/eagle-stream-process-base/target/classes
 at 1448822136862
[ERROR] error: error while loading CharSequence, class file 
'/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/rt.jar(java/lang/CharSequence.class)'
 is broken
[INFO] (class java.lang.RuntimeException/bad constant pool tag 18 at byte 10)
[ERROR] error: error while loading AnnotatedElement, class file 
'/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/rt.jar(java/lang/reflect/AnnotatedElement.class)'
 is broken
[INFO] (class java.lang.RuntimeException/bad constant pool tag 18 at byte 76)
[ERROR] two errors found

Thanks

Bosco

Apache Ranger integration for Audit Logs...

[Don Bosco Durai]

Hi Eagle team

I am excited to see all the activities on this project. I have created a JIRA 
(https://issues.apache.org/jira/browse/EAGLE-59) to track the integration with 
Apache Ranger.  

One way to integrate is for Ranger to send the audit logs in the same way as 
native log format to Kafka. However, Ranger already is doing the normalization 
of the audit format for all the components. So reconstructing might not be a 
good way to go.

I am still getting familiar with the internals of Apache Eagle, but if someone 
can help me or document how a 3rd party source can be integrated with Apache 
Eagle, then it will be great. Also, what is the change required on the 
analytics side to support new data sources? E.g. If we integrate with Ranger 
Audit Logs, we would get audit logs from around 9 components right away. How 
can we use it?

If you are okay, I am willing to work on this JIRA.

Thanks

Bosco
 

Re: Eagle wiki is ready

[Don Bosco Durai]

Seems the wiki is still password protected. 

Also, the Eagle website is pretty rich with content. What will go in the Wiki 
pages?

Bosco





On 11/24/15, 9:58 PM, "Hao Chen"  wrote:

>Cool, thanks Liangfei!
>
>Regards,
>Hao
>
>On Wed, Nov 25, 2015 at 1:11 PM, Henry Saputra 
>wrote:
>
>> Awesome news!
>>
>> On Mon, Nov 23, 2015 at 6:44 PM, sul*fei  wrote:
>> > Thanks to Daniel's work in
>> > https://issues.apache.org/jira/browse/INFRA-10793
>> > https://issues.apache.org/jira/servicedesk/agent/INFRA/issue/INFRA-10812
>> >
>> >
>> > Eagle wiki is now available at
>> https://cwiki.apache.org/confluence/display/EAG/Eagle. We can start
>> putting design/discussion/faq there.
>> >
>> >
>> > Thanks,
>> > Ralph
>> >
>>

Re: Eagle wiki is ready

[Don Bosco Durai]

In the Space tools->Permissions page, you can give “Anonymous” users “View” 
access to all. This way, anyone can view it without logging in...

Bosco



On 11/29/15, 6:32 PM, "Zhang, Edward (GDI Hadoop)" <yonzh...@ebay.com> wrote:

>Hi Bosco,
>
>The apache cwiki looks needs user/password, but isn¹t that required?
>
>We are going to put design documentation to Eagle wiki but leave user
>faced information in Eagle site. Please suggest also.
>
>Thanks
>Edward Zhang
>
>On 11/29/15, 11:04, "Don Bosco Durai" <bo...@apache.org> wrote:
>
>>Seems the wiki is still password protected.
>>
>>Also, the Eagle website is pretty rich with content. What will go in the
>>Wiki pages?
>>
>>Bosco
>>
>>
>>
>>
>>
>>On 11/24/15, 9:58 PM, "Hao Chen" <h...@apache.org> wrote:
>>
>>>Cool, thanks Liangfei!
>>>
>>>Regards,
>>>Hao
>>>
>>>On Wed, Nov 25, 2015 at 1:11 PM, Henry Saputra <henry.sapu...@gmail.com>
>>>wrote:
>>>
>>>> Awesome news!
>>>>
>>>> On Mon, Nov 23, 2015 at 6:44 PM, sul*fei <suliang...@163.com> wrote:
>>>> > Thanks to Daniel's work in
>>>> > https://issues.apache.org/jira/browse/INFRA-10793
>>>> > 
>>>>https://issues.apache.org/jira/servicedesk/agent/INFRA/issue/INFRA-10812
>>>> >
>>>> >
>>>> > Eagle wiki is now available at
>>>> https://cwiki.apache.org/confluence/display/EAG/Eagle. We can start
>>>> putting design/discussion/faq there.
>>>> >
>>>> >
>>>> > Thanks,
>>>> > Ralph
>>>> >
>>>>
>>
>

Re: Build error due to Scala version...

[Don Bosco Durai]

I am using mac.

Bosco




On 11/29/15, 11:10 PM, "Chen, Hao" <hao.ch...@ebay.com> wrote:

>Please install https://www.npmjs.com/ before compiling. And what’s your 
>environment?
>
>Thanks,
>
>
>Hao
>
>On 11/30/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)" <yonzh...@ebay.com> wrote:
>
>>Looks it is related to recent changes by Jilin on NPM support.
>>
>>Thanks
>>Edward
>>
>>On 11/29/15, 20:55, "Don Bosco Durai" <bo...@apache.org> wrote:
>>
>>>Thanks for the suggestion. With JDK 1.7, I went further, but stuck at
>>>other place:
>>>
>>>[INFO] eagle-security-userprofile-common .. SUCCESS [
>>>11.817 s]
>>>[INFO] eagle-webservice ... FAILURE [
>>>0.267 s]
>>>[INFO] eagle-security-hdfs-auditlog ... SKIPPED
>>>[INFO] eagle-security-userprofile-detection ... SKIPPED
>>>[INFO] eagle-security-userprofile-training  SKIPPED
>>>[INFO] eagle-security-hive  SKIPPED
>>>[INFO] eagle-security-hdfs-securitylog  SKIPPED
>>>[INFO] eagle-security-hbase-securitylog ... SKIPPED
>>>[INFO] eagle-external-parent .. SKIPPED
>>>[INFO] eagle-log4jkafka ... SKIPPED
>>>[INFO] eagle-topology-assembly  SKIPPED
>>>[INFO] eagle-assembly . SKIPPED
>>>[INFO] 
>>>
>>>[INFO] BUILD FAILURE
>>>[INFO] 
>>>
>>>[INFO] Total time: 01:48 min
>>>[INFO] Finished at: 2015-11-29T20:43:40-08:00
>>>[INFO] Final Memory: 64M/619M
>>>[INFO] 
>>>
>>>[ERROR] Failed to execute goal
>>>org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (exec-ui-install) on
>>>project eagle-webservice: Command execution failed. Process exited with
>>>an error: 1 (Exit value: 1) -> [Help 1]
>>>[ERROR] 
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>On 11/29/15, 6:59 PM, "Liangfei.Su" <suliang...@gmail.com> wrote:
>>>
>>>>Saw this before, the solution is to use java 1.7, it builds ok when i
>>>>switch to JDK 1.7.
>>>>
>>>>
>>>>Thanks,
>>>>Ralph
>>>>
>>>>On Mon, Nov 30, 2015 at 3:08 AM, Don Bosco Durai <bo...@apache.org>
>>>>wrote:
>>>>
>>>>> I am getting a build error while building Apache Eagle. It seems to be
>>>>> because the Scala version. I have scala-2.11.5 installed on my
>>>>>machine. I
>>>>> can plan downgrading my Scala, but that would break my Kafka 0.9
>>>>>builds.
>>>>> Any suggestions?
>>>>>
>>>>> [INFO] --- maven-scala-plugin:2.15.0:compile (scala-compile-first) @
>>>>> eagle-stream-process-base ---
>>>>> [INFO] Checking for multiple versions of scala
>>>>> [WARNING]  Expected all dependencies to require Scala version: 2.10.0
>>>>> [WARNING]  org.apache.kafka:kafka_2.10:0.8.1.2.2.0.0-2041 requires
>>>>>scala
>>>>> version: 2.10.4
>>>>> [WARNING] Multiple versions of scala libraries detected!
>>>>> [INFO] includes = [**/*.java,**/*.scala,]
>>>>> [INFO] excludes = []
>>>>> [INFO]
>>>>> 
>>>>>/Users/bdurai/git/incubator-eagle/eagle-core/eagle-data-process/eagle-st
>>>>>ream-process-base/src/main/java:-1:
>>>>> info: compiling
>>>>> [INFO]
>>>>> 
>>>>>/Users/bdurai/git/incubator-eagle/eagle-core/eagle-data-process/eagle-st
>>>>>ream-process-base/src/main/scala:-1:
>>>>> info: compiling
>>>>> [INFO] Compiling 10 source files to
>>>>> 
>>>>>/Users/bdurai/git/incubator-eagle/eagle-core/eagle-data-process/eagle-st
>>>>>ream-process-base/target/classes
>>>>> at 1448822136862
>>>>> [ERROR] error: error while loading CharSequence, class file
>>>>> 
>>>>>'/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib
>>>>>/rt.jar(java/lang/CharSequence.class)'
>>>>> is broken
>>>>> [INFO] (class java.lang.RuntimeException/bad constant pool tag 18 at
>>>>>byte
>>>>> 10)
>>>>> [ERROR] error: error while loading AnnotatedElement, class file
>>>>> 
>>>>>'/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib
>>>>>/rt.jar(java/lang/reflect/AnnotatedElement.class)'
>>>>> is broken
>>>>> [INFO] (class java.lang.RuntimeException/bad constant pool tag 18 at
>>>>>byte
>>>>> 76)
>>>>> [ERROR] two errors found
>>>>>
>>>>> Thanks
>>>>>
>>>>> Bosco
>>>>>
>>>>>
>>>>>
>>>
>>

[jira] [Commented] (EAGLE-16) HBase security log integration for HBase security monitoring

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002703#comment-15002703
 ] 

Don Bosco Durai commented on EAGLE-16:
--

It emits the unique raw logs (excluding evtTime) for the interval with the 
event_count and event_dur_ms (check last 2 field in the log messages):

{"repoType":2,"repo":"cl1_hbase","reqUser":"hbase","evtTime":"2015-09-19 
23:07:34.124","access":"get","resource":"hbase:namespace/info","resType":"column-family","action":"read","result":1,"policy":-1,"enforcer":"ranger-acl","cliIP":"172.22.67.225","agentHost":"bdurai-workshop2-1","logType":"RangerAudit","id":"4397bcd2-5723-4d9d-92b7-5a80ae2603e4","seq_num":5,"event_count":2,"event_dur_ms":111}

{"repoType":2,"repo":"cl1_hbase","reqUser":"hbase","evtTime":"2015-09-19 
23:12:33.248","access":"close","resource":"hbase:namespace","resType":"table","action":"admin","result":1,"policy":-1,"enforcer":"ranger-acl","cliIP":"172.22.67.225","agentHost":"bdurai-workshop2-1","logType":"RangerAudit","id":"3840666d-b9d7-4191-99c3-071e077cfe84","seq_num":11,"event_count":2,"event_dur_ms":4}

The interval is configurable. By default it is 5 seconds. It will only emit 
unique logs with their count. Each unique log message will have the evtTime of 
the first message of it's kind.

This is enabled by default for HBase, Kafka and Solr Ranger Plugin.



> HBase security log integration for HBase security monitoring
> 
>
> Key: EAGLE-16
> URL: https://issues.apache.org/jira/browse/EAGLE-16
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Edward Zhang
>Assignee: Zhao, Qingwen
>
> HBase security log is good data source for monitoring user 
> authentication/authorization events, Eagle should incorporate HBase security 
> log



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (EAGLE-16) HBase security log integration for HBase security monitoring

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/EAGLE-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002703#comment-15002703
 ] 

Don Bosco Durai edited comment on EAGLE-16 at 11/12/15 7:27 PM:


It emits the unique raw logs  for the interval with the event_count and 
event_dur_ms (check last 2 field in the log messages). Each unique log message 
will have the evtTime of the first message of it's kind.

{"repoType":2,"repo":"cl1_hbase","reqUser":"hbase","evtTime":"2015-09-19 
23:07:34.124","access":"get","resource":"hbase:namespace/info","resType":"column-family","action":"read","result":1,"policy":-1,"enforcer":"ranger-acl","cliIP":"172.22.67.225","agentHost":"bdurai-workshop2-1","logType":"RangerAudit","id":"4397bcd2-5723-4d9d-92b7-5a80ae2603e4","seq_num":5,"event_count":2,"event_dur_ms":111}

{"repoType":2,"repo":"cl1_hbase","reqUser":"hbase","evtTime":"2015-09-19 
23:12:33.248","access":"close","resource":"hbase:namespace","resType":"table","action":"admin","result":1,"policy":-1,"enforcer":"ranger-acl","cliIP":"172.22.67.225","agentHost":"bdurai-workshop2-1","logType":"RangerAudit","id":"3840666d-b9d7-4191-99c3-071e077cfe84","seq_num":11,"event_count":2,"event_dur_ms":4}

The interval is configurable. By default it is 5 seconds. It will only emit 
unique logs with their count. 

This is enabled by default for HBase, Kafka and Solr Ranger Plugin.




was (Author: bosco):
It emits the unique raw logs (excluding evtTime) for the interval with the 
event_count and event_dur_ms (check last 2 field in the log messages):

{"repoType":2,"repo":"cl1_hbase","reqUser":"hbase","evtTime":"2015-09-19 
23:07:34.124","access":"get","resource":"hbase:namespace/info","resType":"column-family","action":"read","result":1,"policy":-1,"enforcer":"ranger-acl","cliIP":"172.22.67.225","agentHost":"bdurai-workshop2-1","logType":"RangerAudit","id":"4397bcd2-5723-4d9d-92b7-5a80ae2603e4","seq_num":5,"event_count":2,"event_dur_ms":111}

{"repoType":2,"repo":"cl1_hbase","reqUser":"hbase","evtTime":"2015-09-19 
23:12:33.248","access":"close","resource":"hbase:namespace","resType":"table","action":"admin","result":1,"policy":-1,"enforcer":"ranger-acl","cliIP":"172.22.67.225","agentHost":"bdurai-workshop2-1","logType":"RangerAudit","id":"3840666d-b9d7-4191-99c3-071e077cfe84","seq_num":11,"event_count":2,"event_dur_ms":4}

The interval is configurable. By default it is 5 seconds. It will only emit 
unique logs with their count. Each unique log message will have the evtTime of 
the first message of it's kind.

This is enabled by default for HBase, Kafka and Solr Ranger Plugin.



> HBase security log integration for HBase security monitoring
> 
>
> Key: EAGLE-16
> URL: https://issues.apache.org/jira/browse/EAGLE-16
> Project: Eagle
>  Issue Type: New Feature
>Reporter: Edward Zhang
>Assignee: Zhao, Qingwen
>
> HBase security log is good data source for monitoring user 
> authentication/authorization events, Eagle should incorporate HBase security 
> log



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)