Eagle 352 make necessary changes for preparing 0.4.0 release candidate 2 Change for 0.4.0 rc2
Author: Michael, Wu <m...@apache.org> Reviewer: Edward Zhang <yonzhang2...@apache.org> Closes: #249 Project: http://git-wip-us.apache.org/repos/asf/incubator-eagle/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-eagle/commit/23d3ca60 Tree: http://git-wip-us.apache.org/repos/asf/incubator-eagle/tree/23d3ca60 Diff: http://git-wip-us.apache.org/repos/asf/incubator-eagle/diff/23d3ca60 Branch: refs/heads/master Commit: 23d3ca604a37ccb89d7a95e5c75cfba651673b2a Parents: 73c8e02 Author: yonzhang <yonzhang2...@gmail.com> Authored: Tue Jul 5 13:57:45 2016 -0700 Committer: yonzhang <yonzhang2...@gmail.com> Committed: Tue Jul 5 13:57:45 2016 -0700 ---------------------------------------------------------------------- CHANGELOG.txt | 215 +++++++++++++++++++++++- README.md | 10 +- eagle-docs/images/asf_logo.svg | 16 ++ eagle-webservice/src/main/webapp/README.md | 19 +++ pom.xml | 114 ++++++------- 5 files changed, 312 insertions(+), 62 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/23d3ca60/CHANGELOG.txt ---------------------------------------------------------------------- diff --git a/CHANGELOG.txt b/CHANGELOG.txt index a2d84fc..657e31d 100644 --- a/CHANGELOG.txt +++ b/CHANGELOG.txt @@ -1,5 +1,5 @@ -Release Notes - Eagle - Version v0.4.0 +Release Notes - Apache Eagle 0.4.0 (incubating) ** Highlights ** * JBDC Metadata Storage Extension @@ -111,4 +111,215 @@ Release Notes - Eagle - Version v0.4.0 * [EAGLE-309] - Add code formatter template ** Sub-task - * [EAGLE-219] - Use PUT method for updating request when possible in front-end. \ No newline at end of file + * [EAGLE-219] - Use PUT method for updating request when possible in front-end. + + + +Release Notes - Apache Eagle 0.3.1 (incubating) + +** Highlights ** + * Hadoop JMX metric monitoring. + * NameNode log monitoring for GC alerts. + * Eagle UI Modularization (With new customizable UI options). + * Generic Notification Plugin Framework for integrating alerts with various third parties. + * Docker Image for Eagle. + * Provide analytic DSL support. + +** New Feature + * [EAGLE-15] - HBase auditlog integration for HBase security monitoring + * [EAGLE-17] - HDFS security log integration for HDFS security monitoring + * [EAGLE-46] - track the work before moving code to apache site + * [EAGLE-47] - Ability to audit who made changes to Eagle policies + * [EAGLE-53] - Docker Image for Eagle + * [EAGLE-79] - Provide analytic DSL support + * [EAGLE-81] - Notification Plugin Framework + * [EAGLE-100] - provide tools to collect hadoop jmx metrics + * [EAGLE-122] - Create a generic notification plugin for integrating alerts + +** Bug + * [EAGLE-4] - configure parallelism with bolt/spout friendly name + * [EAGLE-6] - group-by user between bolts and spouts for hdfs/hive audit log monitoring + * [EAGLE-10] - Bad way to check kafka installation in eagle-check-env.sh + * [EAGLE-28] - Znode root should be configurable. + * [EAGLE-30] - Topology run in storm local mode should not terminate after Integer.MAX_VALUE milliseconds + * [EAGLE-41] - Eagle policy engine supports customizable ExceptionHandler instead of using FetalExceptionHandler + * [EAGLE-48] - Alert message time is wrong in some cases + * [EAGLE-56] - Building failed for artifact jdk.tools:jdk.tools:jar:1.7 not found + * [EAGLE-69] - http 404 bug in accessing http://localhost:9099/eagle-service + * [EAGLE-70] - Fix Goovy into Groovy in pom + * [EAGLE-74] - in sandbox setup, change package name from eagle. to org.apache.eagle + * [EAGLE-76] - Disable creating newStream for common user role + * [EAGLE-85] - Fix unit test failure. + * [EAGLE-94] - fix the bug of eagle docker in Mac + * [EAGLE-95] - DAM HDFS topology fails to start, if there is invalid policy. + * [EAGLE-104] - Fix the unit test TestHDFSSecuritylogParser + * [EAGLE-118] - Siddhi contains is neither a function extension nor an aggregated attribute extension + * [EAGLE-119] - fix unit testing bugs + * [EAGLE-125] - Add LICENSE in eagle external + * [EAGLE-128] - Fix unit test failure + * [EAGLE-141] - fix some unit testing exception which is thrown within annoymous class + * [EAGLE-142] - Replace AlertDefinitionDAOImpl with PolicyDefinitionDAOImpl + * [EAGLE-148] - Master build failure due to siddhi ql test + * [EAGLE-163] - HDFS topology not working. + +** Improvement + * [EAGLE-7] - expose storm config as individual eagle topology config + * [EAGLE-13] - Eagle machine learning no longer needs hourly aggregation + * [EAGLE-14] - Re-assemble high level hdfs commands for better policy + * [EAGLE-24] - Load skew issue when partition by user + * [EAGLE-29] - Update Alert page for display more infomation + * [EAGLE-38] - reminder for Alerts in the UI + * [EAGLE-40] - Policy detail page support alert list display + * [EAGLE-42] - Customized partition support for Kafka spout + * [EAGLE-44] - previous page should be reloaded after login session is timed out and re-login + * [EAGLE-50] - Eagle internal metric framework + * [EAGLE-52] - Eagle framework should support custom group by function in addition to group by fields + * [EAGLE-55] - JobCompletedConfigServiceURLBuilderImpl in hive running spout miss anonymous paramter + * [EAGLE-58] - Enhance Hive query parse model + * [EAGLE-62] - Add jshint for front end UI building + * [EAGLE-66] - Eagle TypeSafe Stream Processing DSL + * [EAGLE-80] - remove httpd directory + * [EAGLE-86] - Rewrite HDFSAuditLogParser to a non-regular expression version + * [EAGLE-88] - Eagle web-service should have status url + * [EAGLE-99] - policy distribution statistics to be printed in log + * [EAGLE-106] - UI use same chart component + * [EAGLE-107] - Build Status Icon has been enabled for our github repo, please apply it to readme.md + * [EAGLE-108] - a tool tests performace between mongodb3.0 and couchDB1.6.1 + * [EAGLE-109] - add hdfs related ports mapping from docker container to host + * [EAGLE-112] - Should support sum aggregation in front end for slide window function + * [EAGLE-114] - Enable RAT check and fix missing license headers + * [EAGLE-138] - Extend JMX Collector to support "hadoop.namenode.JournalTransaction" + * [EAGLE-143] - Create Eagle release 0.3.1 + * [EAGLE-149] - Enable hadoop jmx metric cases + * [EAGLE-152] - hadoop-metric monitoring automation script + * [EAGLE-158] - Disable org.apache.eagle.datastream.EagleTuple + * [EAGLE-166] - Enhance metric collector script to extract hadoop ha status as metric + +** Task + * [EAGLE-83] - Integrate Jenkins with GitHub for checking pull requests + * [EAGLE-84] - Create scripts for merging PRs automatically + * [EAGLE-102] - ask jenkins-admin's help to setup "Embeddable Build Status Plugin" and make the status visible in Readme.md of incubator-eagle.git + * [EAGLE-117] - update Build Status Link and make it point to CI job building upon master + * [EAGLE-131] - make eagle main CI build pass + * [EAGLE-136] - Setup hadoop metric application + * [EAGLE-154] - hadoop-metric monitoring sandbox starter script + * [EAGLE-164] - Tutorial for setting up the development environment on mac + * [EAGLE-165] - Add more jmx metric cases + +** Sub-task + * [EAGLE-51] - Clean and manage external licensed static resource dependencies in eagle web with npm and grunt + * [EAGLE-60] - HBase sensitivity UI + * [EAGLE-63] - Improve docker image for quick preview + * [EAGLE-64] - Push eagle docker image to docker hub + * [EAGLE-65] - Eagle docker consistent port mapping + * [EAGLE-71] - Fix eagle docker run command + * [EAGLE-75] - Leverage dropwizard metrics for generating Eagle Topology and DataSource Metrics + * [EAGLE-78] - eagle-lib.sh script not work for mac osx and windows + * [EAGLE-130] - Eagle Pipeline DSL: Parser, Compiler, Runner + * [EAGLE-140] - Eagle Pipeline Package and Script + + + +Release Notes - Apache Eagle 0.3.0 (incubating) + +** Highlights ** + * Hadoop JMX metric monitoring. + * NameNode log monitoring for GC alerts. + * Eagle UI Modularization (With new customizable UI options). + * Generic Notification Plugin Framework for integrating alerts with various third parties. + * Docker Image for Eagle. + * Provide analytic DSL support. + +** New Feature + * [EAGLE-15] - HBase auditlog integration for HBase security monitoring + * [EAGLE-17] - HDFS security log integration for HDFS security monitoring + * [EAGLE-46] - track the work before moving code to apache site + * [EAGLE-47] - Ability to audit who made changes to Eagle policies + * [EAGLE-53] - Docker Image for Eagle + * [EAGLE-79] - Provide analytic DSL support + * [EAGLE-81] - Notification Plugin Framework + * [EAGLE-100] - provide tools to collect hadoop jmx metrics + * [EAGLE-122] - Create a generic notification plugin for integrating alerts + +** Bug + * [EAGLE-4] - configure parallelism with bolt/spout friendly name + * [EAGLE-6] - group-by user between bolts and spouts for hdfs/hive audit log monitoring + * [EAGLE-10] - Bad way to check kafka installation in eagle-check-env.sh + * [EAGLE-28] - Znode root should be configurable. + * [EAGLE-30] - Topology run in storm local mode should not terminate after Integer.MAX_VALUE milliseconds + * [EAGLE-41] - Eagle policy engine supports customizable ExceptionHandler instead of using FetalExceptionHandler + * [EAGLE-48] - Alert message time is wrong in some cases + * [EAGLE-56] - Building failed for artifact jdk.tools:jdk.tools:jar:1.7 not found + * [EAGLE-69] - http 404 bug in accessing http://localhost:9099/eagle-service + * [EAGLE-70] - Fix Goovy into Groovy in pom + * [EAGLE-74] - in sandbox setup, change package name from eagle. to org.apache.eagle + * [EAGLE-76] - Disable creating newStream for common user role + * [EAGLE-85] - Fix unit test failure. + * [EAGLE-94] - fix the bug of eagle docker in Mac + * [EAGLE-95] - DAM HDFS topology fails to start, if there is invalid policy. + * [EAGLE-104] - Fix the unit test TestHDFSSecuritylogParser + * [EAGLE-118] - Siddhi contains is neither a function extension nor an aggregated attribute extension + * [EAGLE-119] - fix unit testing bugs + * [EAGLE-125] - Add LICENSE in eagle external + * [EAGLE-128] - Fix unit test failure + * [EAGLE-141] - fix some unit testing exception which is thrown within annoymous class + * [EAGLE-142] - Replace AlertDefinitionDAOImpl with PolicyDefinitionDAOImpl + * [EAGLE-148] - Master build failure due to siddhi ql test + * [EAGLE-163] - HDFS topology not working. + +** Improvement + * [EAGLE-7] - expose storm config as individual eagle topology config + * [EAGLE-13] - Eagle machine learning no longer needs hourly aggregation + * [EAGLE-14] - Re-assemble high level hdfs commands for better policy + * [EAGLE-24] - Load skew issue when partition by user + * [EAGLE-29] - Update Alert page for display more infomation + * [EAGLE-38] - reminder for Alerts in the UI + * [EAGLE-40] - Policy detail page support alert list display + * [EAGLE-42] - Customized partition support for Kafka spout + * [EAGLE-44] - previous page should be reloaded after login session is timed out and re-login + * [EAGLE-50] - Eagle internal metric framework + * [EAGLE-52] - Eagle framework should support custom group by function in addition to group by fields + * [EAGLE-55] - JobCompletedConfigServiceURLBuilderImpl in hive running spout miss anonymous paramter + * [EAGLE-58] - Enhance Hive query parse model + * [EAGLE-62] - Add jshint for front end UI building + * [EAGLE-66] - Eagle TypeSafe Stream Processing DSL + * [EAGLE-80] - remove httpd directory + * [EAGLE-86] - Rewrite HDFSAuditLogParser to a non-regular expression version + * [EAGLE-88] - Eagle web-service should have status url + * [EAGLE-99] - policy distribution statistics to be printed in log + * [EAGLE-106] - UI use same chart component + * [EAGLE-107] - Build Status Icon has been enabled for our github repo, please apply it to readme.md + * [EAGLE-108] - a tool tests performace between mongodb3.0 and couchDB1.6.1 + * [EAGLE-109] - add hdfs related ports mapping from docker container to host + * [EAGLE-112] - Should support sum aggregation in front end for slide window function + * [EAGLE-114] - Enable RAT check and fix missing license headers + * [EAGLE-138] - Extend JMX Collector to support "hadoop.namenode.JournalTransaction" + * [EAGLE-143] - Create Eagle release 0.3.0 + * [EAGLE-149] - Enable hadoop jmx metric cases + * [EAGLE-152] - hadoop-metric monitoring automation script + * [EAGLE-158] - Disable org.apache.eagle.datastream.EagleTuple + * [EAGLE-166] - Enhance metric collector script to extract hadoop ha status as metric + +** Task + * [EAGLE-83] - Integrate Jenkins with GitHub for checking pull requests + * [EAGLE-84] - Create scripts for merging PRs automatically + * [EAGLE-102] - ask jenkins-admin's help to setup "Embeddable Build Status Plugin" and make the status visible in Readme.md of incubator-eagle.git + * [EAGLE-117] - update Build Status Link and make it point to CI job building upon master + * [EAGLE-131] - make eagle main CI build pass + * [EAGLE-136] - Setup hadoop metric application + * [EAGLE-154] - hadoop-metric monitoring sandbox starter script + * [EAGLE-164] - Tutorial for setting up the development environment on mac + * [EAGLE-165] - Add more jmx metric cases + +** Sub-task + * [EAGLE-51] - Clean and manage external licensed static resource dependencies in eagle web with npm and grunt + * [EAGLE-60] - HBase sensitivity UI + * [EAGLE-63] - Improve docker image for quick preview + * [EAGLE-64] - Push eagle docker image to docker hub + * [EAGLE-65] - Eagle docker consistent port mapping + * [EAGLE-71] - Fix eagle docker run command + * [EAGLE-75] - Leverage dropwizard metrics for generating Eagle Topology and DataSource Metrics + * [EAGLE-78] - eagle-lib.sh script not work for mac osx and windows + * [EAGLE-130] - Eagle Pipeline DSL: Parser, Compiler, Runner + * [EAGLE-140] - Eagle Pipeline Package and Script + http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/23d3ca60/README.md ---------------------------------------------------------------------- diff --git a/README.md b/README.md index 9881fc9..cac233f 100755 --- a/README.md +++ b/README.md @@ -53,11 +53,15 @@ The fastest way to get started with Eagle is to run with [docker](https://github As another alternative option, you could [install eagle package in sandbox](https://eagle.incubator.apache.org/docs/deployment-in-sandbox.html) manualy as well. ## Building Eagle (Supports JDK-1.7.x) -Eagle is built using [Apache Maven](https://maven.apache.org/). NPM should be installed (On MAC OS try "brew install node"). To build Eagle, run: +_**Note:**_ - mvn -DskipTests clean package +1. Currently eagle is tested on **JDK-1.7.X**, currently (v0.4.0) not supporting JDK 1.8. +2. **NPM** should be installed (On MAC OS try "brew install node"), this is a prerequisite. +3. Eagle is built using [Apache Maven](https://maven.apache.org/). + +To build Eagle, run: -Note : Currently eagle is tested on JDK-1.7.X . Eagle does not support JDK 1.8. + mvn -DskipTests clean package After successfully building, you will find eagle binary tarball under _eagle-assembly/target/_ http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/23d3ca60/eagle-docs/images/asf_logo.svg ---------------------------------------------------------------------- diff --git a/eagle-docs/images/asf_logo.svg b/eagle-docs/images/asf_logo.svg index 620694c..4bbe5eb 100644 --- a/eagle-docs/images/asf_logo.svg +++ b/eagle-docs/images/asf_logo.svg @@ -1,4 +1,20 @@ <?xml version="1.0" encoding="utf-8"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one or more + ~ contributor license agreements. See the NOTICE file distributed with + ~ this work for additional information regarding copyright ownership. + ~ The ASF licenses this file to You under the Apache License, Version 2.0 + ~ (the "License"); you may not use this file except in compliance with + ~ the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, software + ~ distributed under the License is distributed on an "AS IS" BASIS, + ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + ~ See the License for the specific language governing permissions and + ~ limitations under the License. + --> <!-- Generator: Adobe Illustrator 19.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) --> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <svg version="1.1" id="Layer_2" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/23d3ca60/eagle-webservice/src/main/webapp/README.md ---------------------------------------------------------------------- diff --git a/eagle-webservice/src/main/webapp/README.md b/eagle-webservice/src/main/webapp/README.md index b4168d5..ff5601c 100644 --- a/eagle-webservice/src/main/webapp/README.md +++ b/eagle-webservice/src/main/webapp/README.md @@ -1,3 +1,22 @@ +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + Apache Eagle Web APP == http://git-wip-us.apache.org/repos/asf/incubator-eagle/blob/23d3ca60/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 000cd4e..8c7b6e3 100755 --- a/pom.xml +++ b/pom.xml @@ -1021,65 +1021,65 @@ <goals> <goal>check</goal> </goals> - <configuration> - <useDefaultExcludes>true</useDefaultExcludes> - <excludes> - <!-- Git specific files --> - <exclude>.git/</exclude> - <exclude>.java-version</exclude> - <exclude>.gitignore</exclude> - <!-- IDE specific files--> - <exclude>**/.idea/</exclude> - <exclude>**/.cache-main</exclude> - <exclude>**/.scalastyle/</exclude> - <exclude>**/*.iml</exclude> - <exclude>**/nb-configuration.xml</exclude> - <exclude>**/.classpath</exclude> - <exclude>**/.settings/**</exclude> - <exclude>**/.project</exclude> - <exclude>**/.metadata/</exclude> - <!-- Maven working directory --> - <exclude>**/target/**</exclude> - <!-- Patch files which can be lying around --> - <exclude>**/*.patch</exclude> - <exclude>**/*.rej</exclude> - <!-- Exclude generated files --> - <exclude>**/gen/**</exclude> - <!-- README and test data with exact format --> - <exclude>README*</exclude> - <exclude>**/*.log</exclude> - <exclude>**/eagle.log*</exclude> - <exclude>**/velocity.log*</exclude> - <!-- all json files should be excluded --> - <exclude>**/*.json</exclude> - <exclude>**/resources/eagle.siddhiext</exclude> - <exclude>**/test/resources/securityAuditLog</exclude> - <exclude>**/resources/**/ml-policyDef-UserProfile.txt</exclude> - <exclude>**/test/resources/onelinehiveauditlog.txt</exclude> - <exclude>**/test/resources/ranger-policy-update-request.txt</exclude> - <exclude>**/dev-supports/**/*.json</exclude> - <exclude>**/dev-supports/**/useractivity-agg-json.txt</exclude> - <exclude>**/conf/sandbox-userprofile-topology.conf</exclude> - <exclude>**/kafka-python/**</exclude> - <exclude>**/six/**</exclude> - <!-- Fonts and Images --> - <exclude>**/fonts/**</exclude> - <exclude>**/images/**</exclude> - <exclude>**/lib/js/**</exclude> - <exclude>**/lib/css/**</exclude> - <exclude>**/*.min.js</exclude> - <exclude>**/jquery-*.js</exclude> - <exclude>**/MANIFEST.MF</exclude> - <!-- External dependency script --> - <exclude>**/resource/serf/etc/ambari.json</exclude> - - <!-- TODO: fix it --> - <exclude>**/webapp/**</exclude> - - </excludes> - </configuration> </execution> </executions> + <configuration> + <useDefaultExcludes>true</useDefaultExcludes> + <excludes> + <!-- Git specific files --> + <exclude>.git/</exclude> + <exclude>.java-version</exclude> + <exclude>.gitignore</exclude> + <!-- IDE specific files--> + <exclude>**/.idea/</exclude> + <exclude>**/.cache-main</exclude> + <exclude>**/.scalastyle/</exclude> + <exclude>**/*.iml</exclude> + <exclude>**/nb-configuration.xml</exclude> + <exclude>**/.classpath</exclude> + <exclude>**/.settings/**</exclude> + <exclude>**/.project</exclude> + <exclude>**/.metadata/</exclude> + <!-- Maven working directory --> + <exclude>**/target/**</exclude> + <!-- Patch files which can be lying around --> + <exclude>**/*.patch</exclude> + <exclude>**/*.rej</exclude> + <!-- Exclude generated files --> + <exclude>**/gen/**</exclude> + <!-- README and test data with exact format --> + <exclude>README*</exclude> + <exclude>**/*.log</exclude> + <exclude>**/eagle.log*</exclude> + <exclude>**/velocity.log*</exclude> + <!-- all json files should be excluded --> + <exclude>**/*.json</exclude> + <exclude>**/resources/eagle.siddhiext</exclude> + <exclude>**/test/resources/securityAuditLog</exclude> + <exclude>**/resources/**/ml-policyDef-UserProfile.txt</exclude> + <exclude>**/test/resources/onelinehiveauditlog.txt</exclude> + <exclude>**/test/resources/ranger-policy-update-request.txt</exclude> + <exclude>**/dev-supports/**/*.json</exclude> + <exclude>**/dev-supports/**/useractivity-agg-json.txt</exclude> + <exclude>**/conf/sandbox-userprofile-topology.conf</exclude> + <exclude>**/kafka-python/**</exclude> + <exclude>**/six/**</exclude> + <!-- Fonts and Images --> + <exclude>**/fonts/**</exclude> + <exclude>**/images/**</exclude> + <exclude>**/lib/js/**</exclude> + <exclude>**/lib/css/**</exclude> + <exclude>**/*.min.js</exclude> + <exclude>**/jquery-*.js</exclude> + <exclude>**/MANIFEST.MF</exclude> + <!-- External dependency script --> + <exclude>**/resource/serf/etc/ambari.json</exclude> + + <!-- TODO: fix it --> + <exclude>**/webapp/**</exclude> + + </excludes> + </configuration> </plugin> </plugins> </build>