subject:"Review Request 65885\: ATLAS\-2470 \- Add JanusGraph Cassandra support to Atlas"

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-04-04 Thread David Radley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review200445
---


Ship it!




Ship It!

- David Radley


On March 28, 2018, 3:28 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 28, 2018, 3:28 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   addons/storm-bridge-shim/pom.xml c25fbfe5 
>   addons/storm-bridge/pom.xml 2627d92e 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 10fb7b86 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   pom.xml 9ddc8e18 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
>   test-tools/src/main/resources/solr/solr.xml 041f0058 
>   webapp/pom.xml 284f538f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/6/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-28 Thread Pierre Padovani



> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > pom.xml
> > Line 673 (original), 673 (patched)
> > 
> >
> > I did a build with  :"mvn clean  package -Pdist" without your patch and 
> > it runs through cleanly.
> > 
> > I apply your patch and build with "mvn clean  package 
> > -Pdist,embedded-cassandra-solr" and get a build failure - details below. 
> > During the integration tests, it tries to spin up an Atlas and an embedded 
> > solr. This works slightly differently depending on the build options. I 
> > suspect that there is something in the integration tests for the embedded 
> > cassandra build that is not set up correctly so it is not fidning the 
> > embdedd solr. 
> > 
> >  
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 04:31 min
> > [INFO] Finished at: 2018-03-26T13:49:19+01:00
> > [INFO] Final Memory: 146M/1574M
> > [INFO] 
> > 
> > [ERROR] Failed to execute goal 
> > org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) 
> > on project atlas-repository: ExecutionException: 
> > java.lang.RuntimeException: There was an error in the forked process
> > [ERROR] java.lang.ExceptionInInitializerError
> > [ERROR] at java.lang.Class.forName0(Native Method)
> > [ERROR] at java.lang.Class.forName(Class.java:264)
> > [ERROR] at 
> > org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
> > [ERROR] at 
> > org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
> > [ERROR] at 
> > com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> > [ERROR] at 
> > com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> > [ERROR] at 
> > com.google.inject.spi.Elements.getElements(Elements.java:110)
> > [ERROR] at 
> > com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> > [ERROR] at 
> > com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> > [ERROR] at 
> > com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
> > [ERROR] at 
> > org.testng.TestRunner.getInjector(TestRunner.java:1626)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
> > [ERROR] at 
> > org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:128)
> > [ERROR] at 
> > org.testng.TestRunner.initMethods(TestRunner.java:413)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:239)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:209)
> > [ERROR] at org.testng.TestRunner.(TestRunner.java:157)
> > [ERROR] at 
> > org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
> > [ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:159)
> > [ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:113)
> > [ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
> > [ERROR] at 
> > org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
> > [ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
> > [ERROR] at org.testng.TestNG.run(TestNG.java:1032)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:113)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.main(Fo

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-28 Thread Pierre Padovani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/
---

(Updated March 28, 2018, 3:28 p.m.)


Review request for atlas and David Radley.


Changes
---

This update resolves build issues when using the -Pdist flag. In master, the 
webapp pom.xml leverages the profiles as part of building the war. For the 
-Pdist profile, it will exclude three jars that are required for the 
integration tests to pass. Berkleyje, solr-test-framework, and jts. I've 
updated the pom to copy the three needed dependencies into the 
target/test-classes directory, and append the three jars to the webapp 
classpath. This allows the build to pass regardless of the profile used.


Bugs: ATLAS-2470
https://issues.apache.org/jira/browse/ATLAS-2470


Repository: atlas


Description
---

This updates pom's to add the needed cassandra jars, and adds a dist profile 
embedded-cassandra-solr.


Diffs (updated)
-

  addons/storm-bridge-shim/pom.xml c25fbfe5 
  addons/storm-bridge/pom.xml 2627d92e 
  distro/pom.xml 0103bef6 
  distro/src/bin/atlas_config.py e6415cf4 
  distro/src/bin/atlas_start.py 39be6b7c 
  distro/src/bin/atlas_stop.py 66edd904 
  distro/src/conf/atlas-application.properties 10fb7b86 
  distro/src/conf/atlas-env.sh 68b24e93 
  distro/src/conf/cassandra.yml.template PRE-CREATION 
  distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
  distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
  distro/src/main/assemblies/standalone-package.xml 1881082e 
  docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
  graphdb/janus/pom.xml 143b775f 
  pom.xml 9ddc8e18 
  repository/pom.xml b1d6b1f9 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 cee7374e 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 4d70b7f7 
  
repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
 PRE-CREATION 
  repository/src/test/resources/cassandra_test.yml PRE-CREATION 
  
server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
 7d282b30 
  test-tools/src/main/resources/solr/solr.xml 041f0058 
  webapp/pom.xml 284f538f 


Diff: https://reviews.apache.org/r/65885/diff/6/

Changes: https://reviews.apache.org/r/65885/diff/5-6/


Testing
---

Full build with the new embedded-cassandra-solr, and testing to make sure Atlas 
comes up and is functional.

Be aware that we have been running Cassandra backed JanusGraph for months with 
no issues.

This patch now includes support for Cassandra as an audit store. I've also 
refactored the audit unit tests, and added cassandra specific tests.


Thanks,

Pierre Padovani

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-27 Thread Pierre Padovani



> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > pom.xml
> > Line 673 (original), 673 (patched)
> > 
> >
> > I did a build with  :"mvn clean  package -Pdist" without your patch and 
> > it runs through cleanly.
> > 
> > I apply your patch and build with "mvn clean  package 
> > -Pdist,embedded-cassandra-solr" and get a build failure - details below. 
> > During the integration tests, it tries to spin up an Atlas and an embedded 
> > solr. This works slightly differently depending on the build options. I 
> > suspect that there is something in the integration tests for the embedded 
> > cassandra build that is not set up correctly so it is not fidning the 
> > embdedd solr. 
> > 
> >  
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 04:31 min
> > [INFO] Finished at: 2018-03-26T13:49:19+01:00
> > [INFO] Final Memory: 146M/1574M
> > [INFO] 
> > 
> > [ERROR] Failed to execute goal 
> > org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) 
> > on project atlas-repository: ExecutionException: 
> > java.lang.RuntimeException: There was an error in the forked process
> > [ERROR] java.lang.ExceptionInInitializerError
> > [ERROR] at java.lang.Class.forName0(Native Method)
> > [ERROR] at java.lang.Class.forName(Class.java:264)
> > [ERROR] at 
> > org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
> > [ERROR] at 
> > org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
> > [ERROR] at 
> > com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> > [ERROR] at 
> > com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> > [ERROR] at 
> > com.google.inject.spi.Elements.getElements(Elements.java:110)
> > [ERROR] at 
> > com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> > [ERROR] at 
> > com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> > [ERROR] at 
> > com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
> > [ERROR] at 
> > org.testng.TestRunner.getInjector(TestRunner.java:1626)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
> > [ERROR] at 
> > org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:128)
> > [ERROR] at 
> > org.testng.TestRunner.initMethods(TestRunner.java:413)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:239)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:209)
> > [ERROR] at org.testng.TestRunner.(TestRunner.java:157)
> > [ERROR] at 
> > org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
> > [ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:159)
> > [ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:113)
> > [ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
> > [ERROR] at 
> > org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
> > [ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
> > [ERROR] at org.testng.TestNG.run(TestNG.java:1032)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:113)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.main(Fo

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-26 Thread Pierre Padovani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/
---

(Updated March 26, 2018, 8:07 p.m.)


Review request for atlas and David Radley.


Changes
---

Resolved all indicated issues from review. Of special note, changes were made 
to the storm bridge/shim pom files to force the dropwizard metric jars to match 
the version used by other pieces in Atlas. This avoids random test failures due 
to classpath loading ordering.


Bugs: ATLAS-2470
https://issues.apache.org/jira/browse/ATLAS-2470


Repository: atlas


Description
---

This updates pom's to add the needed cassandra jars, and adds a dist profile 
embedded-cassandra-solr.


Diffs (updated)
-

  addons/storm-bridge-shim/pom.xml c25fbfe5 
  addons/storm-bridge/pom.xml 2627d92e 
  distro/pom.xml 0103bef6 
  distro/src/bin/atlas_config.py e6415cf4 
  distro/src/bin/atlas_start.py 39be6b7c 
  distro/src/bin/atlas_stop.py 66edd904 
  distro/src/conf/atlas-application.properties 10fb7b86 
  distro/src/conf/atlas-env.sh 68b24e93 
  distro/src/conf/cassandra.yml.template PRE-CREATION 
  distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
  distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
  distro/src/main/assemblies/standalone-package.xml 1881082e 
  docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
  graphdb/janus/pom.xml 143b775f 
  pom.xml 9ddc8e18 
  repository/pom.xml b1d6b1f9 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 cee7374e 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 4d70b7f7 
  
repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
 PRE-CREATION 
  repository/src/test/resources/cassandra_test.yml PRE-CREATION 
  
server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
 7d282b30 
  test-tools/src/main/resources/solr/solr.xml 041f0058 


Diff: https://reviews.apache.org/r/65885/diff/5/

Changes: https://reviews.apache.org/r/65885/diff/4-5/


Testing
---

Full build with the new embedded-cassandra-solr, and testing to make sure Atlas 
comes up and is functional.

Be aware that we have been running Cassandra backed JanusGraph for months with 
no issues.

This patch now includes support for Cassandra as an audit store. I've also 
refactored the audit unit tests, and added cassandra specific tests.


Thanks,

Pierre Padovani

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-26 Thread Pierre Padovani



> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > pom.xml
> > Line 673 (original), 673 (patched)
> > 
> >
> > I did a build with  :"mvn clean  package -Pdist" without your patch and 
> > it runs through cleanly.
> > 
> > I apply your patch and build with "mvn clean  package 
> > -Pdist,embedded-cassandra-solr" and get a build failure - details below. 
> > During the integration tests, it tries to spin up an Atlas and an embedded 
> > solr. This works slightly differently depending on the build options. I 
> > suspect that there is something in the integration tests for the embedded 
> > cassandra build that is not set up correctly so it is not fidning the 
> > embdedd solr. 
> > 
> >  
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 04:31 min
> > [INFO] Finished at: 2018-03-26T13:49:19+01:00
> > [INFO] Final Memory: 146M/1574M
> > [INFO] 
> > 
> > [ERROR] Failed to execute goal 
> > org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) 
> > on project atlas-repository: ExecutionException: 
> > java.lang.RuntimeException: There was an error in the forked process
> > [ERROR] java.lang.ExceptionInInitializerError
> > [ERROR] at java.lang.Class.forName0(Native Method)
> > [ERROR] at java.lang.Class.forName(Class.java:264)
> > [ERROR] at 
> > org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
> > [ERROR] at 
> > org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
> > [ERROR] at 
> > com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> > [ERROR] at 
> > com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> > [ERROR] at 
> > com.google.inject.spi.Elements.getElements(Elements.java:110)
> > [ERROR] at 
> > com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> > [ERROR] at 
> > com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> > [ERROR] at 
> > com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
> > [ERROR] at 
> > org.testng.TestRunner.getInjector(TestRunner.java:1626)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
> > [ERROR] at 
> > org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:128)
> > [ERROR] at 
> > org.testng.TestRunner.initMethods(TestRunner.java:413)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:239)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:209)
> > [ERROR] at org.testng.TestRunner.(TestRunner.java:157)
> > [ERROR] at 
> > org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
> > [ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:159)
> > [ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:113)
> > [ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
> > [ERROR] at 
> > org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
> > [ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
> > [ERROR] at org.testng.TestNG.run(TestNG.java:1032)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:113)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.main(Fo

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-26 Thread Pierre Padovani



> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > docs/src/site/twiki/InstallationSteps.twiki
> > Lines 43 (patched)
> > 
> >
> > I suggest mentioningthat you require the MANAGE_LOCAL_SOLR variable to 
> > be set.

This environment variable is managed for you as part of the build. There is an 
issue: https://issues.apache.org/jira/browse/ATLAS-832 which discusses the 
issue. I found that the environment variables in atlas_config.py were not setup 
correctly.


> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > pom.xml
> > Line 673 (original), 673 (patched)
> > 
> >
> > I did a build with  :"mvn clean  package -Pdist" without your patch and 
> > it runs through cleanly.
> > 
> > I apply your patch and build with "mvn clean  package 
> > -Pdist,embedded-cassandra-solr" and get a build failure - details below. 
> > During the integration tests, it tries to spin up an Atlas and an embedded 
> > solr. This works slightly differently depending on the build options. I 
> > suspect that there is something in the integration tests for the embedded 
> > cassandra build that is not set up correctly so it is not fidning the 
> > embdedd solr. 
> > 
> >  
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 04:31 min
> > [INFO] Finished at: 2018-03-26T13:49:19+01:00
> > [INFO] Final Memory: 146M/1574M
> > [INFO] 
> > 
> > [ERROR] Failed to execute goal 
> > org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) 
> > on project atlas-repository: ExecutionException: 
> > java.lang.RuntimeException: There was an error in the forked process
> > [ERROR] java.lang.ExceptionInInitializerError
> > [ERROR] at java.lang.Class.forName0(Native Method)
> > [ERROR] at java.lang.Class.forName(Class.java:264)
> > [ERROR] at 
> > org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
> > [ERROR] at 
> > org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
> > [ERROR] at 
> > com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> > [ERROR] at 
> > com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> > [ERROR] at 
> > com.google.inject.spi.Elements.getElements(Elements.java:110)
> > [ERROR] at 
> > com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> > [ERROR] at 
> > com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> > [ERROR] at 
> > com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
> > [ERROR] at 
> > org.testng.TestRunner.getInjector(TestRunner.java:1626)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
> > [ERROR] at 
> > org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:128)
> > [ERROR] at 
> > org.testng.TestRunner.initMethods(TestRunner.java:413)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:239)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:209)
> > [ERROR] at org.testng.TestRunner.(TestRunner.java:157)
> > [ERROR] at 
> > org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
> > [ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:159)
> > [ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:113)
> > [ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
> > [ERROR] at 
> > org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
> > [ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
> > [ERROR] at org.testng.TestNG.run(TestNG.java:1032)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
> > [ERROR] at 
> > o

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-26 Thread David Radley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199964
---




distro/src/conf/cassandra.yml.template
Lines 21 (patched)


It is not possible to build atlas with a stand alone Cassandra. Also there 
isa typo STANDALONG.



distro/src/conf/cassandra.yml.template
Lines 27 (patched)


we do not seem to be using a end tag convntion - I suggest removing.



distro/src/conf/zookeeper/zoo.cfg.template
Lines 20 (patched)


bad eol character



docs/src/site/twiki/InstallationSteps.twiki
Lines 41 (patched)


I  do not think we should be recommending building without tests.



docs/src/site/twiki/InstallationSteps.twiki
Lines 43 (patched)


I suggest mentioningthat you require the MANAGE_LOCAL_SOLR variable to be 
set.



pom.xml
Line 673 (original), 673 (patched)


I did a build with  :"mvn clean  package -Pdist" without your patch and it 
runs through cleanly.

I apply your patch and build with "mvn clean  package 
-Pdist,embedded-cassandra-solr" and get a build failure - details below. During 
the integration tests, it tries to spin up an Atlas and an embedded solr. This 
works slightly differently depending on the build options. I suspect that there 
is something in the integration tests for the embedded cassandra build that is 
not set up correctly so it is not fidning the embdedd solr. 

 
[INFO] BUILD FAILURE
[INFO] 

[INFO] Total time: 04:31 min
[INFO] Finished at: 2018-03-26T13:49:19+01:00
[INFO] Final Memory: 146M/1574M
[INFO] 

[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on 
project atlas-repository: ExecutionException: java.lang.RuntimeException: There 
was an error in the forked process
[ERROR] java.lang.ExceptionInInitializerError
[ERROR] at java.lang.Class.forName0(Native Method)
[ERROR] at java.lang.Class.forName(Class.java:264)
[ERROR] at 
org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
[ERROR] at 
org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
[ERROR] at 
org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
[ERROR] at 
org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
[ERROR] at 
com.google.inject.AbstractModule.configure(AbstractModule.java:62)
[ERROR] at 
com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
[ERROR] at com.google.inject.spi.Elements.getElements(Elements.java:110)
[ERROR] at 
com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
[ERROR] at 
com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
[ERROR] at 
com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
[ERROR] at org.testng.TestRunner.getInjector(TestRunner.java:1626)
[ERROR] at 
org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
[ERROR] at 
org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
[ERROR] at 
org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
[ERROR] at 
org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:128)
[ERROR] at org.testng.TestRunner.initMethods(TestRunner.java:413)
[ERROR] at org.testng.TestRunner.init(TestRunner.java:239)
[ERROR] at org.testng.TestRunner.init(TestRunner.java:209)
[ERROR] at org.testng.TestRunner.(TestRunner.java:157)
[ERROR] at 
org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
[ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:159)
[ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:113)
[ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
[ERROR] at org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
[ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
[ERROR] at org.testng.TestNG.run(TestNG.java:1032)
[ERROR] at 
org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
[ERROR] at 
org.apache.maven.surefire.testng.TestNGDirectoryTes

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-19 Thread Pierre Padovani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/
---

(Updated March 20, 2018, 12:04 a.m.)


Review request for atlas and David Radley.


Changes
---

Fixed the injection of the logback jars in the distribution, they no longer are 
being pulled in, and logging has returned to normal.

I attempted to do a full build with tests, but the tests in repository failed. 
I do not believe the failure is due to this patch, as the same exact failure 
happens in master. The cassandra based audit tests do pass even if the 
subsequent tests are failing.

As an additional test on this patch, I applied the patch to a clean master with:

git am < ATLAS=2470-4.patch

It did give some warnings on whitespace, but properly applied the patch. I then 
built and ran Atlas with embedded-cassandra-solr and it started up with no 
errors.


Bugs: ATLAS-2470
https://issues.apache.org/jira/browse/ATLAS-2470


Repository: atlas


Description
---

This updates pom's to add the needed cassandra jars, and adds a dist profile 
embedded-cassandra-solr.


Diffs (updated)
-

  distro/pom.xml 0103bef6 
  distro/src/bin/atlas_config.py e6415cf4 
  distro/src/bin/atlas_start.py 39be6b7c 
  distro/src/bin/atlas_stop.py 66edd904 
  distro/src/conf/atlas-application.properties 10fb7b86 
  distro/src/conf/atlas-env.sh 68b24e93 
  distro/src/conf/cassandra.yml.template PRE-CREATION 
  distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
  distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
  distro/src/main/assemblies/standalone-package.xml 1881082e 
  docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
  graphdb/janus/pom.xml 143b775f 
  pom.xml 9ddc8e18 
  repository/pom.xml b1d6b1f9 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 cee7374e 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 4d70b7f7 
  
repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
 PRE-CREATION 
  repository/src/test/resources/cassandra_test.yml PRE-CREATION 
  
server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
 7d282b30 


Diff: https://reviews.apache.org/r/65885/diff/4/

Changes: https://reviews.apache.org/r/65885/diff/3-4/


Testing
---

Full build with the new embedded-cassandra-solr, and testing to make sure Atlas 
comes up and is functional.

Be aware that we have been running Cassandra backed JanusGraph for months with 
no issues.

This patch now includes support for Cassandra as an audit store. I've also 
refactored the audit unit tests, and added cassandra specific tests.


Thanks,

Pierre Padovani

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-19 Thread Pierre Padovani



> On March 19, 2018, 7:28 p.m., David Radley wrote:
> > docs/src/site/twiki/InstallationSteps.twiki
> > Lines 45 (patched)
> > 
> >
> > I built Atlas as per these instructions. I then started atlas. Atlas 
> > started and worked - but when I looked at the application.log there was 
> > only 3 log entries.  
> > 
> > 2018-03-19 12:22:22,447 WARN  - [main:] ~ No meta.properties file under 
> > dir 
> > /Users/david/atlasreview2/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/data/kafka/kafka/meta.properties
> >  (Logging$class:87)
> > 2018-03-19 12:22:22,922 WARN  - [main:] ~ No meta.properties file under 
> > dir 
> > /Users/david/atlasreview2/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/data/kafka/kafka/meta.properties
> >  (Logging$class:87)
> > 2018-03-19 12:22:22,986 INFO  - [NotificationHookConsumer thread-0:] ~ 
> > [atlas-hook-consumer-thread]: Starting (Logging$class:72)
> > 
> > 
> > I would have expected much more logging.
> 
> Pierre Padovani wrote:
> This confused me as well. All of the logging is in the the newest 
> atlas..out file. Which makes sense when log4j is configured to 
> log to a rolling date/time stamped file. In previous builds of Atlas the 
> logging all showed up in the application.log file. This change in behavior 
> happened to me within the last few weeks. Do we know if a change was made to 
> induce this? or Was this induced by my changes?
> 
> I'll do some testing on my side.
> 
> Pierre Padovani wrote:
> Just retested off of master... something in the cassandra jars for Janus 
> must be overriding the Atlas logging configuration. I'll investigate.

Ok... the logback jars are being pulled into the distro... If you remove them 
from server/webapp/atlas/WEB-INF/lib you should have the expected behavior. 
I'll work on updating the patch to exclude those jars.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199456
---


On March 19, 2018, 11:51 a.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 19, 2018, 11:51 a.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 10fb7b86 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/3/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-19 Thread Pierre Padovani



> On March 19, 2018, 7:28 p.m., David Radley wrote:
> > docs/src/site/twiki/InstallationSteps.twiki
> > Lines 45 (patched)
> > 
> >
> > I built Atlas as per these instructions. I then started atlas. Atlas 
> > started and worked - but when I looked at the application.log there was 
> > only 3 log entries.  
> > 
> > 2018-03-19 12:22:22,447 WARN  - [main:] ~ No meta.properties file under 
> > dir 
> > /Users/david/atlasreview2/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/data/kafka/kafka/meta.properties
> >  (Logging$class:87)
> > 2018-03-19 12:22:22,922 WARN  - [main:] ~ No meta.properties file under 
> > dir 
> > /Users/david/atlasreview2/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/data/kafka/kafka/meta.properties
> >  (Logging$class:87)
> > 2018-03-19 12:22:22,986 INFO  - [NotificationHookConsumer thread-0:] ~ 
> > [atlas-hook-consumer-thread]: Starting (Logging$class:72)
> > 
> > 
> > I would have expected much more logging.
> 
> Pierre Padovani wrote:
> This confused me as well. All of the logging is in the the newest 
> atlas..out file. Which makes sense when log4j is configured to 
> log to a rolling date/time stamped file. In previous builds of Atlas the 
> logging all showed up in the application.log file. This change in behavior 
> happened to me within the last few weeks. Do we know if a change was made to 
> induce this? or Was this induced by my changes?
> 
> I'll do some testing on my side.

Just retested off of master... something in the cassandra jars for Janus must 
be overriding the Atlas logging configuration. I'll investigate.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199456
---


On March 19, 2018, 11:51 a.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 19, 2018, 11:51 a.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 10fb7b86 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/3/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-19 Thread Pierre Padovani



> On March 19, 2018, 7:28 p.m., David Radley wrote:
> > docs/src/site/twiki/InstallationSteps.twiki
> > Lines 45 (patched)
> > 
> >
> > I built Atlas as per these instructions. I then started atlas. Atlas 
> > started and worked - but when I looked at the application.log there was 
> > only 3 log entries.  
> > 
> > 2018-03-19 12:22:22,447 WARN  - [main:] ~ No meta.properties file under 
> > dir 
> > /Users/david/atlasreview2/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/data/kafka/kafka/meta.properties
> >  (Logging$class:87)
> > 2018-03-19 12:22:22,922 WARN  - [main:] ~ No meta.properties file under 
> > dir 
> > /Users/david/atlasreview2/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/data/kafka/kafka/meta.properties
> >  (Logging$class:87)
> > 2018-03-19 12:22:22,986 INFO  - [NotificationHookConsumer thread-0:] ~ 
> > [atlas-hook-consumer-thread]: Starting (Logging$class:72)
> > 
> > 
> > I would have expected much more logging.

This confused me as well. All of the logging is in the the newest atlas..out file. Which makes sense when log4j is configured to log to a rolling 
date/time stamped file. In previous builds of Atlas the logging all showed up 
in the application.log file. This change in behavior happened to me within the 
last few weeks. Do we know if a change was made to induce this? or Was this 
induced by my changes?

I'll do some testing on my side.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199456
---


On March 19, 2018, 11:51 a.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 19, 2018, 11:51 a.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 10fb7b86 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/3/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-19 Thread David Radley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199456
---




docs/src/site/twiki/InstallationSteps.twiki
Lines 45 (patched)


I built Atlas as per these instructions. I then started atlas. Atlas 
started and worked - but when I looked at the application.log there was only 3 
log entries.  

2018-03-19 12:22:22,447 WARN  - [main:] ~ No meta.properties file under dir 
/Users/david/atlasreview2/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/data/kafka/kafka/meta.properties
 (Logging$class:87)
2018-03-19 12:22:22,922 WARN  - [main:] ~ No meta.properties file under dir 
/Users/david/atlasreview2/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/data/kafka/kafka/meta.properties
 (Logging$class:87)
2018-03-19 12:22:22,986 INFO  - [NotificationHookConsumer thread-0:] ~ 
[atlas-hook-consumer-thread]: Starting (Logging$class:72)

I would have expected much more logging.


- David Radley


On March 19, 2018, 11:51 a.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 19, 2018, 11:51 a.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 10fb7b86 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/3/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-19 Thread Pierre Padovani



> On March 19, 2018, 12:31 a.m., David Radley wrote:
> > distro/src/conf/atlas-application.properties
> > Line 23 (original), 23 (patched)
> > 
> >
> > Sorry for the delay in reviewing. Unfortunatley the patch now does not 
> > apply to master I get: 
> > git am < ../ATLAS-2470-2.patch 
> > Applying: ATLAS-2470 - JanusGraph Cassandra support
> > .git/rebase-apply/patch:1146: trailing whitespace.
> > # The number of ticks that the initial 
> > error: patch failed: distro/src/conf/atlas-application.properties:24
> > error: distro/src/conf/atlas-application.properties: patch does not 
> > apply
> > .git/rebase-apply/patch:1039: new blank line at EOF.
> > +
> > .git/rebase-apply/patch:2761: new blank line at EOF.
> > +
> > Patch failed at 0001 ATLAS-2470 - JanusGraph Cassandra support

Ok.. I uploaded a new diff after rebasing off of master. However it still 
cannot be applied for similar reasons. A quick google soved my issues...

git am --3way < ATLAS-2470-3.patch

properly applies the patch.

https://www.lullabot.com/articles/git-best-practices-upgrading-the-patch-process


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199398
---


On March 19, 2018, 11:51 a.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 19, 2018, 11:51 a.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 10fb7b86 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/3/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-19 Thread Pierre Padovani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/
---

(Updated March 19, 2018, 11:51 a.m.)


Review request for atlas and David Radley.


Changes
---

Rebased off of master and regenerated the patch. Git is weird... a rebase 
showed no conflicts, but attempting to apply the previous version of the patch 
did.


Bugs: ATLAS-2470
https://issues.apache.org/jira/browse/ATLAS-2470


Repository: atlas


Description
---

This updates pom's to add the needed cassandra jars, and adds a dist profile 
embedded-cassandra-solr.


Diffs (updated)
-

  distro/pom.xml 0103bef6 
  distro/src/bin/atlas_config.py e6415cf4 
  distro/src/bin/atlas_start.py 39be6b7c 
  distro/src/bin/atlas_stop.py 66edd904 
  distro/src/conf/atlas-application.properties 10fb7b86 
  distro/src/conf/atlas-env.sh 68b24e93 
  distro/src/conf/cassandra.yml.template PRE-CREATION 
  distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
  distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
  distro/src/main/assemblies/standalone-package.xml 1881082e 
  docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
  graphdb/janus/pom.xml 143b775f 
  repository/pom.xml b1d6b1f9 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 cee7374e 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 4d70b7f7 
  
repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
 PRE-CREATION 
  repository/src/test/resources/cassandra_test.yml PRE-CREATION 
  
server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
 7d282b30 


Diff: https://reviews.apache.org/r/65885/diff/3/

Changes: https://reviews.apache.org/r/65885/diff/2-3/


Testing
---

Full build with the new embedded-cassandra-solr, and testing to make sure Atlas 
comes up and is functional.

Be aware that we have been running Cassandra backed JanusGraph for months with 
no issues.

This patch now includes support for Cassandra as an audit store. I've also 
refactored the audit unit tests, and added cassandra specific tests.


Thanks,

Pierre Padovani

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-18 Thread David Radley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199398
---




distro/src/conf/atlas-application.properties
Line 23 (original), 23 (patched)


Sorry for the delay in reviewing. Unfortunatley the patch now does not 
apply to master I get: 
git am < ../ATLAS-2470-2.patch 
Applying: ATLAS-2470 - JanusGraph Cassandra support
.git/rebase-apply/patch:1146: trailing whitespace.
# The number of ticks that the initial 
error: patch failed: distro/src/conf/atlas-application.properties:24
error: distro/src/conf/atlas-application.properties: patch does not apply
.git/rebase-apply/patch:1039: new blank line at EOF.
+
.git/rebase-apply/patch:2761: new blank line at EOF.
+
Patch failed at 0001 ATLAS-2470 - JanusGraph Cassandra support


- David Radley


On March 8, 2018, 6:53 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 8, 2018, 6:53 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 355abf44 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/2/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-09 Thread Pierre Padovani



> On March 9, 2018, 1:38 a.m., Apoorv Naik wrote:
> > distro/pom.xml
> > Lines 252 (patched)
> > 
> >
> > Although harmless, it's better to even avoid download if it's not 
> > needed. It's always a good thing to avoid unnecessary jars in the final 
> > distro, makes troubleshooting easier. 
> > 
> > Also this would increase the build time, why to waste CPU/Network 
> > resources on something if it's not needed.

This was removed from the habase profile in the current patch. The directions 
for updating a patch were unclear on the wiki, and I had forgotten to update 
the patch on the review board.


> On March 9, 2018, 1:38 a.m., Apoorv Naik wrote:
> > distro/src/main/assemblies/standalone-package.xml
> > Lines 83 (patched)
> > 
> >
> > ZK might be available in certain deployments and this would add 
> > unnecessary ZK jars to classpath. Potential to cause mess ups in production 
> > environments.

I'm not sure this is the case. The zk directory shows up at the root of the 
atlas directory, not within the server classpath directory. Additionally, the 
updated patch only downloads it and installs it if the embedded-cassandra-solr 
profile is used. So this particular operation is a noop unless that profile is 
used.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198893
---


On March 8, 2018, 6:53 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 8, 2018, 6:53 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 355abf44 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/2/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-08 Thread Apoorv Naik


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198893
---




distro/pom.xml
Lines 252 (patched)


Although harmless, it's better to even avoid download if it's not needed. 
It's always a good thing to avoid unnecessary jars in the final distro, makes 
troubleshooting easier. 

Also this would increase the build time, why to waste CPU/Network resources 
on something if it's not needed.



distro/src/main/assemblies/standalone-package.xml
Lines 83 (patched)


ZK might be available in certain deployments and this would add unnecessary 
ZK jars to classpath. Potential to cause mess ups in production environments.


- Apoorv Naik


On March 8, 2018, 6:53 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 8, 2018, 6:53 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 355abf44 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   repository/pom.xml b1d6b1f9 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  cee7374e 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  4d70b7f7 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
>  PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   
> server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
>  7d282b30 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/2/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also 
> refactored the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-08 Thread Pierre Padovani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/
---

(Updated March 8, 2018, 6:53 p.m.)


Review request for atlas and David Radley.


Bugs: ATLAS-2470
https://issues.apache.org/jira/browse/ATLAS-2470


Repository: atlas


Description
---

This updates pom's to add the needed cassandra jars, and adds a dist profile 
embedded-cassandra-solr.


Diffs
-

  distro/pom.xml 0103bef6 
  distro/src/bin/atlas_config.py e6415cf4 
  distro/src/bin/atlas_start.py 39be6b7c 
  distro/src/bin/atlas_stop.py 66edd904 
  distro/src/conf/atlas-application.properties 355abf44 
  distro/src/conf/atlas-env.sh 68b24e93 
  distro/src/conf/cassandra.yml.template PRE-CREATION 
  distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
  distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
  distro/src/main/assemblies/standalone-package.xml 1881082e 
  docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
  graphdb/janus/pom.xml 143b775f 
  repository/pom.xml b1d6b1f9 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 cee7374e 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 4d70b7f7 
  
repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
 PRE-CREATION 
  repository/src/test/resources/cassandra_test.yml PRE-CREATION 
  
server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
 7d282b30 


Diff: https://reviews.apache.org/r/65885/diff/2/


Testing (updated)
---

Full build with the new embedded-cassandra-solr, and testing to make sure Atlas 
comes up and is functional.

Be aware that we have been running Cassandra backed JanusGraph for months with 
no issues.

This patch now includes support for Cassandra as an audit store. I've also 
refactored the audit unit tests, and added cassandra specific tests.


Thanks,

Pierre Padovani

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-08 Thread Pierre Padovani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/
---

(Updated March 8, 2018, 6:51 p.m.)


Review request for atlas and David Radley.


Bugs: ATLAS-2470
https://issues.apache.org/jira/browse/ATLAS-2470


Repository: atlas


Description
---

This updates pom's to add the needed cassandra jars, and adds a dist profile 
embedded-cassandra-solr.


Diffs (updated)
-

  distro/pom.xml 0103bef6 
  distro/src/bin/atlas_config.py e6415cf4 
  distro/src/bin/atlas_start.py 39be6b7c 
  distro/src/bin/atlas_stop.py 66edd904 
  distro/src/conf/atlas-application.properties 355abf44 
  distro/src/conf/atlas-env.sh 68b24e93 
  distro/src/conf/cassandra.yml.template PRE-CREATION 
  distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
  distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
  distro/src/main/assemblies/standalone-package.xml 1881082e 
  docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
  graphdb/janus/pom.xml 143b775f 
  repository/pom.xml b1d6b1f9 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 cee7374e 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 4d70b7f7 
  
repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
 PRE-CREATION 
  repository/src/test/resources/cassandra_test.yml PRE-CREATION 
  
server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
 7d282b30 


Diff: https://reviews.apache.org/r/65885/diff/2/

Changes: https://reviews.apache.org/r/65885/diff/1-2/


Testing
---

Full build with the new embedded-cassandra-solr, and testing to make sure Atlas 
comes up and is functional.

Be aware that we have been running Cassandra backed JanusGraph for months with 
no issues.


Thanks,

Pierre Padovani

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-08 Thread Pierre Padovani



> On March 8, 2018, 6:02 p.m., Apoorv Naik wrote:
> > distro/pom.xml
> > Lines 252 (patched)
> > 
> >
> > Looks like this will be run for all profiles, can you make this 
> > specific to cassandra profile ?

Does it matter here? This just downloads the resource. It will only be used if 
the embedded-cassandra-solr dist profile is used, otherwise the download is 
ignored.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198890
---


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-08 Thread Apoorv Naik


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198890
---




distro/pom.xml
Lines 252 (patched)


Looks like this will be run for all profiles, can you make this specific to 
cassandra profile ?


- Apoorv Naik


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-08 Thread Pierre Padovani



> On March 4, 2018, 11:07 a.m., David Radley wrote:
> > My review comments are from my initial look at the code; I hope to try 
> > running this patch to verify it works for me
> 
> Pierre Padovani wrote:
> I'll update the patch with the above changes today or tomorrow as I have 
> time.
> 
> Pierre Padovani wrote:
> I've updated the ATLAS-2470 with a new patch that should fix all of these 
> issues.
> 
> Pierre Padovani wrote:
> Hi David, I think I'm missing something. The instructions on the 'Using 
> GIT with Atlas' page are unclear on patch revisions. Do I create another 
> review, or do I somehow update this review?
> 
> David Radley wrote:
> It is best to use the same review for subsequent patches- so then it is 
> possible to compare between versions. I have updated the wiki with this 
> information - thanks.

Updated with a new version of the patch that now includes Audit support.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198593
---


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-05 Thread David Radley



> On March 4, 2018, 11:04 a.m., David Radley wrote:
> > distro/pom.xml
> > Lines 302 (patched)
> > 
> >
> > for consistency should we have  as well?
> 
> Pierre Padovani wrote:
> You cannot just have cassandra without a solr/elasticsearch install 
> unless you are using DSE (the Datastax Enterprise with integrates with solr). 
> Based on cursory reading over on JanusGraph, they do not recommend production 
> deployments of JanusGraph with Cassandra embedded as the performance profiles 
> are not entirely predictable. Again I would address this in the main 
> documentation task.

Yes I was thinking that cassandra-embedded to differentiate between the 
production version - maybe you can add this when the extenral cassandra support 
is put in.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198584
---


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-05 Thread David Radley



> On March 4, 2018, 11:07 a.m., David Radley wrote:
> > My review comments are from my initial look at the code; I hope to try 
> > running this patch to verify it works for me
> 
> Pierre Padovani wrote:
> I'll update the patch with the above changes today or tomorrow as I have 
> time.
> 
> Pierre Padovani wrote:
> I've updated the ATLAS-2470 with a new patch that should fix all of these 
> issues.
> 
> Pierre Padovani wrote:
> Hi David, I think I'm missing something. The instructions on the 'Using 
> GIT with Atlas' page are unclear on patch revisions. Do I create another 
> review, or do I somehow update this review?

It is best to use the same review for subsequent patches- so then it is 
possible to compare between versions. I have updated the wiki with this 
information - thanks.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198593
---


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-05 Thread Pierre Padovani



> On March 4, 2018, 11:07 a.m., David Radley wrote:
> > My review comments are from my initial look at the code; I hope to try 
> > running this patch to verify it works for me
> 
> Pierre Padovani wrote:
> I'll update the patch with the above changes today or tomorrow as I have 
> time.
> 
> Pierre Padovani wrote:
> I've updated the ATLAS-2470 with a new patch that should fix all of these 
> issues.

Hi David, I think I'm missing something. The instructions on the 'Using GIT 
with Atlas' page are unclear on patch revisions. Do I create another review, or 
do I somehow update this review?


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198593
---


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-04 Thread Pierre Padovani



> On March 4, 2018, 11:07 a.m., David Radley wrote:
> > My review comments are from my initial look at the code; I hope to try 
> > running this patch to verify it works for me
> 
> Pierre Padovani wrote:
> I'll update the patch with the above changes today or tomorrow as I have 
> time.

I've updated the ATLAS-2470 with a new patch that should fix all of these 
issues.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198593
---


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-04 Thread Pierre Padovani



> On March 4, 2018, 11:04 a.m., David Radley wrote:
> > distro/pom.xml
> > Lines 258 (patched)
> > 
> >
> > I see that you are downloading Zookeeper. Many components require 
> > zookeeper - so it may already exist on the system. I think for production 
> > you would want to run with an external Zookeeper. 
> > 
> > I suggest we allow this as a build option. I realise this jira is for 
> > the embedded Cssandra build only - but I could envisage you needing 
> > non-embedded cassandra builds as well; I thought I would suggest this here 
> > - if this is a requirement for you, you  may decide you want to address 
> > this in a separate Jira.

Yes, this was meant only for the embedded side of things, which would typically 
only happen for a development installation or a Docker image. My intention is 
to address setup and config for cassandra in a production environment in the 
documentation. I have another task specific to updating the documentation 
around this. My hope was to get this merged to master first, then I can get the 
Dockerfile built out that supports this configuration in another task.


> On March 4, 2018, 11:04 a.m., David Radley wrote:
> > distro/pom.xml
> > Lines 302 (patched)
> > 
> >
> > for consistency should we have  as well?

You cannot just have cassandra without a solr/elasticsearch install unless you 
are using DSE (the Datastax Enterprise with integrates with solr). Based on 
cursory reading over on JanusGraph, they do not recommend production 
deployments of JanusGraph with Cassandra embedded as the performance profiles 
are not entirely predictable. Again I would address this in the main 
documentation task.


> On March 4, 2018, 11:04 a.m., David Radley wrote:
> > distro/src/bin/atlas_start.py
> > Lines 121 (patched)
> > 
> >
> > For an embedded hbase build, it will use the hbase zk and the embeded 
> > solr. I assume this line ( and the matching stop) should not be called for 
> > the embedded hbase build (which is not using the downloaded zk).

I could swear I tested this, and it required I spin up a zookeeper to get it to 
work. I just reset to master and retested (after fixing the atlas_config.py) 
and it worked without a local ZK start. So I'll remove this from the profile.


> On March 4, 2018, 11:04 a.m., David Radley wrote:
> > docs/src/site/twiki/InstallationSteps.twiki
> > Lines 38 (patched)
> > 
> >
> > We did not have to do with before this change for embedded hbase solr. 
> > I think these notes should not apply to the embedded hbase solr profile.

Yep, as stated before I'll remove it, and find a way to only do it for a 
cassandra deploy.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198584
---


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-04 Thread Pierre Padovani



> On March 4, 2018, 11:07 a.m., David Radley wrote:
> > My review comments are from my initial look at the code; I hope to try 
> > running this patch to verify it works for me

I'll update the patch with the above changes today or tomorrow as I have time.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198593
---


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-04 Thread David Radley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198593
---



My review comments are from my initial look at the code; I hope to try running 
this patch to verify it works for me

- David Radley


On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-04 Thread David Radley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review198584
---

distro/pom.xml
Lines 258 (patched)

I see that you are downloading Zookeeper. Many components require zookeeper 
- so it may already exist on the system. I think for production you would want 
to run with an external Zookeeper. 

I suggest we allow this as a build option. I realise this jira is for the 
embedded Cssandra build only - but I could envisage you needing non-embedded 
cassandra builds as well; I thought I would suggest this here - if this is a 
requirement for you, you  may decide you want to address this in a separate 
Jira.

distro/pom.xml
Lines 302 (patched)

for consistency should we have  as well?

distro/src/bin/atlas_start.py
Lines 121 (patched)

For an embedded hbase build, it will use the hbase zk and the embeded solr. 
I assume this line ( and the matching stop) should not be called for the 
embedded hbase build (which is not using the downloaded zk).

distro/src/conf/cassandra.yml
Lines 22 (patched)

typo

distro/src/conf/cassandra.yml
Lines 24 (patched)

I suggest the install could text replace these to the correct values. Is it 
possible for it to refer to an environment variable with the absolute path, 
then the script pick up that environment variable?

distro/src/conf/zookeeper/zoo.cfg
Lines 20 (patched)

bad end of line character here and a few places below

docs/src/site/twiki/InstallationSteps.twiki
Lines 38 (patched)

We did not have to do with before this change for embedded hbase solr. I 
think these notes should not apply to the embedded hbase solr profile.

- David Radley

On March 2, 2018, 4:51 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> ---
> 
> (Updated March 2, 2018, 4:51 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
> https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile 
> embedded-cassandra-solr.
> 
> 
> Diffs
> -
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/cassandra.yml PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/1/
> 
> 
> Testing
> ---
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure 
> Atlas comes up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months 
> with no issues.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-02 Thread Pierre Padovani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/
---

Review request for atlas and David Radley.


Bugs: ATLAS-2470
https://issues.apache.org/jira/browse/ATLAS-2470


Repository: atlas


Description
---

This updates pom's to add the needed cassandra jars, and adds a dist profile 
embedded-cassandra-solr.


Diffs
-

  distro/pom.xml 0103bef6 
  distro/src/bin/atlas_config.py e6415cf4 
  distro/src/bin/atlas_start.py 39be6b7c 
  distro/src/bin/atlas_stop.py 66edd904 
  distro/src/conf/cassandra.yml PRE-CREATION 
  distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
  distro/src/conf/zookeeper/zoo.cfg PRE-CREATION 
  distro/src/main/assemblies/standalone-package.xml 1881082e 
  docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
  graphdb/janus/pom.xml 143b775f 


Diff: https://reviews.apache.org/r/65885/diff/1/


Testing
---

Full build with the new embedded-cassandra-solr, and testing to make sure Atlas 
comes up and is functional.

Be aware that we have been running Cassandra backed JanusGraph for months with 
no issues.


Thanks,

Pierre Padovani

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

32 matches

Site Navigation

Mail list logo

Footer information