[jira] [Commented] (ATLAS-74) Create notification framework
[ https://issues.apache.org/jira/browse/ATLAS-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630074#comment-14630074 ] Suma Shivaprasad commented on ATLAS-74: --- Can you pls upload a reviewboard? Create notification framework - Key: ATLAS-74 URL: https://issues.apache.org/jira/browse/ATLAS-74 Project: Atlas Issue Type: Sub-task Reporter: Shwetha G S Assignee: Shwetha G S Attachments: ATLAS-74.patch Add notification framework with kafka as default implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-59) De-duping of entities
[ https://issues.apache.org/jira/browse/ATLAS-59?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-59: - Parent Issue: ATLAS-73 (was: ATLAS-58) De-duping of entities - Key: ATLAS-59 URL: https://issues.apache.org/jira/browse/ATLAS-59 Project: Atlas Issue Type: Sub-task Reporter: Shwetha G S With hive hook posting entities to be registered in message, we need a way of de-duping the entities on some value on server. For example, for type hive_db, the de-duping should be on clustername and dbname. The clients should be able to specify what the de-duping criteria is -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-73) Notification framework
Shwetha G S created ATLAS-73: Summary: Notification framework Key: ATLAS-73 URL: https://issues.apache.org/jira/browse/ATLAS-73 Project: Atlas Issue Type: New Feature Reporter: Shwetha G S Assignee: Shwetha G S We need notification framework for two purposes: 1. The atlas hook in the external components can post the entities to be registered as a message so that the processing in hook is minimal and the entity registration is reliable 2. Need notifications on type and entity mutations which can be used in policy engines -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-74) Create notification framework
Shwetha G S created ATLAS-74: Summary: Create notification framework Key: ATLAS-74 URL: https://issues.apache.org/jira/browse/ATLAS-74 Project: Atlas Issue Type: Sub-task Reporter: Shwetha G S Assignee: Shwetha G S Add notification framework with kafka as default implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-46) Different data directory with restart
[
https://issues.apache.org/jira/browse/ATLAS-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629294#comment-14629294
]
Shwetha G S commented on ATLAS-46:
--
Yes, full path works on properties file. But commons-configuration library that
we use still doesn't support variables in properties file. That feature is
still in beta and is not production ready.
Yes, we could move to hadoop style configuration. Thats bigger change and lets
address that in another jira if required
Different data directory with restart
-
Key: ATLAS-46
URL: https://issues.apache.org/jira/browse/ATLAS-46
Project: Atlas
Issue Type: Bug
Reporter: Shwetha G S
Assignee: Shwetha G S
Priority: Critical
Fix For: trunk
Attachments: ATLAS-46.patch, application.log
Atlas start picks up different data directories - initially
./server/webapp/atlas/data and after restart ./data. This must be because of
relative path in the configuration
{noformat}
apache-atlas-0.6-incubating-SNAPSHOT$ ps -ef | grep -i atlas
504 6437 1 0 11:51AM ttys0000:07.66
/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/bin/java
-Datlas.log.dir=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/logs
-Datlas.log.file=application.log
-Datlas.home=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT
-Datlas.conf=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/conf
-Xmx1024m -classpath
/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/conf:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas/WEB-INF/classes:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/*:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/libext/*
org.apache.atlas.Main -app
/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas
apache-atlas-0.6-incubating-SNAPSHOT$ find . -name data
./server/webapp/atlas/data
./server/webapp/atlas/data/es/data
./server/webapp/atlas/lib/jquery/src/data
apache-atlas-0.6-incubating-SNAPSHOT$ ./bin/atlas_stop.py
apache-atlas-0.6-incubating-SNAPSHOT$ ps -ef | grep atlas
504 6457 1 0 11:54AM ttys0000:07.50
/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/bin/java
-Datlas.log.dir=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/logs
-Datlas.log.file=application.log
-Datlas.home=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT
-Datlas.conf=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/conf
-Xmx1024m -classpath
/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/conf:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas/WEB-INF/classes:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/*:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/libext/*
org.apache.atlas.Main -app
/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas
apache-atlas-0.6-incubating-SNAPSHOT$ find . -name data
./data
./data/es/data
./server/webapp/atlas/data
./server/webapp/atlas/data/es/data
./server/webapp/atlas/lib/jquery/src/data
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-58) Make hive hook reliable
[ https://issues.apache.org/jira/browse/ATLAS-58?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-58: - Issue Type: Task (was: New Feature) Make hive hook reliable --- Key: ATLAS-58 URL: https://issues.apache.org/jira/browse/ATLAS-58 Project: Atlas Issue Type: Task Reporter: Shwetha G S Assignee: Shwetha G S Currently, hive hook executes in background thread pool and is an best effort approach to register entities. But this needs to be reliable for data governance to be effective One way is - in hive hook, add the entities to some messaging framework and atlas server can read the entities from the message and register in atlas. Since, posting message is faster, we can do it synchronously and hence reliable entity registration. We can start with kafka for messaging, but any other messaging framework should be pluggable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-74) Create notification framework
[ https://issues.apache.org/jira/browse/ATLAS-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-74: - Attachment: ATLAS-74.patch Create notification framework - Key: ATLAS-74 URL: https://issues.apache.org/jira/browse/ATLAS-74 Project: Atlas Issue Type: Sub-task Reporter: Shwetha G S Assignee: Shwetha G S Attachments: ATLAS-74.patch Add notification framework with kafka as default implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-72) Use atlas.rest.address on server
[ https://issues.apache.org/jira/browse/ATLAS-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629737#comment-14629737 ] Erik Bergenholtz commented on ATLAS-72: --- There are [at least] 3 factors that should be considered: 1. User Experience 2. Consistency with other components to eliminate/reduce confusion (related to #1) 3. Manageability Use atlas.rest.address on server Key: ATLAS-72 URL: https://issues.apache.org/jira/browse/ATLAS-72 Project: Atlas Issue Type: Improvement Reporter: Shwetha G S On server, there are separate configs for should enable ssl, and port. Instead, it can use the client side config atlas.rest.address to derive ssl, port and host to bind to -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-67) Validate with secure zookeeper server for titan interaction with hbase and solr
[ https://issues.apache.org/jira/browse/ATLAS-67?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-67: -- Summary: Validate with secure zookeeper server for titan interaction with hbase and solr (was: Enable secure zookeeper communication for titan interaction with hbase and solr) Validate with secure zookeeper server for titan interaction with hbase and solr --- Key: ATLAS-67 URL: https://issues.apache.org/jira/browse/ATLAS-67 Project: Atlas Issue Type: Task Reporter: Suma Shivaprasad Attachments: org.apache.atlas.hive.hook.NegativeSSLAndKerberosHiveHookIT.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-46) Different data directory with restart
[
https://issues.apache.org/jira/browse/ATLAS-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629919#comment-14629919
]
Jonathan Maron commented on ATLAS-46:
-
What I propose is that we pre-process the variables/tokens before passing the
configuration properties on to titan etc.
Different data directory with restart
-
Key: ATLAS-46
URL: https://issues.apache.org/jira/browse/ATLAS-46
Project: Atlas
Issue Type: Bug
Reporter: Shwetha G S
Assignee: Shwetha G S
Priority: Critical
Fix For: trunk
Attachments: ATLAS-46.patch, application.log
Atlas start picks up different data directories - initially
./server/webapp/atlas/data and after restart ./data. This must be because of
relative path in the configuration
{noformat}
apache-atlas-0.6-incubating-SNAPSHOT$ ps -ef | grep -i atlas
504 6437 1 0 11:51AM ttys0000:07.66
/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/bin/java
-Datlas.log.dir=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/logs
-Datlas.log.file=application.log
-Datlas.home=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT
-Datlas.conf=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/conf
-Xmx1024m -classpath
/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/conf:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas/WEB-INF/classes:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/*:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/libext/*
org.apache.atlas.Main -app
/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas
apache-atlas-0.6-incubating-SNAPSHOT$ find . -name data
./server/webapp/atlas/data
./server/webapp/atlas/data/es/data
./server/webapp/atlas/lib/jquery/src/data
apache-atlas-0.6-incubating-SNAPSHOT$ ./bin/atlas_stop.py
apache-atlas-0.6-incubating-SNAPSHOT$ ps -ef | grep atlas
504 6457 1 0 11:54AM ttys0000:07.50
/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/bin/java
-Datlas.log.dir=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/logs
-Datlas.log.file=application.log
-Datlas.home=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT
-Datlas.conf=/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/conf
-Xmx1024m -classpath
/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/conf:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas/WEB-INF/classes:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/*:/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/libext/*
org.apache.atlas.Main -app
/Users/sshivalingamurthy/git/atlas-clone/target/apache-atlas-0.6-incubating-SNAPSHOT/server/webapp/atlas
apache-atlas-0.6-incubating-SNAPSHOT$ find . -name data
./data
./data/es/data
./server/webapp/atlas/data
./server/webapp/atlas/data/es/data
./server/webapp/atlas/lib/jquery/src/data
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-75) Atlas Hive integration
Linda George created ATLAS-75: - Summary: Atlas Hive integration Key: ATLAS-75 URL: https://issues.apache.org/jira/browse/ATLAS-75 Project: Atlas Issue Type: New Feature Affects Versions: 0.5-incubating Reporter: Linda George -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-76) Atlas business classification
Linda George created ATLAS-76: - Summary: Atlas business classification Key: ATLAS-76 URL: https://issues.apache.org/jira/browse/ATLAS-76 Project: Atlas Issue Type: New Feature Affects Versions: 0.5-incubating Reporter: Linda George -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-76) Atlas business classification
[ https://issues.apache.org/jira/browse/ATLAS-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630107#comment-14630107 ] Linda George commented on ATLAS-76: --- cc'ing [~aahn] Atlas business classification - Key: ATLAS-76 URL: https://issues.apache.org/jira/browse/ATLAS-76 Project: Atlas Issue Type: New Feature Affects Versions: 0.5-incubating Reporter: Linda George -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-75) Atlas Hive integration
[ https://issues.apache.org/jira/browse/ATLAS-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630106#comment-14630106 ] Linda George commented on ATLAS-75: --- cc'ing [~aahn] Atlas Hive integration -- Key: ATLAS-75 URL: https://issues.apache.org/jira/browse/ATLAS-75 Project: Atlas Issue Type: New Feature Affects Versions: 0.5-incubating Reporter: Linda George -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 36550: Integration with Hbase and solr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36550/ --- Review request for atlas, Jonathan Maron, Shwetha GS, and Seetharam Venkatesh. Bugs: ATLAS-37 https://issues.apache.org/jira/browse/ATLAS-37 Repository: atlas Description --- Fixed dependencies and code, tests for Titan to work with Hbase and solr Diffs - addons/hive-bridge/pom.xml 2617aeb addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/BaseSSLAndKerberosTest.java 5ae6fe2 addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/NegativeSSLAndKerberosHiveHookIT.java 629a9f3 addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/SSLAndKerberosHiveHookIT.java b8f29aa addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/SSLHiveHookIT.java 54bfaf6 addons/hive-bridge/src/test/resources/application.properties 50eefed client/src/test/java/org/apache/atlas/security/BaseSecurityTest.java 598f56b pom.xml 6e7c10c repository/pom.xml b23c289 repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java e7353f9 repository/src/main/java/org/apache/atlas/repository/graph/GraphProvider.java 5472081 repository/src/main/java/org/apache/atlas/repository/graph/TitanGraphProvider.java 3c1d151 repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 9556272 repository/src/test/java/org/apache/atlas/RepositoryServiceLoadingTest.java 4195955 repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java 67cdd78 repository/src/test/java/org/apache/atlas/discovery/HiveLineageServiceTest.java ce00f8c repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java fb697d0 repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java c774123 repository/src/test/resources/application.properties 20c0c51 src/conf/application.properties c6b16cc webapp/pom.xml 0ae4a05 webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 18ce17c webapp/src/main/java/org/apache/atlas/web/service/EmbeddedServer.java 88200f0 webapp/src/main/java/org/apache/atlas/web/service/SecureEmbeddedServer.java 174dafb webapp/src/main/resources/application.properties c9b8408 webapp/src/test/java/org/apache/atlas/web/filters/MetadataAuthenticationKerberosFilterIT.java 95acc57 webapp/src/test/java/org/apache/atlas/web/filters/MetadataAuthenticationSimpleFilterIT.java 5ea2e15 webapp/src/test/java/org/apache/atlas/web/listeners/LoginProcessorIT.java 54a7a7b webapp/src/test/java/org/apache/atlas/web/listeners/TestGuiceServletConfig.java PRE-CREATION webapp/src/test/java/org/apache/atlas/web/security/BaseSecurityTest.java PRE-CREATION webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerIT.java 3e27532 webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerITBase.java db110ca webapp/src/test/webapp/WEB-INF/web.xml PRE-CREATION Diff: https://reviews.apache.org/r/36550/diff/ Testing --- Ran all UTs and ITS. Except the hive hook security Its, all other tests succeed. Thanks, Suma Shivaprasad
[jira] [Created] (ATLAS-77) Atlas business classification
Linda George created ATLAS-77: - Summary: Atlas business classification Key: ATLAS-77 URL: https://issues.apache.org/jira/browse/ATLAS-77 Project: Atlas Issue Type: New Feature Affects Versions: 0.5-incubating Reporter: Linda George -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-77) Atlas business classification
[ https://issues.apache.org/jira/browse/ATLAS-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630109#comment-14630109 ] Linda George commented on ATLAS-77: --- cc'ing [~aahn] Atlas business classification - Key: ATLAS-77 URL: https://issues.apache.org/jira/browse/ATLAS-77 Project: Atlas Issue Type: New Feature Affects Versions: 0.5-incubating Reporter: Linda George -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-77) REST API to create new connections to Atlas
[ https://issues.apache.org/jira/browse/ATLAS-77?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linda George updated ATLAS-77: -- Summary: REST API to create new connections to Atlas (was: Atlas business classification) REST API to create new connections to Atlas --- Key: ATLAS-77 URL: https://issues.apache.org/jira/browse/ATLAS-77 Project: Atlas Issue Type: New Feature Affects Versions: 0.5-incubating Reporter: Linda George -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-56) atlas_config.py should give an informative error if jar or java binaries can't be found
[ https://issues.apache.org/jira/browse/ATLAS-56?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630628#comment-14630628 ] Aaron Dossett commented on ATLAS-56: Does anyone have feedback on the unit tests? atlas_config.py should give an informative error if jar or java binaries can't be found --- Key: ATLAS-56 URL: https://issues.apache.org/jira/browse/ATLAS-56 Project: Atlas Issue Type: Improvement Affects Versions: trunk Reporter: Aaron Dossett Assignee: Aaron Dossett Priority: Minor Fix For: trunk Attachments: ATLAS-56.patch, ATLAS-56_2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
