Build failed in Jenkins: PreCommit-ATLAS-Build-Test #359-master-KafkaTest.patch

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2656: updated entity notifications to replace AtlasEntity with

--
Started by user rmani
[EnvInject] - Loading node environment variables.
Building remotely on H0 (Hadoop) in workspace 

[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Done
Cloning the remote Git repository
Cloning repository http://git-wip-us.apache.org/repos/asf/atlas.git
 > git init  # 
 > timeout=10
Fetching upstream changes from http://git-wip-us.apache.org/repos/asf/atlas.git
 > git --version # timeout=10
 > git fetch --tags --progress http://git-wip-us.apache.org/repos/asf/atlas.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url 
 > http://git-wip-us.apache.org/repos/asf/atlas.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # 
 > timeout=10
 > git config remote.origin.url 
 > http://git-wip-us.apache.org/repos/asf/atlas.git # timeout=10
Fetching upstream changes from http://git-wip-us.apache.org/repos/asf/atlas.git
 > git fetch --tags --progress http://git-wip-us.apache.org/repos/asf/atlas.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 570f24231b69b119fd79b8481d65e3e122cdc51f (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 570f24231b69b119fd79b8481d65e3e122cdc51f
Commit message: "ATLAS-2656: updated entity notifications to replace 
AtlasEntity with AtlasEntityHeader"
 > git rev-list --no-walk 5fed48d969764ae72e8b87c204a2f481cb089716 # timeout=10
[EnvInject] - Mask passwords that will be passed as build parameters.
Set build name.
New build name is '#359-master-KafkaTest.patch'
Copying file to LOCAL_PATCH
[PreCommit-ATLAS-Build-Test] $ /bin/bash /tmp/jenkins4479377803567308641.sh
+ export TOOLS_HOME=/home/jenkins/tools
+ TOOLS_HOME=/home/jenkins/tools
+ export ANT_HOME=/home/jenkins/tools/ant/latest
+ ANT_HOME=/home/jenkins/tools/ant/latest
+ export FORREST_HOME=/home/jenkins/tools/forrest/latest
+ FORREST_HOME=/home/jenkins/tools/forrest/latest
+ export JAVA5_HOME=/home/jenkins/tools/java/latest1.5
+ JAVA5_HOME=/home/jenkins/tools/java/latest1.5
+ export FINDBUGS_HOME=/home/jenkins/tools/findbugs/latest
+ FINDBUGS_HOME=/home/jenkins/tools/findbugs/latest
+ export CLOVER_HOME=/home/jenkins/tools/clover/latest
+ CLOVER_HOME=/home/jenkins/tools/clover/latest
+ export MAVEN_HOME=/home/jenkins/tools/maven/latest
+ MAVEN_HOME=/home/jenkins/tools/maven/latest
+ export 
PATH=/home/jenkins/tools/java/latest1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/jenkins/tools/java/latest1.8/bin:/home/jenkins/tools/ant/latest/bin:/home/jenkins/tools/maven/latest/bin
+ 
PATH=/home/jenkins/tools/java/latest1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/jenkins/tools/java/latest1.8/bin:/home/jenkins/tools/ant/latest/bin:/home/jenkins/tools/maven/latest/bin
+ export 'MAVEN_OPTS=-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 
-noverify'
+ MAVEN_OPTS='-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -noverify'
+ export 'ANT_OPTS=-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 
-noverify'
+ ANT_OPTS='-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -noverify'
+ env
BUILD_URL=https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/359/
XDG_SESSION_ID=86
SKIP_UT=false
HUDSON_SERVER_COOKIE=f4ebd1e6b0d976e8
SHELL=/bin/bash
FORREST_HOME=/home/jenkins/tools/forrest/latest
SSH_CLIENT=62.210.60.235 34030 22
TOOLS_HOME=/home/jenkins/tools
BUILD_TAG=jenkins-PreCommit-ATLAS-Build-Test-359
GIT_PREVIOUS_COMMIT=5fed48d969764ae72e8b87c204a2f481cb089716
ROOT_BUILD_CAUSE=MANUALTRIGGER
ANT_HOME=/home/jenkins/tools/ant/latest
WORKSPACE=
JOB_URL=https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/
ANT_OPTS=-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -noverify
BRANCH=master
RUN_CHANGES_DISPLAY_URL=https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/359/display/redirect?page=changes
USER=jenkins
GIT_AUTHOR_NAME=jenkins
GIT_COMMITTER_NAME=jenkins
MAVEN_OPTS=-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -noverify
NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat
GIT_COMMIT=570f24231b69b119fd79b8481d65e3e122cdc51f
JENKINS_HOME=/x1/jenkins/jenkins-home
MAVEN_HOME=/home/jenkins/tools/maven/latest
MAIL=/var/mail/jenkins
PATH=/home/jenkins/tools/java/latest1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/jenkins/tools/java/latest1.8/bin:/home/jenkins/tools/ant/latest/bin:/home/jenkins/tools/maven/latest/bin
RUN_DISPLAY_URL=https://builds.apache.org/job/PreCommit-

Re: Apache Atlas 1.0 release preparation

[Nixon Rodrigues]

Madhan,

Thanks for the heads up for Atlas 1.0 release.

+1 for the release.


Best regards,

Nixon




On Wed, May 9, 2018, 6:13 AM Madhan Neethiraj  wrote:

> Atlas team,
>
>
>
> We have been busy adding multiple major features and improvements in
> master branch for months now. And it has been more than 3 months since
> Apache Atlas 1.0-alpha was released. It is time for GA release of Apache
> Atlas 1.0!
>
>
>
> Here is a short list of major features/updates:
>
>   - support for propagation of classifications along entity relationships
> – like lineage
>
>   - fine-grained metadata security, which enables access controls up to an
> entity instance level
>
>   - introduction of Glossary feature
>
>   - introduction of relationships as a first-class type
>
>   - replace use of Titan Graph DB with JanusGraph DB
>
>   - DSL rewrite, to replace use of Scala based implementation with Antlr
>
>   - introduction of V2 style notifications
>
>   - performance improvements in Atlas Hooks, by switching to use V2 style
> notifications
>
>   - significant updates in Atlas Web UI
>
>   - introduction of Atlas hook for HBase
>
>   - framework for Open Metadata & Governance (tech-preview)
>
>   - support for Cassandra and Elasticsearch (tech-preview)
>
>
>
> Here are the details of release activities, along with timelines for each:
>
>   - tentative date for the release will be next Friday, May-18-2018
>
>   - creation of release branch from current master: end of May-10 (PDT)
>
>   - send first release candidate for vote: end of May-12 (PDT)
>
>
>
> If you have any updates that needs to be in 1.0 release, please prioritize
> them to be committed before May-10.
>
>
>
> I volunteer to be the release manager for Apache Atlas 1.0 release.
>
>
>
> Regards,
>
> Madhan
>
>
>
>
>
>
>
>

Apache Atlas 1.0 release preparation

[Madhan Neethiraj]

Atlas team,

 

We have been busy adding multiple major features and improvements in master 
branch for months now. And it has been more than 3 months since Apache Atlas 
1.0-alpha was released. It is time for GA release of Apache Atlas 1.0!

 

Here is a short list of major features/updates:

  - support for propagation of classifications along entity relationships – 
like lineage

  - fine-grained metadata security, which enables access controls up to an 
entity instance level

  - introduction of Glossary feature

  - introduction of relationships as a first-class type

  - replace use of Titan Graph DB with JanusGraph DB

  - DSL rewrite, to replace use of Scala based implementation with Antlr

  - introduction of V2 style notifications

  - performance improvements in Atlas Hooks, by switching to use V2 style 
notifications

  - significant updates in Atlas Web UI

  - introduction of Atlas hook for HBase

  - framework for Open Metadata & Governance (tech-preview)

  - support for Cassandra and Elasticsearch (tech-preview)

 

Here are the details of release activities, along with timelines for each:

  - tentative date for the release will be next Friday, May-18-2018

  - creation of release branch from current master: end of May-10 (PDT)

  - send first release candidate for vote: end of May-12 (PDT)

 

If you have any updates that needs to be in 1.0 release, please prioritize them 
to be committed before May-10.

 

I volunteer to be the release manager for Apache Atlas 1.0 release.

 

Regards,

Madhan

 

 

 

Re: Review Request 67001: ATLAS-2656: updated entity notifications to replace AtlasEntity with AtlasEntityHeader

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67001/#review202716
---


Ship it!




Ship It!

- Sarath Subramanian


On May 8, 2018, 12:49 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67001/
> ---
> 
> (Updated May 8, 2018, 12:49 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, keval bhatt, Abhay 
> Kulkarni, Mehul Parikh, Nixon Rodrigues, Ruchi Solani, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2656
> https://issues.apache.org/jira/browse/ATLAS-2656
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - replaced AtlasEntity used in entity notifications with AtlasEntityHeader 
> (which includes fewer attributes), which can improve performance - especially 
> for entities having large number of attributes or attributes having large 
> array/map (like a hive_table containing 1000s of columns).
> - updated attribute-def with addition of 'includeInNotification' flag, which 
> can be used to specify if the attribute should be included in the 
> notificaiton. By default, all unique-attributes of the entity will be 
> included in the notification.
> - moved EntityNotificationV2 from org.apache.atlas.v1.model.notification 
> package to org.apache.atlas.model.notification.EntityNotificaiton
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1020-fs_model.json 1d7e9d3a1 
>   addons/models/1000-Hadoop/1030-hive_model.json 68a5c849d 
>   addons/models/1000-Hadoop/1060-hbase_model.json ec8dd4a40 
>   addons/models/1000-Hadoop/1070-kafka_model.json 9ddc379f8 
>   addons/models/1000-Hadoop/1080-storm_model.json b008c7ab2 
>   
> intg/src/main/java/org/apache/atlas/model/notification/EntityNotification.java
>  f70eb3f00 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> a4c707a91 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 1b6af94f2 
>   intg/src/main/java/org/apache/atlas/utils/AtlasJson.java 7d65bb646 
>   
> intg/src/main/java/org/apache/atlas/v1/model/notification/EntityNotificationV2.java
>  a8dfd235a 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 94242adcc 
>   
> repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
>  eb58b5e4b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
>  44fb95fd2 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
>  26f3165df 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  b46a1b776 
>   
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
>  ab17ae41e 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> 6f6a7ee9b 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/TypedefsJerseyResourceIT.java
>  5ee8d30b7 
> 
> 
> Diff: https://reviews.apache.org/r/67001/diff/1/
> 
> 
> Testing
> ---
> 
> - all UTs and ITs passed successfully - 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/356/
> - verified that the notifications include all unique attributes and 
> attributes having includeInNotification=true
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Jenkins build is back to normal : Atlas-master-IntegrationTests #1277

[Apache Jenkins Server]

See 

[jira] [Created] (ATLAS-2660) [Glossary] Add feature toggle for cross glossary category children

[Apoorv Naik (JIRA)]

Apoorv Naik created ATLAS-2660:
--

 Summary: [Glossary] Add feature toggle for cross glossary category 
children
 Key: ATLAS-2660
 URL: https://issues.apache.org/jira/browse/ATLAS-2660
 Project: Atlas
  Issue Type: Bug
Reporter: Apoorv Naik
Assignee: Apoorv Naik
 Fix For: 1.0.0


ATLAS-2655 added a restriction around the child category association within the 
same glossary. This JIRA adds the feature toggle capability so that this 
restriction can be controlled using an atlas property or some other mechanism



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-2655) [Glossary] Restrict category child to be within same glossary

[Apoorv Naik (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468060#comment-16468060
 ] 

Apoorv Naik commented on ATLAS-2655:


[~davidrad] Thanks for the suggestion. I'll take this feature toggle in a 
subsequent patch.

> [Glossary] Restrict category child to be within same glossary
> -
>
> Key: ATLAS-2655
> URL: https://issues.apache.org/jira/browse/ATLAS-2655
> Project: Atlas
>  Issue Type: Bug
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 1.0.0
>
>
> Currently no such check exists.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: Atlas-master-IntegrationTests #1276

[Apache Jenkins Server]

See 


Changes:

[apoorvnaik] ATLAS-2659: IllegalStateException handling in Collection pipeline 
code

--
[...truncated 679.00 KB...]
127.0.0.1 - - [08/May/2018:20:02:38 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tabley18reavuee@primary
 HTTP/1.1" 200 4980 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:38 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tabley18reavuee@primary
 HTTP/1.1" 200 4980 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:38 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=TRUNCATETABLE-%3E:default.tabley18reavuee@primary:1525809756000
 HTTP/1.1" 404 186 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:43 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=TRUNCATETABLE-%3E:default.tabley18reavuee@primary:1525809756000
 HTTP/1.1" 200 1162 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:43 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=TRUNCATETABLE-%3E:default.tabley18reavuee@primary:1525809756000
 HTTP/1.1" 200 1162 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:43 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/18ec7cd2-c35e-4409-8c74-a838cec271ab 
HTTP/1.1" 200 1162 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:43 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/79fa7115-bb41-45bc-9eeb-bb3eb146d1c9 
HTTP/1.1" 200 5252 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:43 +] "GET 
//localhost:31000/api/atlas/v2/lineage/79fa7115-bb41-45bc-9eeb-bb3eb146d1c9?depth=0&direction=INPUT
 HTTP/1.1" 200 135 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:47 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablef2c6yzbzdd@primary:1525809764000-%3E
 HTTP/1.1" 404 178 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablef2c6yzbzdd@primary:1525809764000-%3E
 HTTP/1.1" 200 2078 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablef2c6yzbzdd@primary:1525809764000-%3E
 HTTP/1.1" 200 2078 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/678811a5-c29c-42bc-a5c1-64fb2b0be069 
HTTP/1.1" 200 2078 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/ffe65700-f6e9-4cad-bed1-267ff66a110c 
HTTP/1.1" 200 5419 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/8acb56d3-2da8-41a3-bbcf-ce87b3d6592d 
HTTP/1.1" 200 1416 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hdfs_path?attr:qualifiedName=pfile:/home/jenkins/jenkins-slave/workspace/atlas-master-integrationtests/addons/hive-bridge/target/somedfspath1-data-vwkxtaatah
 HTTP/1.1" 200 1416 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hdfs_path?attr:qualifiedName=pfile:/home/jenkins/jenkins-slave/workspace/atlas-master-integrationtests/addons/hive-bridge/target/somedfspath1-data-vwkxtaatah
 HTTP/1.1" 200 1416 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tablef2c6yzbzdd@primary
 HTTP/1.1" 200 5419 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tablef2c6yzbzdd@primary
 HTTP/1.1" 200 5419 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:52 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/ffe65700-f6e9-4cad-bed1-267ff66a110c 
HTTP/1.1" 200 5419 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:56 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tablef2c6yzbzdd@primary
 HTTP/1.1" 200 5419 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:56 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tablef2c6yzbzdd@primary
 HTTP/1.1" 200 5419 "-" "Java/1.8.0_172"
127.0.0.1 - - [08/May/2018:20:02:56 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablef2c6yzbzdd@primary:1525809764000-%3E

Re: Review Request 67011: ATLAS-2659: IllegalStateException handling in Collection pipeline code

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67011/#review202672
---


Ship it!




Ship It!

- Madhan Neethiraj


On May 8, 2018, 5:55 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67011/
> ---
> 
> (Updated May 8, 2018, 5:55 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2659
> https://issues.apache.org/jira/browse/ATLAS-2659
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See JIRA
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> 95c0aa5bf 
> 
> 
> Diff: https://reviews.apache.org/r/67011/diff/1/
> 
> 
> Testing
> ---
> 
> PreCommit: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/357/
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Review Request 67011: ATLAS-2659: IllegalStateException handling in Collection pipeline code

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67011/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-2659
https://issues.apache.org/jira/browse/ATLAS-2659


Repository: atlas


Description
---

See JIRA


Diffs
-

  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
95c0aa5bf 


Diff: https://reviews.apache.org/r/67011/diff/1/


Testing
---

PreCommit: 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/357/


Thanks,

Apoorv Naik

[jira] [Updated] (ATLAS-2659) [Glossary] IllegalStateException handling in Collection pipeline code

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2659:
---
Attachment: 0001-ATLAS-2659-IllegalStateException-handling-in-Collect.patch

> [Glossary] IllegalStateException handling in Collection pipeline code 
> --
>
> Key: ATLAS-2659
> URL: https://issues.apache.org/jira/browse/ATLAS-2659
> Project: Atlas
>  Issue Type: Bug
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 
> 0001-ATLAS-2659-IllegalStateException-handling-in-Collect.patch
>
>
> toMap collection pipeline raises IllegalStateException when a duplicate key 
> is encountered and this leads to 500 internal server error. This fix 
> mitigates the 500 error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2659) [Glossary] IllegalStateException handling in Collection pipeline code

[Apoorv Naik (JIRA)]

Apoorv Naik created ATLAS-2659:
--

 Summary: [Glossary] IllegalStateException handling in Collection 
pipeline code 
 Key: ATLAS-2659
 URL: https://issues.apache.org/jira/browse/ATLAS-2659
 Project: Atlas
  Issue Type: Bug
Reporter: Apoorv Naik
Assignee: Apoorv Naik
 Fix For: 1.0.0


toMap collection pipeline raises IllegalStateException when a duplicate key is 
encountered and this leads to 500 internal server error. This fix mitigates the 
500 error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (ATLAS-2657) re-indexing solr in Atlas

[Ashutosh Mestry (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467640#comment-16467640
 ] 

Ashutosh Mestry edited comment on ATLAS-2657 at 5/8/18 4:20 PM:


[~kwohting] Please take a look at [[Atlas Index Repair 
Tool][[https://github.com/apache/atlas/tree/branch-0.8/tools/atlas-index-repair-kit]]]
 We had had anticipated this and hence built and committed this. Right now the 
tool is available on branch-0.8, since this is the last released version. We 
have not made this available on JanusGraph yet. 

Please take a look at the README file and let us know if this works for you.


was (Author: ashutoshm):
[~kwohting] Please take a look at [[Atlas Index Repair 
Tool][[https://github.com/apache/atlas/tree/branch-0.8/tools/atlas-index-repair-kit]].]
 We had had anticipated this and hence built and committed this. Right now the 
tool is available on branch-0.8, since this is the last released version. We 
have not made this available on JanusGraph yet. 

Please take a look at the README file and let us know if this works for you.

> re-indexing solr in Atlas
> -
>
> Key: ATLAS-2657
> URL: https://issues.apache.org/jira/browse/ATLAS-2657
> Project: Atlas
>  Issue Type: Wish
>Affects Versions: 1.0.0-alpha
>Reporter: kwohting
>Assignee: Ashutosh Mestry
>Priority: Minor
>
> hi, guys, 
> I'm from Alibaba Inc, We use Atlas as metastore with Hbase&Solr backends.
> we are highly concerned about  the stability and the ability to quick 
> recovery.
>  
> when some disaster happens: Hbase has it's own snapshot(which is safe), while 
> Solr's index maybe broken or inconsistency.  we see there are some tedious 
> procedures in 
> janusgraph([http://docs.janusgraph.org/latest/index-admin.html)|http://docs.janusgraph.org/latest/index-admin.html),].
>  but as a matter of fact,  if Atlas can provide some scripts to rebuild the 
> index just in few commands, it would be much more operational and efficient.
>  
> thanks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (ATLAS-2657) re-indexing solr in Atlas

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry reassigned ATLAS-2657:
--

Assignee: Ashutosh Mestry

> re-indexing solr in Atlas
> -
>
> Key: ATLAS-2657
> URL: https://issues.apache.org/jira/browse/ATLAS-2657
> Project: Atlas
>  Issue Type: Wish
>Affects Versions: 1.0.0-alpha
>Reporter: kwohting
>Assignee: Ashutosh Mestry
>Priority: Minor
>
> hi, guys, 
> I'm from Alibaba Inc, We use Atlas as metastore with Hbase&Solr backends.
> we are highly concerned about  the stability and the ability to quick 
> recovery.
>  
> when some disaster happens: Hbase has it's own snapshot(which is safe), while 
> Solr's index maybe broken or inconsistency.  we see there are some tedious 
> procedures in 
> janusgraph([http://docs.janusgraph.org/latest/index-admin.html)|http://docs.janusgraph.org/latest/index-admin.html),].
>  but as a matter of fact,  if Atlas can provide some scripts to rebuild the 
> index just in few commands, it would be much more operational and efficient.
>  
> thanks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-2657) re-indexing solr in Atlas

[Ashutosh Mestry (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467640#comment-16467640
 ] 

Ashutosh Mestry commented on ATLAS-2657:


[~kwohting] Please take a look at [[Atlas Index Repair 
Tool][[https://github.com/apache/atlas/tree/branch-0.8/tools/atlas-index-repair-kit]].]
 We had had anticipated this and hence built and committed this. Right now the 
tool is available on branch-0.8, since this is the last released version. We 
have not made this available on JanusGraph yet. 

Please take a look at the README file and let us know if this works for you.

> re-indexing solr in Atlas
> -
>
> Key: ATLAS-2657
> URL: https://issues.apache.org/jira/browse/ATLAS-2657
> Project: Atlas
>  Issue Type: Wish
>Affects Versions: 1.0.0-alpha
>Reporter: kwohting
>Priority: Minor
>
> hi, guys, 
> I'm from Alibaba Inc, We use Atlas as metastore with Hbase&Solr backends.
> we are highly concerned about  the stability and the ability to quick 
> recovery.
>  
> when some disaster happens: Hbase has it's own snapshot(which is safe), while 
> Solr's index maybe broken or inconsistency.  we see there are some tedious 
> procedures in 
> janusgraph([http://docs.janusgraph.org/latest/index-admin.html)|http://docs.janusgraph.org/latest/index-admin.html),].
>  but as a matter of fact,  if Atlas can provide some scripts to rebuild the 
> index just in few commands, it would be much more operational and efficient.
>  
> thanks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66064: ATLAS-2478 Elasticsearch support broken with JanusGraph 0.2.0

[Pierre Padovani]

> On May 7, 2018, 11:35 p.m., Ashutosh Mestry wrote:
> > graphdb/janus/pom.xml
> > Lines 121 (patched)
> > 
> >
> > This is getting included even when the profile is not selected.

Per the comment about elasticsearch jars in the main distribution. 

The new version is safer now after Elastic removed all of the shaded jars... 
and smaller.

We either keep them, or force them to download and copy it into their 
distribution during setup/installation.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66064/#review202602
---


On May 8, 2018, 3 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66064/
> ---
> 
> (Updated May 8, 2018, 3 p.m.)
> 
> 
> Review request for atlas, David Radley, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2478
> https://issues.apache.org/jira/browse/ATLAS-2478
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch fixes the Elasticsearch support for JanusGraph 0.2.0 and updates 
> documentation.
> 
> Included with this patch is an update to the berkley-elasticsearch profile to 
> automatically download and include elasticsearch as a side application much 
> like solr is. Updates to the start/stop/conf scripts are included as well.
> 
> NOTE: This patch includes a **BACKWARDS INCOMPATIBLE** change to 
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java. There 
> are six constants that are incorrectly named with a '.' (dot). This is not 
> supported in Elasticsearch 5 and beyond when defining a mapping **UNLESS** 
> the field names can be collectively thought of as an object. In the case of 
> the fields defined in the Constants.java file, 'type' is defined as a string 
> field, and 'type.name' is also defined as a string field. Elasticsearch sees 
> this as an error, since it cannot convert type to an object. The fix included 
> simply changes the field names from using a '.' (dot) to an '_' (underscore). 
> This should NOT affect compatibility with hbase/solr for new installs. For 
> existing installations, a reindex will be required as the field names will 
> have changed.
> 
> **Query**: There is a way we can simplify integration/unit tests for the 
> in-memory graph store by using a maven plugin that will download and run an 
> elasticsearch node. This is nothing more than a maven change, and change to 
> the atlas-application.properties to switch to elasticsearch from solr. I did 
> not implement this, but am curious if this change would be desired. If so, 
> this can be done with a separate ticket.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 16a3aaa15 
>   distro/pom.xml 6431fd86d 
>   distro/src/bin/atlas_config.py 9062da649 
>   distro/src/bin/atlas_start.py 61d69eb21 
>   distro/src/bin/atlas_stop.py 94c3d6d46 
>   distro/src/conf/atlas-env.sh 053cbd500 
>   distro/src/main/assemblies/standalone-package.xml dc88add56 
>   docs/src/site/twiki/Configuration.twiki 63c3fce96 
>   docs/src/site/twiki/HighAvailability.twiki 4270d0974 
>   docs/src/site/twiki/InstallationSteps.twiki dca0618e3 
>   graphdb/janus/pom.xml 016a09c33 
>   pom.xml 3469a393a 
>   webapp/pom.xml 03b84087f 
> 
> 
> Diff: https://reviews.apache.org/r/66064/diff/4/
> 
> 
> Testing
> ---
> 
> We currently use this fix with our Atlas setup in a fork of the Atlas code, 
> and have found no issues with it. Additionally, with the update to the 
> berkley-elasticsearch profile, I have extensively tested that profile to make 
> sure that management of Elasticsearch functioned correctly.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 66064: ATLAS-2478 Elasticsearch support broken with JanusGraph 0.2.0

[Pierre Padovani]

> On May 7, 2018, 11:41 p.m., Ashutosh Mestry wrote:
> > On executing ./bin/atlas_start.py displays these messages:
> > ./target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/conf/atlas-env.sh:
> >  line 65: MANAGE_EMBEDDED_CASSANDRA=${cassandra.embedded}: bad substitution
> > ./target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/conf/atlas-env.sh:
> >  line 68: MANAGE_LOCAL_ELASTICSEARCH=${elasticsearch.managed}: bad 
> > substitution
> > Exception: ('Could not find hbase-site.xml in %s. Please set env var 
> > HBASE_CONF_DIR to the hbase client conf dir', 
> > './target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/hbase/conf')
> > Traceback (most recent call last):
> >   File "./bin/atlas_start.py", line 163, in 
> >   File "./bin/atlas_start.py", line 92, in main
> > Exception: ('Could not find hbase-site.xml in %s. Please set env var 
> > HBASE_CONF_DIR to the hbase client conf dir', 
> > './target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/hbase/conf')

Fixed in the updated patch.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66064/#review202603
---


On May 8, 2018, 3 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66064/
> ---
> 
> (Updated May 8, 2018, 3 p.m.)
> 
> 
> Review request for atlas, David Radley, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2478
> https://issues.apache.org/jira/browse/ATLAS-2478
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch fixes the Elasticsearch support for JanusGraph 0.2.0 and updates 
> documentation.
> 
> Included with this patch is an update to the berkley-elasticsearch profile to 
> automatically download and include elasticsearch as a side application much 
> like solr is. Updates to the start/stop/conf scripts are included as well.
> 
> NOTE: This patch includes a **BACKWARDS INCOMPATIBLE** change to 
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java. There 
> are six constants that are incorrectly named with a '.' (dot). This is not 
> supported in Elasticsearch 5 and beyond when defining a mapping **UNLESS** 
> the field names can be collectively thought of as an object. In the case of 
> the fields defined in the Constants.java file, 'type' is defined as a string 
> field, and 'type.name' is also defined as a string field. Elasticsearch sees 
> this as an error, since it cannot convert type to an object. The fix included 
> simply changes the field names from using a '.' (dot) to an '_' (underscore). 
> This should NOT affect compatibility with hbase/solr for new installs. For 
> existing installations, a reindex will be required as the field names will 
> have changed.
> 
> **Query**: There is a way we can simplify integration/unit tests for the 
> in-memory graph store by using a maven plugin that will download and run an 
> elasticsearch node. This is nothing more than a maven change, and change to 
> the atlas-application.properties to switch to elasticsearch from solr. I did 
> not implement this, but am curious if this change would be desired. If so, 
> this can be done with a separate ticket.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 16a3aaa15 
>   distro/pom.xml 6431fd86d 
>   distro/src/bin/atlas_config.py 9062da649 
>   distro/src/bin/atlas_start.py 61d69eb21 
>   distro/src/bin/atlas_stop.py 94c3d6d46 
>   distro/src/conf/atlas-env.sh 053cbd500 
>   distro/src/main/assemblies/standalone-package.xml dc88add56 
>   docs/src/site/twiki/Configuration.twiki 63c3fce96 
>   docs/src/site/twiki/HighAvailability.twiki 4270d0974 
>   docs/src/site/twiki/InstallationSteps.twiki dca0618e3 
>   graphdb/janus/pom.xml 016a09c33 
>   pom.xml 3469a393a 
>   webapp/pom.xml 03b84087f 
> 
> 
> Diff: https://reviews.apache.org/r/66064/diff/4/
> 
> 
> Testing
> ---
> 
> We currently use this fix with our Atlas setup in a fork of the Atlas code, 
> and have found no issues with it. Additionally, with the update to the 
> berkley-elasticsearch profile, I have extensively tested that profile to make 
> sure that management of Elasticsearch functioned correctly.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 66064: ATLAS-2478 Elasticsearch support broken with JanusGraph 0.2.0

[Pierre Padovani]

> On May 7, 2018, 10:39 p.m., Madhan Neethiraj wrote:
> > pom.xml
> > Line 675 (original), 676 (patched)
> > 
> >
> > Will this change end up including elasticsearch libraries even when 
> > building for solr?

Yes... however the new 5.x version of the Elasticsearch jars no longer shade 
3rd party libraries so are far safer than the earlier versions. So we have a 
choice, include the jars, or force users who wish to use Elasticsearch as the 
search backend to copy in the jars manually.


- Pierre


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66064/#review202599
---


On May 8, 2018, 3 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66064/
> ---
> 
> (Updated May 8, 2018, 3 p.m.)
> 
> 
> Review request for atlas, David Radley, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2478
> https://issues.apache.org/jira/browse/ATLAS-2478
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch fixes the Elasticsearch support for JanusGraph 0.2.0 and updates 
> documentation.
> 
> Included with this patch is an update to the berkley-elasticsearch profile to 
> automatically download and include elasticsearch as a side application much 
> like solr is. Updates to the start/stop/conf scripts are included as well.
> 
> NOTE: This patch includes a **BACKWARDS INCOMPATIBLE** change to 
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java. There 
> are six constants that are incorrectly named with a '.' (dot). This is not 
> supported in Elasticsearch 5 and beyond when defining a mapping **UNLESS** 
> the field names can be collectively thought of as an object. In the case of 
> the fields defined in the Constants.java file, 'type' is defined as a string 
> field, and 'type.name' is also defined as a string field. Elasticsearch sees 
> this as an error, since it cannot convert type to an object. The fix included 
> simply changes the field names from using a '.' (dot) to an '_' (underscore). 
> This should NOT affect compatibility with hbase/solr for new installs. For 
> existing installations, a reindex will be required as the field names will 
> have changed.
> 
> **Query**: There is a way we can simplify integration/unit tests for the 
> in-memory graph store by using a maven plugin that will download and run an 
> elasticsearch node. This is nothing more than a maven change, and change to 
> the atlas-application.properties to switch to elasticsearch from solr. I did 
> not implement this, but am curious if this change would be desired. If so, 
> this can be done with a separate ticket.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 16a3aaa15 
>   distro/pom.xml 6431fd86d 
>   distro/src/bin/atlas_config.py 9062da649 
>   distro/src/bin/atlas_start.py 61d69eb21 
>   distro/src/bin/atlas_stop.py 94c3d6d46 
>   distro/src/conf/atlas-env.sh 053cbd500 
>   distro/src/main/assemblies/standalone-package.xml dc88add56 
>   docs/src/site/twiki/Configuration.twiki 63c3fce96 
>   docs/src/site/twiki/HighAvailability.twiki 4270d0974 
>   docs/src/site/twiki/InstallationSteps.twiki dca0618e3 
>   graphdb/janus/pom.xml 016a09c33 
>   pom.xml 3469a393a 
>   webapp/pom.xml 03b84087f 
> 
> 
> Diff: https://reviews.apache.org/r/66064/diff/4/
> 
> 
> Testing
> ---
> 
> We currently use this fix with our Atlas setup in a fork of the Atlas code, 
> and have found no issues with it. Additionally, with the update to the 
> berkley-elasticsearch profile, I have extensively tested that profile to make 
> sure that management of Elasticsearch functioned correctly.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

Re: Review Request 66064: ATLAS-2478 Elasticsearch support broken with JanusGraph 0.2.0

[Pierre Padovani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66064/
---

(Updated May 8, 2018, 3 p.m.)


Review request for atlas, David Radley, Madhan Neethiraj, and Sarath 
Subramanian.


Changes
---

Fixes based on review


Bugs: ATLAS-2478
https://issues.apache.org/jira/browse/ATLAS-2478


Repository: atlas


Description
---

This patch fixes the Elasticsearch support for JanusGraph 0.2.0 and updates 
documentation.

Included with this patch is an update to the berkley-elasticsearch profile to 
automatically download and include elasticsearch as a side application much 
like solr is. Updates to the start/stop/conf scripts are included as well.

NOTE: This patch includes a **BACKWARDS INCOMPATIBLE** change to 
/atlas/common/src/main/java/org/apache/atlas/repository/Constants.java. There 
are six constants that are incorrectly named with a '.' (dot). This is not 
supported in Elasticsearch 5 and beyond when defining a mapping **UNLESS** the 
field names can be collectively thought of as an object. In the case of the 
fields defined in the Constants.java file, 'type' is defined as a string field, 
and 'type.name' is also defined as a string field. Elasticsearch sees this as 
an error, since it cannot convert type to an object. The fix included simply 
changes the field names from using a '.' (dot) to an '_' (underscore). This 
should NOT affect compatibility with hbase/solr for new installs. For existing 
installations, a reindex will be required as the field names will have changed.

**Query**: There is a way we can simplify integration/unit tests for the 
in-memory graph store by using a maven plugin that will download and run an 
elasticsearch node. This is nothing more than a maven change, and change to the 
atlas-application.properties to switch to elasticsearch from solr. I did not 
implement this, but am curious if this change would be desired. If so, this can 
be done with a separate ticket.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 16a3aaa15 
  distro/pom.xml 6431fd86d 
  distro/src/bin/atlas_config.py 9062da649 
  distro/src/bin/atlas_start.py 61d69eb21 
  distro/src/bin/atlas_stop.py 94c3d6d46 
  distro/src/conf/atlas-env.sh 053cbd500 
  distro/src/main/assemblies/standalone-package.xml dc88add56 
  docs/src/site/twiki/Configuration.twiki 63c3fce96 
  docs/src/site/twiki/HighAvailability.twiki 4270d0974 
  docs/src/site/twiki/InstallationSteps.twiki dca0618e3 
  graphdb/janus/pom.xml 016a09c33 
  pom.xml 3469a393a 
  webapp/pom.xml 03b84087f 


Diff: https://reviews.apache.org/r/66064/diff/4/

Changes: https://reviews.apache.org/r/66064/diff/3-4/


Testing
---

We currently use this fix with our Atlas setup in a fork of the Atlas code, and 
have found no issues with it. Additionally, with the update to the 
berkley-elasticsearch profile, I have extensively tested that profile to make 
sure that management of Elasticsearch functioned correctly.


Thanks,

Pierre Padovani

[jira] [Updated] (ATLAS-2122) OMAG Server

[Bogdan Sava (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bogdan Sava updated ATLAS-2122:
---
Attachment: 0001-ATLAS-2122-running-OMAG-API-in-ATLAS.patch

> OMAG Server
> ---
>
> Key: ATLAS-2122
> URL: https://issues.apache.org/jira/browse/ATLAS-2122
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 
> 0001-ATLAS-2122-OMAG-Server-Application-and-OMRS-APIs.patch, 
> 0001-ATLAS-2122-running-OMAG-API-in-ATLAS.patch
>
>
> This Jira Assembles the Open Metadata and Governance components into a 
> configurable server to support the OMAS Access Layer, Repository or native 
> metadata repository.
> The OMAG server is a Spring Boot Application with a simple REST API. It takes 
> the user Id of the administrator, the server name and cohort name as path 
> variables, other parameters are passed as request parameters. These are 
> examples of the commands to set up the server using defaults. 
> Query configuration 
> GET http://localhost:8080/omag/admin/{userId}/{serverName}/configuration/ 
> Set server type name: 
> POST 
> http://localhost:8080/omag/admin/{userId}/{serverName}/server-type?typeName={name}
>  
> Set organization name: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/organization-name?name={organizationName}
>  
> Enable the access services: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/access-services/mode?serviceMode=ENABLED
>  
> Enable the Atlas graph repository: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/mode?repositoryMode=LOCAL_GRAPH_REPOSITORY
>  
> Enable the in-memory repository: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/mode?repositoryMode=IN_MEMORY_REPOSITORY
>  
> Enable server as a repository proxy: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/mode?repositoryMode=REPOSITORY_PROXY
>  
> To add the local repository connection for the repository proxy 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/proxy-details?connectorProvider={javaClassName}&url={nativeServerURL}
>  
> To add the local repository's event mapper: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/event-mapper-details?connectorProvider={javaClassName}&eventSource={resourceName}
>  
> To enable access to a cohort 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/cohort/{cohortName}/mode?serviceMode=ENABLED
>  
> To remove the local repository 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/mode?repositoryMode=NO_LOCAL_REPOSITORY
>  
> To disable the access services 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/access-services/mode?serviceMode=DISABLED
>  
> To disconnect from a cohort 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/cohort/{cohortName}/mode?serviceMode=DISABLED
>  
> To start up OMRS/OMAS services 
> POST http://localhost:8080/omag/admin//{userId}/{serverName}/instance 
> To shutdown OMRS/OMAS services 
> DELETE 
> http://localhost:8080/omag/admin//{userId}/{serverName}/instance?permanent=false
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2658) UI - gId undefined error appears if user toggles to category when there are no terms and category

[Tejas Rawool (JIRA)]

Tejas Rawool created ATLAS-2658:
---

 Summary: UI - gId undefined error appears if user toggles to 
category when there are no terms and category
 Key: ATLAS-2658
 URL: https://issues.apache.org/jira/browse/ATLAS-2658
 Project: Atlas
  Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Tejas Rawool
 Fix For: 1.0.0
 Attachments: SSS21.png

Steps to Reproduce:-

1.Go to category tab

2.Now switch to category when the category list is empty

 

Error:-

gId undefined error appears in console



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-2655) [Glossary] Restrict category child to be within same glossary

[David Radley (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467152#comment-16467152
 ] 

David Radley commented on ATLAS-2655:
-

It can be very useful to be able to have category children in other glossaries. 
For example re- using a 'standard' glossary categorizations across an 
organization.  
 If you need to restrict - could you put it under control of a feature toggle. 

> [Glossary] Restrict category child to be within same glossary
> -
>
> Key: ATLAS-2655
> URL: https://issues.apache.org/jira/browse/ATLAS-2655
> Project: Atlas
>  Issue Type: Bug
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 1.0.0
>
>
> Currently no such check exists.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2464) Information View OMAS Listener

[Ruxandra Rosu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruxandra Rosu updated ATLAS-2464:
-
Attachment: 0001-IV-OMAS-initial-patch-08.05.patch

> Information View OMAS Listener
> --
>
> Key: ATLAS-2464
> URL: https://issues.apache.org/jira/browse/ATLAS-2464
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Maryna Strelchuk
>Assignee: Ruxandra Rosu
>Priority: Major
> Attachments: 0001-IV-OMAS-initial-patch-08.05.patch, 
> initial_patch_iv.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2657) re-indexing solr in Atlas

[kwohting (JIRA)]

kwohting created ATLAS-2657:
---

 Summary: re-indexing solr in Atlas
 Key: ATLAS-2657
 URL: https://issues.apache.org/jira/browse/ATLAS-2657
 Project: Atlas
  Issue Type: Wish
Affects Versions: 1.0.0-alpha
Reporter: kwohting


hi, guys, 

I'm from Alibaba Inc, We use Atlas as metastore with Hbase&Solr backends.

we are highly concerned about  the stability and the ability to quick recovery.

 

when some disaster happens: Hbase has it's own snapshot(which is safe), while 
Solr's index maybe broken or inconsistency.  we see there are some tedious 
procedures in 
janusgraph([http://docs.janusgraph.org/latest/index-admin.html)|http://docs.janusgraph.org/latest/index-admin.html),].
 but as a matter of fact,  if Atlas can provide some scripts to rebuild the 
index just in few commands, it would be much more operational and efficient.

 

thanks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66949: ATLAS-2523: changes to accept external GUIDs and manage homeIds

[Graham Wallis]

> On May 4, 2018, 12:04 p.m., David Radley wrote:
> > common/src/main/java/org/apache/atlas/repository/Constants.java
> > Lines 80 (patched)
> > 
> >
> > I suggest a comment here to describe how an Atlas developer should 
> > understand this field. 
> > 
> > A concern I have, is that if the users go through the om aPI then the 
> > entities with a non-null homId will never be updated - apart from as part 
> > of the asynchronous cnotification process from anotehr server. The Atlas 
> > developer using the local Atlas API, needs to be aware not to change the 
> > entities and relationships with a none null homeId. Ideally we should 
> > police this.

I can certainly add a comment to say what the field is for. I see your point 
about the policing of updates but I might suggest that should be a separate 
JIRA.


> On May 4, 2018, 12:04 p.m., David Radley wrote:
> > intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java
> > Lines 61 (patched)
> > 
> >
> > I am wondering why you are adding a Logger and not using it om this 
> > change.

I had debug statements in the code during development and testing; they have 
been removed for brevity but I forgot to clear out the logger.


> On May 4, 2018, 12:04 p.m., David Radley wrote:
> > intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java
> > Line 384 (original), 389 (patched)
> > 
> >
> > I suggest adding a comment here, around why you are checking for '-' as 
> > the first character. 
> > 
> > What happens if another repository creates an identifier starting with 
> > a '-' character?
> > 
> > to cope with theis case - I am suggest that the logic should check for 
> > a null homeId and a guid starting character of '-' (or null) we should say 
> > the guid is not assigned.

I like that - I wrote the GUID handling change before adding the homeId system 
attribute. We could (now) include a check for homeId (not null) in the GUID 
validity checking.


> On May 4, 2018, 12:04 p.m., David Radley wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
> > Line 709 (original), 709 (patched)
> > 
> >
> > why are we leaving in commented out code

No valid reason - I wanted to highlight the effect of the change by having the 
'before' and 'after' - but if the patch is seen as generally OK I should remove 
it.


- Graham


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66949/#review202433
---


On May 4, 2018, 10:55 a.m., Graham Wallis wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66949/
> ---
> 
> (Updated May 4, 2018, 10:55 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-2523: Changes to accept external GUIDs and manage homeIds
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 
> c570be2ccbc41a426b0f093b4a33163092223f2f 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 
> e6a7f19633a1642f6b415db99e20c0641df9463b 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> d04daa5d283fdba90b917f38eba6c937021352df 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
> 1b6af94f24eb7d99953f72815167c868a9bd9efd 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> edf10da9533803234342dc3313ea1024c5e7030f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  9fcba6ddaa1b74b2c81b980bcb455b614f4a4ed7 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  d51adad80e133d636ef84883d395126ae0150af5 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  3b9a287f6986ff6bca26fc20b2f94ec1700dbe1e 
> 
> 
> Diff: https://reviews.apache.org/r/66949/diff/1/
> 
> 
> Testing
> ---
> 
> Functional testing of these changes to save reference copies of entities and 
> relationships
> 
> 
> Thanks,
> 
> Graham Wallis
> 
>

Review Request 67001: ATLAS-2656: updated entity notifications to replace AtlasEntity with AtlasEntityHeader

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67001/
---

Review request for atlas, Apoorv Naik, Ashutosh Mestry, keval bhatt, Abhay 
Kulkarni, Mehul Parikh, Nixon Rodrigues, Ruchi Solani, and Sarath Subramanian.


Bugs: ATLAS-2656
https://issues.apache.org/jira/browse/ATLAS-2656


Repository: atlas


Description
---

- replaced AtlasEntity used in entity notifications with AtlasEntityHeader 
(which includes fewer attributes), which can improve performance - especially 
for entities having large number of attributes or attributes having large 
array/map (like a hive_table containing 1000s of columns).
- updated attribute-def with addition of 'includeInNotification' flag, which 
can be used to specify if the attribute should be included in the notificaiton. 
By default, all unique-attributes of the entity will be included in the 
notification.
- moved EntityNotificationV2 from org.apache.atlas.v1.model.notification 
package to org.apache.atlas.model.notification.EntityNotificaiton


Diffs
-

  addons/models/1000-Hadoop/1020-fs_model.json 1d7e9d3a1 
  addons/models/1000-Hadoop/1030-hive_model.json 68a5c849d 
  addons/models/1000-Hadoop/1060-hbase_model.json ec8dd4a40 
  addons/models/1000-Hadoop/1070-kafka_model.json 9ddc379f8 
  addons/models/1000-Hadoop/1080-storm_model.json b008c7ab2 
  
intg/src/main/java/org/apache/atlas/model/notification/EntityNotification.java 
f70eb3f00 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
a4c707a91 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 1b6af94f2 
  intg/src/main/java/org/apache/atlas/utils/AtlasJson.java 7d65bb646 
  
intg/src/main/java/org/apache/atlas/v1/model/notification/EntityNotificationV2.java
 a8dfd235a 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 94242adcc 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
 eb58b5e4b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
 44fb95fd2 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
 26f3165df 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
 b46a1b776 
  
webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
 ab17ae41e 
  webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
6f6a7ee9b 
  
webapp/src/test/java/org/apache/atlas/web/integration/TypedefsJerseyResourceIT.java
 5ee8d30b7 


Diff: https://reviews.apache.org/r/67001/diff/1/


Testing
---

- all UTs and ITs passed successfully - 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/356/
- verified that the notifications include all unique attributes and attributes 
having includeInNotification=true


Thanks,

Madhan Neethiraj

[jira] [Updated] (ATLAS-2656) Update entity notifications to replace AtlasEntity with AtlasEntityHeader

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-2656:

Attachment: ATLAS-2656.patch

> Update entity notifications to replace AtlasEntity with AtlasEntityHeader
> -
>
> Key: ATLAS-2656
> URL: https://issues.apache.org/jira/browse/ATLAS-2656
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Attachments: ATLAS-2656.patch
>
>
> Replacing AtlasEntity used in entity notifications with AtlasEntityHeader 
> (which includes fewer attributes) can improve performance, especially for 
> entities having large number of attribute or attributes having large number 
> of array/map elements (like a hive_table containing 1000s of columns).
> In addition, it should be possible to specify which entity attributes to 
> include in the notification via type-def - for example by updating 
> AtlasAttributeDef with addition of a flag named 'includeInNotification'.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)