[jira] [Commented] (ATLAS-3934) Dockerfile should build from local repository rather than pulling code from git

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198617#comment-17198617
 ] 

ASF subversion and git services commented on ATLAS-3934:


Commit 182c1bdb35d3817ddb82511da1c39acd971eedbc in atlas's branch 
refs/heads/branch-2.0 from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=182c1bd ]

ATLAS-3934: docker scripts to build and run Apache Atlas in containers

(cherry picked from commit 9b1d4c387fb56d08bfc86150b0cff7d071540109)


> Dockerfile should build from local repository rather than pulling code from 
> git
> ---
>
> Key: ATLAS-3934
> URL: https://issues.apache.org/jira/browse/ATLAS-3934
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Syed Atif Akhtar
>Assignee: Madhan Neethiraj
>Priority: Trivial
>  Labels: atlas, build, development_environment, docker
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3934.patch
>
>
> *Expected Behaviour:*
>  When a developer wants to test out their changes using the dockerfile in dev 
> support, they should be able to create the dockerfile with atlas built and 
> mount from a local directory with the changes they have done.
> *Current Behaviour:*
> The dockerfile currently defined in dev-support pulls the source code from 
> git instead of building the code that the user has defined
>  
> {code:java}
> RUN git clone https://github.com/apache/atlas.git -b master{code}
>  
> *Suggestion:*
>  Maintain separate dockerfiles for publishing an image of Atlas vs development



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3934) Dockerfile should build from local repository rather than pulling code from git

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198616#comment-17198616
 ] 

ASF subversion and git services commented on ATLAS-3934:


Commit 9b1d4c387fb56d08bfc86150b0cff7d071540109 in atlas's branch 
refs/heads/master from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=9b1d4c3 ]

ATLAS-3934: docker scripts to build and run Apache Atlas in containers


> Dockerfile should build from local repository rather than pulling code from 
> git
> ---
>
> Key: ATLAS-3934
> URL: https://issues.apache.org/jira/browse/ATLAS-3934
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Syed Atif Akhtar
>Assignee: Madhan Neethiraj
>Priority: Trivial
>  Labels: atlas, build, development_environment, docker
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3934.patch
>
>
> *Expected Behaviour:*
>  When a developer wants to test out their changes using the dockerfile in dev 
> support, they should be able to create the dockerfile with atlas built and 
> mount from a local directory with the changes they have done.
> *Current Behaviour:*
> The dockerfile currently defined in dev-support pulls the source code from 
> git instead of building the code that the user has defined
>  
> {code:java}
> RUN git clone https://github.com/apache/atlas.git -b master{code}
>  
> *Suggestion:*
>  Maintain separate dockerfiles for publishing an image of Atlas vs development



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72865: ATLAS-3934: docker scripts to build and run Apache Atlas in containers

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72865/#review221901
---




dev-support/atlas-docker/README.md
Lines 76 (patched)


consider including Atlas Home directory and log file location in README


- Sarath Subramanian


On Sept. 11, 2020, 10:55 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72865/
> ---
> 
> (Updated Sept. 11, 2020, 10:55 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, keval bhatt, Mehul 
> Parikh, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, Sharmadha Sainath, 
> and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3934
> https://issues.apache.org/jira/browse/ATLAS-3934
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - added docker files, docker-compose files and scripts to build Atlas in a 
> container. The image to build Atlas supports following volumns:
> -- scripts: build scripts in the local directory will be made available in 
> the container
> -- dist:archives from the build will be copied into this volume
> -- .m2: if provided, build uses the give .m2 directory, which can help 
> avoid large downloads
> -- patches: patche files, if present, will be applied on top of the specified 
> github branch
> -- src: if provided, builds from the sources in this directory (instead 
> of from sources from github)
> 
> - added docker file, docker-compose file and scripts to install and run Atlas 
> in a container.
> 
> - note: this patch depends on updates in ATLAS-3939, which is under review
> 
> 
> Diffs
> -
> 
>   dev-support/atlas-docker/.dockerignore PRE-CREATION 
>   dev-support/atlas-docker/Dockerfile.atlas PRE-CREATION 
>   dev-support/atlas-docker/Dockerfile.atlas-base PRE-CREATION 
>   dev-support/atlas-docker/Dockerfile.atlas-build PRE-CREATION 
>   dev-support/atlas-docker/README.md 5296f05ba 
>   dev-support/atlas-docker/dist/.gitignore PRE-CREATION 
>   dev-support/atlas-docker/docker-compose.atlas-base.yml PRE-CREATION 
>   dev-support/atlas-docker/docker-compose.atlas-build.yml PRE-CREATION 
>   dev-support/atlas-docker/docker-compose.atlas.yml PRE-CREATION 
>   dev-support/atlas-docker/patches/.gitignore PRE-CREATION 
>   dev-support/atlas-docker/scripts/atlas-build.sh PRE-CREATION 
>   dev-support/atlas-docker/scripts/atlas.sh PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/72865/diff/1/
> 
> 
> Testing
> ---
> 
> - verified build with the following command:
>docker-compose -f docker-compose.atlas-base.yml -f 
> docker-compose.atlas-build.yml up
> 
> - verified install and running of Atlas with the following command:
>docker-compose -f docker-compose.atlas-base.yml -f 
> docker-compose.atlas.yml up -d
> 
> - Ran quick start successfully
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 72865: ATLAS-3934: docker scripts to build and run Apache Atlas in containers

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72865/#review221900
---


Ship it!




Validated the docker build process using docker-compose. 

Atlas builds fine and starts in container. Ran quickstart and verified entities 
are created fine.

- Sarath Subramanian


On Sept. 11, 2020, 10:55 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72865/
> ---
> 
> (Updated Sept. 11, 2020, 10:55 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, keval bhatt, Mehul 
> Parikh, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, Sharmadha Sainath, 
> and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3934
> https://issues.apache.org/jira/browse/ATLAS-3934
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - added docker files, docker-compose files and scripts to build Atlas in a 
> container. The image to build Atlas supports following volumns:
> -- scripts: build scripts in the local directory will be made available in 
> the container
> -- dist:archives from the build will be copied into this volume
> -- .m2: if provided, build uses the give .m2 directory, which can help 
> avoid large downloads
> -- patches: patche files, if present, will be applied on top of the specified 
> github branch
> -- src: if provided, builds from the sources in this directory (instead 
> of from sources from github)
> 
> - added docker file, docker-compose file and scripts to install and run Atlas 
> in a container.
> 
> - note: this patch depends on updates in ATLAS-3939, which is under review
> 
> 
> Diffs
> -
> 
>   dev-support/atlas-docker/.dockerignore PRE-CREATION 
>   dev-support/atlas-docker/Dockerfile.atlas PRE-CREATION 
>   dev-support/atlas-docker/Dockerfile.atlas-base PRE-CREATION 
>   dev-support/atlas-docker/Dockerfile.atlas-build PRE-CREATION 
>   dev-support/atlas-docker/README.md 5296f05ba 
>   dev-support/atlas-docker/dist/.gitignore PRE-CREATION 
>   dev-support/atlas-docker/docker-compose.atlas-base.yml PRE-CREATION 
>   dev-support/atlas-docker/docker-compose.atlas-build.yml PRE-CREATION 
>   dev-support/atlas-docker/docker-compose.atlas.yml PRE-CREATION 
>   dev-support/atlas-docker/patches/.gitignore PRE-CREATION 
>   dev-support/atlas-docker/scripts/atlas-build.sh PRE-CREATION 
>   dev-support/atlas-docker/scripts/atlas.sh PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/72865/diff/1/
> 
> 
> Testing
> ---
> 
> - verified build with the following command:
>docker-compose -f docker-compose.atlas-base.yml -f 
> docker-compose.atlas-build.yml up
> 
> - verified install and running of Atlas with the following command:
>docker-compose -f docker-compose.atlas-base.yml -f 
> docker-compose.atlas.yml up -d
> 
> - Ran quick start successfully
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

[jira] [Resolved] (ATLAS-3944) Logs of import scripts should go to Atlas default log directory

[Deep Singh (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deep Singh resolved ATLAS-3944.
---
Resolution: Fixed

> Logs of import scripts should go to Atlas default log directory
> ---
>
> Key: ATLAS-3944
> URL: https://issues.apache.org/jira/browse/ATLAS-3944
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: trunk, 2.1.0
>Reporter: Deep Singh
>Assignee: Deep Singh
>Priority: Major
> Fix For: trunk
>
> Attachments: 0001-Setting-default-log-directory-to-var-log-atlas.patch
>
>
> Import scripts like import-hbase.sh try to dump logs in a log directory 
> inside the base directory. Most of the users running import scripts have 
> read-only access to the base directory, this leads to the following error 
> exception.
> java.io.FileNotFoundException: 
> /opt/cloudera/parcels/CDH/lib/atlas/logs/import-hbase.log (No such file or 
> directory)
> Although there is a mechanism to set the log location to the desired 
> directory using env variable ATLAS_LOG_DIR, but by default, the script must 
> try to dump logs in Atlas default logs directory (/var/log/atlas)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3939) Add build profile for berkeleydb-solr

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198479#comment-17198479
 ] 

ASF subversion and git services commented on ATLAS-3939:


Commit 58ea1a8baaf163c973d77585b29ee192650cc863 in atlas's branch 
refs/heads/branch-2.0 from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=58ea1a8 ]

ATLAS-3939: added profile berkeley-solr

(cherry picked from commit 7690e51d10f008c1e13a4c85f9aa8821fee33f4a)


> Add build profile for berkeleydb-solr
> -
>
> Key: ATLAS-3939
> URL: https://issues.apache.org/jira/browse/ATLAS-3939
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3939.patch
>
>
> Atlas supports a number of build profiles, like 
> external-hbase-sol/embedded-hbase-solr/embedded-cassandra-solr/berkeley-elasticsearch,
>  for various combinations of backend stores an index stores. This should be 
> extended to support berkeley-solr as well; this profile can be useful for 
> dev/testing of Atlas.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3939) Add build profile for berkeleydb-solr

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198477#comment-17198477
 ] 

ASF subversion and git services commented on ATLAS-3939:


Commit 7690e51d10f008c1e13a4c85f9aa8821fee33f4a in atlas's branch 
refs/heads/master from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=7690e51 ]

ATLAS-3939: added profile berkeley-solr


> Add build profile for berkeleydb-solr
> -
>
> Key: ATLAS-3939
> URL: https://issues.apache.org/jira/browse/ATLAS-3939
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3939.patch
>
>
> Atlas supports a number of build profiles, like 
> external-hbase-sol/embedded-hbase-solr/embedded-cassandra-solr/berkeley-elasticsearch,
>  for various combinations of backend stores an index stores. This should be 
> extended to support berkeley-solr as well; this profile can be useful for 
> dev/testing of Atlas.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72879: ATLAS-3946 : Filter TypeDefs in Metrics API and show data accordingly

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72879/#review221892
---




repository/src/main/java/org/apache/atlas/services/MetricsService.java
Lines 147 (patched)


entityDef.subTypes includes only direct sub-types. To get sub-types in all 
levels, consider the following:

  AtlasEntityType entityType = 
typeRegistry.getEntityType(entityDef.getName();
  
  for (String type : entityType.getTypeAndAllSubTypes()) {
... earlier implementation
  }


- Madhan Neethiraj


On Sept. 18, 2020, 8:59 a.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72879/
> ---
> 
> (Updated Sept. 18, 2020, 8:59 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3946
> https://issues.apache.org/jira/browse/ATLAS-3946
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Filter TypeDefs in Metrics API and show data accordingly, currently data is 
> shown for all types even if user do not have access to those types.
> 
> This patch filterTypeDefs after authorization and updates the data for 
> required types.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/services/MetricsService.java 
> 7917f32e5 
>   webapp/src/main/java/org/apache/atlas/web/security/AtlasSecurityConfig.java 
> e74a9e93a 
> 
> 
> Diff: https://reviews.apache.org/r/72879/diff/3/
> 
> 
> Testing
> ---
> 
> Tested Statistics UI from different users with different policies for 
> entityType and classfication.
> 
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/25/console
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

[jira] [Updated] (ATLAS-3911) UI: Type system management

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3911:
---
Attachment: ATLAS-3911-1.patch

> UI: Type system management
> --
>
> Key: ATLAS-3911
> URL: https://issues.apache.org/jira/browse/ATLAS-3911
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3911-1.patch, ATLAS-3911.patch, 
> Type_System_Tree.png, Type_System_Tree_Details.png
>
>
> !Type_System_Tree.png|width=482,height=237!
>  
> !Type_System_Tree_Details.png|width=509,height=249!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72879: ATLAS-3946 : Filter TypeDefs in Metrics API and show data accordingly

[Nixon Rodrigues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72879/
---

(Updated Sept. 18, 2020, 8:59 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, and 
Sarath Subramanian.


Changes
---

Handled review comment from Madhan for

>> Shouldn't entityDefs be used here, instead of 
>> typeRegistry.getAllEntityTypes()? Please review.

Used filtered entityDefs instead typeRegistry.getAllEntityTypes()


Bugs: ATLAS-3946
https://issues.apache.org/jira/browse/ATLAS-3946


Repository: atlas


Description
---

Filter TypeDefs in Metrics API and show data accordingly, currently data is 
shown for all types even if user do not have access to those types.

This patch filterTypeDefs after authorization and updates the data for required 
types.


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/services/MetricsService.java 
7917f32e5 
  webapp/src/main/java/org/apache/atlas/web/security/AtlasSecurityConfig.java 
e74a9e93a 


Diff: https://reviews.apache.org/r/72879/diff/3/

Changes: https://reviews.apache.org/r/72879/diff/2-3/


Testing
---

Tested Statistics UI from different users with different policies for 
entityType and classfication.

https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/25/console


Thanks,

Nixon Rodrigues