[jira] [Commented] (ATLAS-1824) Cannot create hive lineage when using query "insert into table X select A.Y, B.Z from A,B"

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030557#comment-16030557
 ] 

ASF GitHub Bot commented on ATLAS-1824:
---

GitHub user developersLxz opened a pull request:

https://github.com/apache/incubator-atlas/pull/29

fix ATLAS-1824

fix bug  
https://issues.apache.org/jira/browse/ATLAS-1824

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/developersLxz/incubator-atlas fix-ATLAS-1824

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-atlas/pull/29.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #29


commit 30fc8ebe28b5d2c5d3c747ffcdd59ba0a0c938b4
Author: Linzs <850589...@qq.com>
Date:   2017-05-31T02:11:27Z

fix ATLAS-1824




> Cannot create hive lineage when using query "insert into table X select A.Y, 
> B.Z from A,B"
> --
>
> Key: ATLAS-1824
> URL: https://issues.apache.org/jira/browse/ATLAS-1824
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.7-incubating, 0.8-incubating, 0.7.1-incubating, 
> 0.9-incubating, 0.8.1-incubating
>Reporter: qinglin,xia
>  Labels: atlas
> Attachments: 0001-fix-ATLAS-1824.patch, 
> failed_lineage_input_output.png, failed_lineage.png
>
>
> Problem: 
> I am using atlas to create hive lineage, I found that when using the hive 
> query "insert into table A select B.name ,C.name from B, C", the lineage 
> between A, B and C cannot be created. 
> Steps to reproduce: 
> 1. log in to the hive shell
> 2. create hive table a, b,c and insert into data into a,b and c
> 3. using query "insert into table datalake_demo.d9 select a.id as key, a.id, 
> a.aname ,b.bname,c.cname from datalake_demo.a,datalake_demo.b,datalake_demo.c 
> where a.id = b.id and b.id = c.id" to create hive table d9
> Expected Result:
> hive_process is created , input of the process is a, b, c, output of the 
> process is datalake_demo.d9, and the process could be shown on atlas UI
> Actual Result: 
> hive_process is created, input of the process is a, b, b, d9, output of the 
> process is null, and the process could not be shown on atlas UI, according to 
> the attached pic "failed_lineage.png" and "failed_lineage_input_output.png"
> PS: We are working on fixing the issue!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-atlas pull request #29: fix ATLAS-1824

[developersLxz]

GitHub user developersLxz opened a pull request:

https://github.com/apache/incubator-atlas/pull/29

fix ATLAS-1824

fix bug  
https://issues.apache.org/jira/browse/ATLAS-1824

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/developersLxz/incubator-atlas fix-ATLAS-1824

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-atlas/pull/29.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #29


commit 30fc8ebe28b5d2c5d3c747ffcdd59ba0a0c938b4
Author: Linzs <850589...@qq.com>
Date:   2017-05-31T02:11:27Z

fix ATLAS-1824




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (ATLAS-1824) Cannot create hive lineage when using query "insert into table X select A.Y, B.Z from A,B"

[Xinzhi,Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xinzhi,Luo updated ATLAS-1824:
--
Attachment: 0001-fix-ATLAS-1824.patch

This patch fix the bug

> Cannot create hive lineage when using query "insert into table X select A.Y, 
> B.Z from A,B"
> --
>
> Key: ATLAS-1824
> URL: https://issues.apache.org/jira/browse/ATLAS-1824
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.7-incubating, 0.8-incubating, 0.7.1-incubating, 
> 0.9-incubating, 0.8.1-incubating
>Reporter: qinglin,xia
>  Labels: atlas
> Attachments: 0001-fix-ATLAS-1824.patch, 
> failed_lineage_input_output.png, failed_lineage.png
>
>
> Problem: 
> I am using atlas to create hive lineage, I found that when using the hive 
> query "insert into table A select B.name ,C.name from B, C", the lineage 
> between A, B and C cannot be created. 
> Steps to reproduce: 
> 1. log in to the hive shell
> 2. create hive table a, b,c and insert into data into a,b and c
> 3. using query "insert into table datalake_demo.d9 select a.id as key, a.id, 
> a.aname ,b.bname,c.cname from datalake_demo.a,datalake_demo.b,datalake_demo.c 
> where a.id = b.id and b.id = c.id" to create hive table d9
> Expected Result:
> hive_process is created , input of the process is a, b, c, output of the 
> process is datalake_demo.d9, and the process could be shown on atlas UI
> Actual Result: 
> hive_process is created, input of the process is a, b, b, d9, output of the 
> process is null, and the process could not be shown on atlas UI, according to 
> the attached pic "failed_lineage.png" and "failed_lineage_input_output.png"
> PS: We are working on fixing the issue!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[NOTICE] Managing Podling Rosters

[John D. Ament]

Podlings,

I wanted to raise a point to all, hence the direct emails.

The Whimsy PMC has put together a roster tool for managing the committers
in a podling.  Rather than managing the committer list in your status file,
this will keep track and make sure we list out everyone's valid IDs.

If you haven't accessed it, it can be found at
https://whimsy.apache.org/roster/ppmc/

The Incubator guide has been updated to reflect this
http://incubator.apache.org/guides/ppmc.html#podling+rosters

Please reach out if you have any questions.

John

Podling Report Reminder - June 2017

[johndament]

Dear podling,

This email was sent by an automated system on behalf of the Apache
Incubator PMC. It is an initial reminder to give you plenty of time to
prepare your quarterly board report.

The board meeting is scheduled for Wed, 21 June 2017, 10:30 am PDT.
The report for your podling will form a part of the Incubator PMC
report. The Incubator PMC requires your report to be submitted 2 weeks
before the board meeting, to allow sufficient time for review and
submission (Wed, June 07).

Please submit your report with sufficient time to allow the Incubator
PMC, and subsequently board members to review and digest. Again, the
very latest you should submit your report is 2 weeks prior to the board
meeting.

Thanks,

The Apache Incubator PMC

Submitting your Report

--

Your report should contain the following:

*   Your project name
*   A brief description of your project, which assumes no knowledge of
the project or necessarily of its field
*   A list of the three most important issues to address in the move
towards graduation.
*   Any issues that the Incubator PMC or ASF Board might wish/need to be
aware of
*   How has the community developed since the last report
*   How has the project developed since the last report.
*   How does the podling rate their own maturity.

This should be appended to the Incubator Wiki page at:

https://wiki.apache.org/incubator/June2017

Note: This is manually populated. You may need to wait a little before
this page is created from a template.

Mentors
---

Mentors should review reports for their project(s) and sign them off on
the Incubator wiki page. Signing off reports shows that you are
following the project - projects that are not signed may raise alarms
for the Incubator PMC.

Incubator PMC

Re: Incubator report for June 2017 - Draft for review

[Madhan Neethiraj]

Hemanth,

Thanks for compiling the report. I updated the report (inline below) for the 
following:
 - included names of new contributors
 - fixed the URL for ‘Resolved JIRAs’ [3]
 - updated the counts for create/resolved JIRAs

Rest of the contents look good.

Thanks,
Madhan




On 5/29/17, 10:03 AM, "Hemanth Yamijala"  wrote:

Atlas community,

Please find attached the draft of the report to submit to Incubator as part 
of the quarterly reporting process. Please provide feedback on any factual 
errors, missing information etc.

We need to submit this by June 7th.

Thanks
hemanth
=

Atlas

Apache Atlas is a scalable and extensible set of core foundational 
governance
services that enables enterprises to effectively and efficiently meet their
compliance requirements within Hadoop and allows integration with the 
complete
enterprise data ecosystem

Atlas has been incubating since 2015-05-05.

Three most important issues to address in the move towards graduation:

We do not see any issues in moving towards graduation. The dev community 
feel that we are ready for graduation as discussed in this mail thread: 
https://s.apache.org/94xI. We have intimated the IPMC about this and are 
working towards answering some queries raised in the Incubator mailing list 
regarding the same.


Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

None that we are aware of.

How has the community developed since the last report?

1. Madhan Neethiraj was voted in to the Apache Atlas PPMC.
2. The following new committers were added:
 - Ayub Khan Pathan
 - Apoorv Naik
 - Kalyani Kashikar
 - Neeru Gupta
 - Nixon Rodriguez
 - Sarath Subramanian
3. Following contributors have been added to JIRA.
 - Jan Hentschel
 - Qinglin Xia
 - Laxmikanth Malladi
 - Shi Wang
 - Christopher Grote
 - Graham Wallis

How has the project developed since the last report?

1. Apache Atlas community released version 0.8 of the software on March 31 
2017. [1]
2. Several new features are in progress or have finished development 
including enhanced metadata and governance capabilities through typesystem 
improvements (modeling relationships, glossary), a Virtual Data Connector 
project, improvements to search capabilities, Export/Import of Atlas graph 
metadata etc.
3. 239 new issues were filed between March 1st 2017 and May 29th [2]. 189 
issues were resolved in the same time [3]

How would you assess the podling's maturity?
Please feel free to add your own commentary.

  [ ] Initial setup
  [ ] Working towards first release
  [ ] Community building
  [X] Nearing graduation
  [ ] Other:

Date of last release:

  2017-03-31

When were the last committers or PPMC members elected?

1. PPMC Members - Madhan Neetiraj  (March 5 2017)
2. Committer - Ayub Pathan (March 20 2017)

References:

[1] 
https://lists.apache.org/thread.html/739d520597ea0bf93e9b5649044bc6a92c4660bbc44ffdd94f5bcb21@%3Cdev.atlas.apache.org%3E
[2] 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20ATLAS%20AND%20created%20%3E%3D%202017-03-01%20AND%20created%20%3C%3D%202017-05-29
[3] 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20ATLAS%20AND%20resolutiondate%20%3E%3D%202017-03-01%20AND%20resolutiondate%20%3C%3D%202017-05-29%20ORDER%20BY%20resolutiondate%2C%20createdDate%20DESC



Signed-off-by:

  [ ](atlas) Arun Murthy
 Comments:
  [ ](atlas) Chris Douglas
 Comments:
  [ ](atlas) Jakob Homan
 Comments:
  [ ](atlas) Vinod Kumar Vavilapalli
 Comments:

IPMC/Shepherd notes:

Re: Review Request 56959: Import API: Added Support for Transforming Entities During Import

[Ashutosh Mestry]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56959/
---

(Updated May 30, 2017, 7:50 p.m.)


Review request for atlas and Madhan Neethiraj.


Changes
---

Updates:
- Addressed review comments.
- Renamed unit test suite to reflect functionality.


Bugs: ATLAS-1825
https://issues.apache.org/jira/browse/ATLAS-1825


Repository: atlas


Description
---

Please refer to [ATLAS-1825](https://issues.apache.org/jira/browse/ATLAS-1825) 
for scenarios and solution approach.

Impact
==
* New class _AtlasImportTransform_ is now part of _AtlasImportRequest_. This 
holds a map of entity-type to attribute and the tranforms to be applied.
* New class _AtlasImportTransformBuiler_ reads the requests specified in the 
query string of the import API and creates a _AtlasImportTransfor_ structure.
* _AtlasImportTransformBuiler_ has new classes that support transformations. 
They are:
* _replace_ Implements find and replace on attribute values.
* _lowercase_ Converts attribute value to lower case. 
* Additional classes that unit test these classes have been added.

Implementation Approach
===
* _ZipSource_ now accepts _AtlasImportTransform_ object. It applies the 
transform every entity that is fetched from the source.
* _Import API_ now accepts multi-part input. This allows for elegantly 
specifying import parameters as well as the binary input from the file.

CURL


*Setup*
Create file with these contents call it _importTransform.json_:
```javascript
{ "options": {
"transforms": {
"hive_db": {
"qualifiedName": [
"replace:@cl1:@cl2"
]
},
"hive_table": {
"qualifiedName": [
"lowercase",
"replace:@cl1:@cl2"
]
}
}
}
}
```

```
curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F request=@../docs/importTransform.json -F 
data=@../docs/Stocks-2.zip "http://localhost:21000/api/atlas/admin/import;
```

Note: _-g_ needs to be added to CURL calls to prevent gobble check. This is not 
required anymore.

Notice the output from the API, it has the _AtlasRequest_ that was passed as 
input.

Documentation
=
Need to update TWIKI documents.


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 
7530196d 
  pom.xml 4360604f 
  
repository/src/main/java/org/apache/atlas/repository/impexp/EntityAttributeTransformer.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java 
9ec15e00 
  
repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransformers.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransforms.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java 
87c9f0eb 
  
repository/src/test/java/org/apache/atlas/repository/impexp/AtlasImportRequestTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/impexp/EntityAttributeTransformerTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
 08e9ee81 
  
repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformersJSONTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformersTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/impexp/ZipSourceTest.java 
8b042d7a 
  webapp/pom.xml 4132912d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
01f9a860 


Diff: https://reviews.apache.org/r/56959/diff/8/

Changes: https://reviews.apache.org/r/56959/diff/7-8/


Testing
---

**Unit Test**
* Added tests for *AtlasImportTransformBuilder* and contained classes.
* Updated *ZipSourceTest*
* Added tests to *ImportServiceTest* to exercise new option.

**Functional Test**
* Small size data export & import.
* Medium size data export & import.

**Performance Tests**
Not exhaustive. Simple stop watch tests to compare against baseline.


Thanks,

Ashutosh Mestry

Build failed in Jenkins: apache-atlas-nightly #761

[Apache Jenkins Server]

See 


Changes:

[apoorvnaik] UT fixes

[madhan] ATLAS-1848: update UI to remove json2.js dependency

--
[...truncated 273.87 KB...]
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 54 source files to 

[WARNING] 
:
 Some input files use or override a deprecated API.
[WARNING] 
:
 Recompile with -Xlint:deprecation for details.
[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-repository 
---
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[WARNING] file.encoding cannot be set as system property, use 
-Dfile.encoding=... instead

---
 T E S T S
---
Running org.apache.atlas.repository.audit.HBaseBasedAuditRepositoryTest
Running org.apache.atlas.repository.audit.InMemoryAuditRepositoryTest
Running org.apache.atlas.repository.store.graph.AtlasEntityDefStoreV1Test
Running 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateHardDeleteV1Test
Running org.apache.atlas.util.CompiledQueryCacheKeyTest
Running org.apache.atlas.repository.store.graph.v1.HardDeleteHandlerV1Test
Running org.apache.atlas.repository.graph.TestIntSequence
Running org.apache.atlas.repository.store.graph.v1.SoftDeleteHandlerV1Test
Running org.apache.atlas.repository.impexp.ImportServiceTest
Running org.apache.atlas.repository.graph.GraphHelperMockTest
Running org.apache.atlas.repository.impexp.ImportServiceReportingTest
Running org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest
Running org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateV1Test
Running org.apache.atlas.repository.impexp.ZipSourceTest
Running org.apache.atlas.repository.audit.HBaseBasedAuditRepositoryHATest
Running org.apache.atlas.repository.graph.GraphBackedSearchIndexerTest
Running 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateSoftDeleteV1Test
Running org.apache.atlas.repository.graph.AbstractGremlinQueryOptimizerTest
Running org.apache.atlas.repository.BaseTest
Running org.apache.atlas.repository.graph.GraphBackedRepositorySoftDeleteTest
Running org.apache.atlas.repository.graph.GraphBackedSearchIndexerMockTest
Running org.apache.atlas.repository.store.graph.v1.AtlasDeleteHandlerV1Test
Running org.apache.atlas.repository.typestore.StoreBackedTypeCacheTest
Running org.apache.atlas.repository.impexp.ExportServiceTest
Running org.apache.atlas.repository.graph.GraphRepoMapperScaleTest
Running org.apache.atlas.repository.graph.Gremlin2QueryOptimizerTest
Running org.apache.atlas.repository.graph.ReverseReferenceUpdateHardDeleteTest
Running org.apache.atlas.repository.typestore.GraphBackedTypeStoreTest
Running org.apache.atlas.repository.impexp.ZipSinkTest
Running org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1Test
Running org.apache.atlas.repository.graph.ReverseReferenceUpdateSoftDeleteTest
Running org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStoreTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.307 sec - in 
org.apache.atlas.repository.impexp.ImportServiceReportingTest
Running org.apache.atlas.repository.graph.GraphHelperTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.751 sec - in 
org.apache.atlas.repository.audit.InMemoryAuditRepositoryTest
Running org.apache.atlas.repository.graph.Gremlin3QueryOptimizerTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.447 sec - in 
org.apache.atlas.repository.store.graph.v1.AtlasDeleteHandlerV1Test
Running org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.995 sec - in 
org.apache.atlas.repository.audit.HBaseBasedAuditRepositoryHATest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, 

Re: Review Request 56959: Import API: Added Support for Transforming Entities During Import

[Ashutosh Mestry]

> On May 30, 2017, 6:56 p.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/repository/impexp/EntityAttributeTransformer.java
> > Lines 29 (patched)
> > 
> >
> > possible NPE, add null checks for entityWithExtInfo

Added test.


- Ashutosh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56959/#review176328
---


On May 25, 2017, 9:32 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56959/
> ---
> 
> (Updated May 25, 2017, 9:32 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1825
> https://issues.apache.org/jira/browse/ATLAS-1825
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Please refer to 
> [ATLAS-1825](https://issues.apache.org/jira/browse/ATLAS-1825) for scenarios 
> and solution approach.
> 
> Impact
> ==
> * New class _AtlasImportTransform_ is now part of _AtlasImportRequest_. This 
> holds a map of entity-type to attribute and the tranforms to be applied.
> * New class _AtlasImportTransformBuiler_ reads the requests specified in the 
> query string of the import API and creates a _AtlasImportTransfor_ structure.
> * _AtlasImportTransformBuiler_ has new classes that support transformations. 
> They are:
> * _replace_ Implements find and replace on attribute values.
> * _lowercase_ Converts attribute value to lower case. 
> * Additional classes that unit test these classes have been added.
> 
> Implementation Approach
> ===
> * _ZipSource_ now accepts _AtlasImportTransform_ object. It applies the 
> transform every entity that is fetched from the source.
> * _Import API_ now accepts multi-part input. This allows for elegantly 
> specifying import parameters as well as the binary input from the file.
> 
> CURL
> 
> 
> *Setup*
> Create file with these contents call it _importTransform.json_:
> ```javascript
> { "options": {
> "transforms": {
> "hive_db": {
> "qualifiedName": [
> "replace:@cl1:@cl2"
> ]
> },
> "hive_table": {
> "qualifiedName": [
> "lowercase",
> "replace:@cl1:@cl2"
> ]
> }
> }
> }
> }
> ```
> 
> ```
> curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F request=@../docs/importTransform.json -F 
> data=@../docs/Stocks-2.zip "http://localhost:21000/api/atlas/admin/import;
> ```
> 
> Note: _-g_ needs to be added to CURL calls to prevent gobble check. This is 
> not required anymore.
> 
> Notice the output from the API, it has the _AtlasRequest_ that was passed as 
> input.
> 
> Documentation
> =
> Need to update TWIKI documents.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 
> 7530196d 
>   pom.xml a93b8ad9 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/EntityAttributeTransformer.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
>  9ec15e00 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransformers.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransforms.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java 
> 87c9f0eb 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasImportRequestTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/EntityAttributeTransformerTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  08e9ee81 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformersTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/TransformEvaluatorTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ZipSourceTest.java
>  8b042d7a 
>   
> repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java
>  d4900838 
>   webapp/pom.xml 4132912d 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 01f9a860 
> 
> 
> Diff: https://reviews.apache.org/r/56959/diff/7/
> 
> 
> Testing
> ---
> 
> **Unit Test**
> * Added tests for *AtlasImportTransformBuilder* and contained classes.
> * Updated *ZipSourceTest*
> * Added tests to *ImportServiceTest* to exercise new option.
> 
> 

[jira] [Updated] (ATLAS-1848) UI- Remove json2.js library from atlas

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-1848:

Description: Update Atlas to avoid using JSON licensed work; please refer 
to https://www.apache.org/legal/resolved#json for more details.

> UI- Remove json2.js library from atlas
> --
>
> Key: ATLAS-1848
> URL: https://issues.apache.org/jira/browse/ATLAS-1848
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating, 0.9-incubating
>Reporter: Kalyani Kashikar
>Assignee: Kalyani Kashikar
> Fix For: 0.8-incubating, 0.9-incubating
>
> Attachments: ATLAS-1848.patch
>
>
> Update Atlas to avoid using JSON licensed work; please refer to 
> https://www.apache.org/legal/resolved#json for more details.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 56959: Import API: Added Support for Transforming Entities During Import

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56959/#review176328
---




repository/src/main/java/org/apache/atlas/repository/impexp/EntityAttributeTransformer.java
Lines 29 (patched)


possible NPE, add null checks for entityWithExtInfo



repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransformers.java
Lines 67 (patched)


this line could be part of a new else block above. It would be good to pass 
the "key" string in the error message to know the invalid transformer name in 
log.



repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java
Line 31 (original), 31 (patched)


revert these changes, it is already committed.


- Sarath Subramanian


On May 25, 2017, 2:32 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56959/
> ---
> 
> (Updated May 25, 2017, 2:32 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1825
> https://issues.apache.org/jira/browse/ATLAS-1825
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Please refer to 
> [ATLAS-1825](https://issues.apache.org/jira/browse/ATLAS-1825) for scenarios 
> and solution approach.
> 
> Impact
> ==
> * New class _AtlasImportTransform_ is now part of _AtlasImportRequest_. This 
> holds a map of entity-type to attribute and the tranforms to be applied.
> * New class _AtlasImportTransformBuiler_ reads the requests specified in the 
> query string of the import API and creates a _AtlasImportTransfor_ structure.
> * _AtlasImportTransformBuiler_ has new classes that support transformations. 
> They are:
> * _replace_ Implements find and replace on attribute values.
> * _lowercase_ Converts attribute value to lower case. 
> * Additional classes that unit test these classes have been added.
> 
> Implementation Approach
> ===
> * _ZipSource_ now accepts _AtlasImportTransform_ object. It applies the 
> transform every entity that is fetched from the source.
> * _Import API_ now accepts multi-part input. This allows for elegantly 
> specifying import parameters as well as the binary input from the file.
> 
> CURL
> 
> 
> *Setup*
> Create file with these contents call it _importTransform.json_:
> ```javascript
> { "options": {
> "transforms": {
> "hive_db": {
> "qualifiedName": [
> "replace:@cl1:@cl2"
> ]
> },
> "hive_table": {
> "qualifiedName": [
> "lowercase",
> "replace:@cl1:@cl2"
> ]
> }
> }
> }
> }
> ```
> 
> ```
> curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F request=@../docs/importTransform.json -F 
> data=@../docs/Stocks-2.zip "http://localhost:21000/api/atlas/admin/import;
> ```
> 
> Note: _-g_ needs to be added to CURL calls to prevent gobble check. This is 
> not required anymore.
> 
> Notice the output from the API, it has the _AtlasRequest_ that was passed as 
> input.
> 
> Documentation
> =
> Need to update TWIKI documents.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 
> 7530196d 
>   pom.xml a93b8ad9 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/EntityAttributeTransformer.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
>  9ec15e00 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransformers.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransforms.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java 
> 87c9f0eb 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasImportRequestTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/EntityAttributeTransformerTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  08e9ee81 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformersTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/TransformEvaluatorTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ZipSourceTest.java
>  8b042d7a 
>   
> 

[jira] [Commented] (ATLAS-1848) UI- Remove json2.js library from atlas

[Ashutosh Mestry (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029855#comment-16029855
 ] 

Ashutosh Mestry commented on ATLAS-1848:


Changes look good. Thanks!

I suppose this was necessary since Atlas does not support browsers IE7 and 
below. It would helpful if you could update Description with some details.

> UI- Remove json2.js library from atlas
> --
>
> Key: ATLAS-1848
> URL: https://issues.apache.org/jira/browse/ATLAS-1848
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating, 0.9-incubating
>Reporter: Kalyani Kashikar
>Assignee: Kalyani Kashikar
> Fix For: 0.8-incubating, 0.9-incubating
>
> Attachments: ATLAS-1848.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1390) IBM Graph implementation of graph DB abstraction

[Apoorv Naik (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029833#comment-16029833
 ] 

Apoorv Naik commented on ATLAS-1390:


[~grahamwallis] is validating this change. There's another JIRA ATLAS-1757 for 
tracking the issues with different graph builds.

> IBM Graph implementation of graph DB abstraction
> 
>
> Key: ATLAS-1390
> URL: https://issues.apache.org/jira/browse/ATLAS-1390
> Project: Atlas
>  Issue Type: New Feature
>Reporter: David Kantor
>Assignee: Jeffrey Hagelberg
> Attachments: rb56724.patch
>
>
> Support running Atlas against IBM Graph as the graph backend via IBM Graph 
> implementation of the graph DB abstraction.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1825) Import API: Ability to Apply Transforms to Attributes During Import

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-1825:
---
Attachment: (was: ATLAS-1825-Import-transform.2.patch)

> Import API: Ability to Apply Transforms to Attributes During Import
> ---
>
> Key: ATLAS-1825
> URL: https://issues.apache.org/jira/browse/ATLAS-1825
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
> Attachments: ATLAS-1825-Import-transform.4.patch
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> h3. Scenarios
> *Case 1*
> In a situation where, data exported from a cluster is imported to a different 
> cluster, it is necessary to make changes to attributes of entities such that 
> their values represent their environment
> Example: Entities exported from cluster called _'cl1'_ will have _@cl1_ in 
> their qualifiedName attribute. When these entities are imported to a cluster 
> called _'cl2'_ the qualifiedName attribute should read _@cl2_ to correctly 
> reflect their new environment.
> *Case 2*
> Exported data contains an attribute that has value that does not conform to 
> the conventions followed within the cluster. This can be corrected by 
> exporting the data and reimporting it with an applied transform.
> Example: Entities were created with an attribute value as upper case, this 
> needs to be lower case to conform to a convention.
> h3. Solution
> As part of Import API, it will help to be able to specify in a generic manner 
> the transforms that could be applied to an entity type.
> The implementation should be extensible so that new transforms could be added 
> in future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1825) Import API: Ability to Apply Transforms to Attributes During Import

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-1825:
---
Attachment: ATLAS-1825-Import-transform.4.patch

> Import API: Ability to Apply Transforms to Attributes During Import
> ---
>
> Key: ATLAS-1825
> URL: https://issues.apache.org/jira/browse/ATLAS-1825
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
> Attachments: ATLAS-1825-Import-transform.4.patch
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> h3. Scenarios
> *Case 1*
> In a situation where, data exported from a cluster is imported to a different 
> cluster, it is necessary to make changes to attributes of entities such that 
> their values represent their environment
> Example: Entities exported from cluster called _'cl1'_ will have _@cl1_ in 
> their qualifiedName attribute. When these entities are imported to a cluster 
> called _'cl2'_ the qualifiedName attribute should read _@cl2_ to correctly 
> reflect their new environment.
> *Case 2*
> Exported data contains an attribute that has value that does not conform to 
> the conventions followed within the cluster. This can be corrected by 
> exporting the data and reimporting it with an applied transform.
> Example: Entities were created with an attribute value as upper case, this 
> needs to be lower case to conform to a convention.
> h3. Solution
> As part of Import API, it will help to be able to specify in a generic manner 
> the transforms that could be applied to an entity type.
> The implementation should be extensible so that new transforms could be added 
> in future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Build failed in Jenkins: apache-atlas-nightly #760

[Apache Jenkins Server]

See 


Changes:

[kalyani.kashikar] ATLAS-1817.1 : UI :Fix for TagLayout page when tag is not 
associated

--
[...truncated 273.49 KB...]
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 54 source files to 

[WARNING] 
:
 Some input files use or override a deprecated API.
[WARNING] 
:
 Recompile with -Xlint:deprecation for details.
[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-repository 
---
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[WARNING] file.encoding cannot be set as system property, use 
-Dfile.encoding=... instead

---
 T E S T S
---
Running org.apache.atlas.repository.impexp.ImportServiceReportingTest
Running org.apache.atlas.repository.impexp.ZipSinkTest
Running org.apache.atlas.repository.impexp.ExportServiceTest
Running org.apache.atlas.repository.store.graph.AtlasEntityDefStoreV1Test
Running org.apache.atlas.repository.impexp.ZipSourceTest
Running org.apache.atlas.repository.typestore.StoreBackedTypeCacheTest
Running org.apache.atlas.repository.audit.InMemoryAuditRepositoryTest
Running org.apache.atlas.repository.typestore.GraphBackedTypeStoreTest
Running org.apache.atlas.repository.BaseTest
Running org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest
Running org.apache.atlas.repository.audit.HBaseBasedAuditRepositoryTest
Running org.apache.atlas.repository.audit.HBaseBasedAuditRepositoryHATest
Running org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1Test
Running org.apache.atlas.repository.impexp.ImportServiceTest
Running org.apache.atlas.repository.graph.GraphHelperMockTest
Running org.apache.atlas.repository.graph.Gremlin2QueryOptimizerTest
Running org.apache.atlas.repository.graph.AbstractGremlinQueryOptimizerTest
Running 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateHardDeleteV1Test
Running org.apache.atlas.repository.graph.GraphRepoMapperScaleTest
Running org.apache.atlas.repository.store.graph.v1.HardDeleteHandlerV1Test
Running org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryTest
Running org.apache.atlas.repository.graph.GraphBackedSearchIndexerTest
Running org.apache.atlas.repository.graph.GraphBackedRepositorySoftDeleteTest
Running org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStoreTest
Running org.apache.atlas.repository.graph.Gremlin3QueryOptimizerTest
Running org.apache.atlas.repository.store.graph.v1.AtlasDeleteHandlerV1Test
Running org.apache.atlas.repository.store.graph.v1.SoftDeleteHandlerV1Test
Running org.apache.atlas.repository.graph.TestIntSequence
Running org.apache.atlas.repository.graph.ReverseReferenceUpdateHardDeleteTest
Running org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateV1Test
Running 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateSoftDeleteV1Test
Running org.apache.atlas.repository.graph.GraphHelperTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.424 sec - in 
org.apache.atlas.repository.impexp.ImportServiceReportingTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.806 sec - in 
org.apache.atlas.repository.BaseTest
Running org.apache.atlas.repository.graph.ReverseReferenceUpdateSoftDeleteTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.183 sec - in 
org.apache.atlas.repository.store.graph.v1.AtlasDeleteHandlerV1Test
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.661 sec - in 
org.apache.atlas.repository.audit.HBaseBasedAuditRepositoryHATest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.145 sec - in 
org.apache.atlas.repository.impexp.ZipSinkTest
Running org.apache.atlas.repository.graph.GraphBackedSearchIndexerMockTest

[jira] [Commented] (ATLAS-1817) UI : n+1 pages are created, even though only n*25 instances are retrieved.

[Kalyani Kashikar (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029376#comment-16029376
 ] 

Kalyani Kashikar commented on ATLAS-1817:
-

Committed patch on following branches

* master :  
https://git1-us-west.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=9d6ad848
* 0.8-incubating : 
https://git1-us-west.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=08c60a20


> UI : n+1 pages are created, even though only n*25 instances are retrieved.
> --
>
> Key: ATLAS-1817
> URL: https://issues.apache.org/jira/browse/ATLAS-1817
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: trunk, 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Kalyani Kashikar
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1817.1.patch, ATLAS-1817.patch
>
>
> 1.Created 50 tables.
> 2.First page showed 1-25 tables , second page 26 -50 tables .
> 3.Clicking Next button navigates to 3rd page with 26-50 as range with empty 
> response.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1848) UI- Remove json2.js library from atlas

[Kalyani Kashikar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kalyani Kashikar updated ATLAS-1848:

Attachment: ATLAS-1848.patch

> UI- Remove json2.js library from atlas
> --
>
> Key: ATLAS-1848
> URL: https://issues.apache.org/jira/browse/ATLAS-1848
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating, 0.9-incubating
>Reporter: Kalyani Kashikar
>Assignee: Kalyani Kashikar
> Fix For: 0.8-incubating, 0.9-incubating
>
> Attachments: ATLAS-1848.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1817) UI : n+1 pages are created, even though only n*25 instances are retrieved.

[Kalyani Kashikar (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029301#comment-16029301
 ] 

Kalyani Kashikar commented on ATLAS-1817:
-

Found regression for this issue.
TagLayout page shows blank when tag is not associated with any entity. 

Patch is available for this issue in ATLAS-1817.1.patch attached



> UI : n+1 pages are created, even though only n*25 instances are retrieved.
> --
>
> Key: ATLAS-1817
> URL: https://issues.apache.org/jira/browse/ATLAS-1817
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: trunk, 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Kalyani Kashikar
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1817.1.patch, ATLAS-1817.patch
>
>
> 1.Created 50 tables.
> 2.First page showed 1-25 tables , second page 26 -50 tables .
> 3.Clicking Next button navigates to 3rd page with 26-50 as range with empty 
> response.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (ATLAS-1817) UI : n+1 pages are created, even though only n*25 instances are retrieved.

[Kalyani Kashikar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kalyani Kashikar reopened ATLAS-1817:
-

> UI : n+1 pages are created, even though only n*25 instances are retrieved.
> --
>
> Key: ATLAS-1817
> URL: https://issues.apache.org/jira/browse/ATLAS-1817
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: trunk, 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Kalyani Kashikar
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1817.1.patch, ATLAS-1817.patch
>
>
> 1.Created 50 tables.
> 2.First page showed 1-25 tables , second page 26 -50 tables .
> 3.Clicking Next button navigates to 3rd page with 26-50 as range with empty 
> response.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1817) UI : n+1 pages are created, even though only n*25 instances are retrieved.

[Kalyani Kashikar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kalyani Kashikar updated ATLAS-1817:

Attachment: ATLAS-1817.1.patch

> UI : n+1 pages are created, even though only n*25 instances are retrieved.
> --
>
> Key: ATLAS-1817
> URL: https://issues.apache.org/jira/browse/ATLAS-1817
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: trunk, 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Kalyani Kashikar
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1817.1.patch, ATLAS-1817.patch
>
>
> 1.Created 50 tables.
> 2.First page showed 1-25 tables , second page 26 -50 tables .
> 3.Clicking Next button navigates to 3rd page with 26-50 as range with empty 
> response.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1827) HiveHookIT failures due to missing files in target folder

[Yussuf Shaikh (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029285#comment-16029285
 ] 

Yussuf Shaikh commented on ATLAS-1827:
--

Thanks [~sarath.ku...@gmail.com] for looking at this issue.
I use "mvn clean install" command to run the tests from parent directory.

> HiveHookIT failures due to missing files in target folder
> -
>
> Key: ATLAS-1827
> URL: https://issues.apache.org/jira/browse/ATLAS-1827
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: 0.9-incubating
> Environment: # cat /etc/lsb-release
> DISTRIB_ID=Ubuntu
> DISTRIB_RELEASE=14.04
> DISTRIB_CODENAME=trusty
> DISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS"
> # uname -a
> Linux 3b9700711ca1 3.19.0-37-generic #42-Ubuntu SMP Fri Nov 20 18:22:05 UTC 
> 2015 x86_64 x86_64 x86_64 GNU/Linux
>Reporter: Yussuf Shaikh
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-1827.patch
>
>
> Below tests fail because Hive cannot read the test files from 
> addons/hive-bridge/target folder.
> org.apache.atlas.hive.hook.HiveHookIT.testLoadDFSPathPartitioned
> org.apache.atlas.hive.hook.HiveHookIT.testLoadLocalPath
> org.apache.atlas.hive.hook.HiveHookIT.testLoadLocalPathIntoPartition
> Stack trace for testLoadLocalPath:
> 2017-05-24 06:37:37,150 ERROR - [main:] ~ FAILED: SemanticException Line 1:23 
> Invalid path 
> ''file:///var/lib/jenkins/workspace/atlas-test/addons/hive-bridge/./target/load-data-7KV7AalW6D'':
>  No files matching path 
> file:/var/lib/jenkins/workspace/atlas-test/addons/hive-bridge/target/load-data-7KV7AalW6D
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:23 Invalid path 
> ''file:///var/lib/jenkins/workspace/atlas-test/addons/hive-bridge/./target/load-data-7KV7AalW6D'':
>  No files matching path 
> file:/var/lib/jenkins/workspace/atlas-test/addons/hive-bridge/target/load-data-7KV7AalW6D
>   at 
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.applyConstraints(LoadSemanticAnalyzer.java:139)
>   at 
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:230)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
>   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
>   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
>   at 
> org.apache.atlas.hive.HiveITBase.runCommandWithDelay(HiveITBase.java:119)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1847) Open Metadata and Governance

[Mandy Chessell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell updated ATLAS-1847:
--
Description: 
This JIRA covers the architecture and tracking of the open metadata and 
governance initiative for Apache Atlas.

Open metadata and governance is a moon-shot type of project to create a open 
set of APIs, types and interchange protocols to allow all metadata repositories 
to share and exchange metadata.  From this common base, it adds governance, 
discovery and access frameworks to automate the collection, management and use 
of metadata across an enterprise.  The result is an enterprise catalog of data 
resources that are transparently assessed, governed and used in order to 
deliver maximum value to the enterprise.

Delivering this capability as open source is a critical part of the project 
since multiple vendors must buy into this ecosystem.  They are not going to do 
this if one organization dominates the technology base.  Thus the open metadata 
and governance technology must be freely available with an open source 
governance model that allows a community of organizations and practitioners to 
develop and evolve the base and then use it in their offerings and deployments.

The proposal is to use Apache Atlas as the open source reference implementation 
for open metadata and governance.  Apache Alas will support an open metadata 
and governance compliant repository plus provide the adapters and interchange 
formats to allow other metadata repositories to connect into the ecosystem.

  was:This JIRA covers the architecture and tracking of the open metadata and 
governance initiative for Apache Atlas.


> Open Metadata and Governance
> 
>
> Key: ATLAS-1847
> URL: https://issues.apache.org/jira/browse/ATLAS-1847
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>  Labels: OpenMetadata, open-metadata
>
> This JIRA covers the architecture and tracking of the open metadata and 
> governance initiative for Apache Atlas.
> Open metadata and governance is a moon-shot type of project to create a open 
> set of APIs, types and interchange protocols to allow all metadata 
> repositories to share and exchange metadata.  From this common base, it adds 
> governance, discovery and access frameworks to automate the collection, 
> management and use of metadata across an enterprise.  The result is an 
> enterprise catalog of data resources that are transparently assessed, 
> governed and used in order to deliver maximum value to the enterprise.
> Delivering this capability as open source is a critical part of the project 
> since multiple vendors must buy into this ecosystem.  They are not going to 
> do this if one organization dominates the technology base.  Thus the open 
> metadata and governance technology must be freely available with an open 
> source governance model that allows a community of organizations and 
> practitioners to develop and evolve the base and then use it in their 
> offerings and deployments.
> The proposal is to use Apache Atlas as the open source reference 
> implementation for open metadata and governance.  Apache Alas will support an 
> open metadata and governance compliant repository plus provide the adapters 
> and interchange formats to allow other metadata repositories to connect into 
> the ecosystem.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ATLAS-1848) UI- Remove json2.js library from atlas

[Kalyani Kashikar (JIRA)]

Kalyani Kashikar created ATLAS-1848:
---

 Summary: UI- Remove json2.js library from atlas
 Key: ATLAS-1848
 URL: https://issues.apache.org/jira/browse/ATLAS-1848
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.8-incubating, 0.9-incubating
Reporter: Kalyani Kashikar
Assignee: Kalyani Kashikar
 Fix For: 0.8-incubating, 0.9-incubating






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ATLAS-1847) Open Metadata and Governance

[Mandy Chessell (JIRA)]

Mandy Chessell created ATLAS-1847:
-

 Summary: Open Metadata and Governance
 Key: ATLAS-1847
 URL: https://issues.apache.org/jira/browse/ATLAS-1847
 Project: Atlas
  Issue Type: New Feature
Reporter: Mandy Chessell
Assignee: Mandy Chessell


This JIRA covers the architecture and tracking of the open metadata and 
governance initiative for Apache Atlas.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 58803: ATLAS-1390: IBM graph implementation

[Graham Wallis]

> On May 10, 2017, 2:28 p.m., Graham Wallis wrote:
> > I managed to get this to build and ran it against an IBM Graph instance. 
> > But there are a number of things that needed changes/workarounds to get it 
> > working:
> > 
> > 1. The atlas-graphdb-ibm-graph build relies on the java-8 build profile 
> > being activated so that it can set skipMain and skip to false, so that the 
> > ibm-graph component gets built.  The java-8 profile has an activation rule 
> > of:
> > 
> > 
> > java-8
> > 
> > 1.8
> > 
> > 
> >   .. etc
> > ..but even when running with a 1.8 Java environment the profile was not 
> > activated. It was therefore necessary to explicitly activate the profile, 
> > by including -Pjava-8 on the maven command line. With both the java-8 and 
> > ibm-graph profiles explicitly named, it worked and the ibm-graph component 
> > was built.
> > 
> > 2. When the ibm-graph profile is built, the resulting packaged jar contains 
> > both ibm-graph and titan0. The different graph providers are assumed to be 
> > mutually exclusive as indicated by the maven dependency exclusions (e.g. if 
> > titan0, exclude titan0 and ibm-graph). There must be an unintended 
> > transitive dependency on titan0. I didn't bother to fix this (in the pom 
> > files) because I'm not sure that mutual exclusion is actually the best 
> > approach so I raised this as a question on JIRA ATLAS-1757.
> > 
> > 3. With the above changes, the ibm-graph profile builds, but when testing 
> > it (atlas_start.py followed by quick_start.py) there is an NPE when trying 
> > to loadGraphUpdateScript. This is because the readStream is null because 
> > the script file path is incorrectly specified: the web root is 
> > "src/main/resources" so the groovy script uri should be 
> > "graphUpdater.groovy" rather than "src/main/resources/graphUpdater.groovy".
> > 
> > 4. With the above changes, it is possible to connect to an IBM Graph 
> > instance and to run through the quick_start.py, but not completely cleanly. 
> > I detected the following errors:
> > 
> > 2017-05-09 15:42:53,586 INFO  - [pool-1-thread-10:] ~ 
> > test-1-graph:Test-1-Graph: e26d0d19-7163-4687-9608-6113d915cfc5:  begin 
> > transaction e26d0d19-7163-4687-9608-6113d915cfc5 (IBMGraphGraph:974)
> > 2017-05-09 15:42:53,587 INFO  - [pool-1-thread-10 - 
> > 55ad3ab1-8acb-4647-9874-5b7741b8d2b6:] ~ test-1-graph:Test-1-Graph: 
> > bf922368-abaa-4354-b942-d63b40044c56:  begin transaction 
> > bf922368-abaa-4354-b942-d63b40044c56 (IBMGraphGraph:974)
> > 2017-05-09 15:42:53,678 ERROR - [pool-1-thread-10 - 
> > 55ad3ab1-8acb-4647-9874-5b7741b8d2b6:] ~ Error handling a request: 
> > e417ee56dcfcd88d (ExceptionMapperUtil:32)
> > org.apache.atlas.ibmgraph.exception.BadRequestException: Request POST 
> > https://ibmgraph-alpha.eu-gb.bluemix.net/be6aad23-26ca-4776-9b2a-b3ccffa90f63/test-1-graph/gremlin
> > {"gremlin":"import java.util.function.Function;import 
> > org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.PartitionStrategy;import
> >  static org.apache.tinkerpop.gremlin.process.traversal.P.*;def 
> > g=graph.traversal();g.V().has('__typeName', T.in, typeNames) 
> > [startIdx.. >  failed.  Failure response: HttpResponse[Status = 400 - Bad Request, IBM 
> > Graph Request Id = c20f2b5e-b693-41f3-982c-2dc6c14791b5]
> > Response Body:
> > {"code":"BadRequestError","message":"[Static type checking] - No such 
> > property: in for class: java.lang.Class 
> > , [Static type checking] - Cannot 
> > find matching method 
> > org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal#has(java.lang.String,
> >  java.lang.Object, java.util.ArrayList ). 
> > Please check if the declared type is right and if the method exists."}
> > at 
> > org.apache.atlas.ibmgraph.http.HttpRequestHandler.handleFailures(HttpRequestHandler.java:250)
> > at 
> > org.apache.atlas.ibmgraph.http.HttpRequestHandler.translateResponse(HttpRequestHandler.java:199)
> > at 
> > org.apache.atlas.ibmgraph.http.HttpRequestHandler.processRequest_(HttpRequestHandler.java:140)
> > at 
> > org.apache.atlas.ibmgraph.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:108)
> > at 
> > org.apache.atlas.ibmgraph.api.GraphDatabaseClient.processMultiValuedRequest(GraphDatabaseClient.java:124)
> > at 
> > org.apache.atlas.ibmgraph.api.GraphDatabaseClient.executeGremlin(GraphDatabaseClient.java:239)
> > at 
> > org.apache.atlas.ibmgraph.api.GraphDatabaseClient.executeGremlin(GraphDatabaseClient.java:227)
> > 
> > I think the problem here is that Atlas is trying to use Tinkerpop 2 syntax 
> > with a Tinkerpop 3 graph; hence the confusion looking for an 'in' property 
> > for the class 'T'. In TP3, there are first-class predicates, such as 
> > within(), that I think should be used for this 

Re: [VOTE] Graduate Apache Atlas Project from Incubator

[Graham Wallis]

+1 from me.

Best regards,
  Graham

Graham Wallis
IBM Analytics Emerging Technology Center
Internet: graham_wal...@uk.ibm.com 
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44-1962-815356Tie: 7-245356




From:   Suma Shivaprasad 
To: dev@atlas.incubator.apache.org
Date:   25/05/2017 00:23
Subject:[VOTE] Graduate Apache Atlas Project from Incubator



Dear Atlas Community Members,

Following the discussion in Atlas community on graduating Apache Atlas
Project from Incubation to a Top Level Project, starting this
voting thread within the Atlas community. I appreciate mentors and
community members in making this happen and thanks for their
support. Apache Atlas Project has come a long way since entering into
incubation during May 2015 and the project summary shows the
details on this.

Please vote on the Apache Atlas Project resolution that is found in
bottom of this email.

[ ] +1 Graduate Apache Atlas from the Incubator.
[ ] +0 No opinion
[ ] -1 Don't graduate Apache Atlas from the Incubator ( please provide
the reason)

This VOTE will be opened for next 72 hours.

If this VOTE succeeds, a similar VOTE will be done in general@incubator
mailing list. If that succeeds, a resolution will be
included in the next Apache Board Meeting.

Here is my Vote +1 (binding)

Thanks all for your participation
Suma


Project Summary:

=
http://incubator.apache.org/projects/atlas.html

Project website:
=

http://atlas.incubator.apache.org

Project Documentation:
===

http://atlas.incubator.apache.org/index.html
Architecture - http://atlas.incubator.apache.org/Architecture.html
Installation guide - http://atlas.incubator.apache.org/InstallationSteps.
html
QuickStart Guide- http://atlas.incubator.apache.org/QuickStart.html
REST API - http://atlas.incubator.apache.org/api/v2/index.html
Atlas Release Guide - https://cwiki.apache.org/confluence/display/ATLAS/
Developer+Resources

Project maturity Assessment:
===

https://cwiki.apache.org/confluence/display/ATLAS/
Apache+Atlas+Project+Maturity+Model

Proposed PMC size: 38

Total number of committers  : 38 members
Total number of contributors : 62 members

PMC affiliation (* indicated chair)

* Hortonworks (13)
   IBM (3)
   Aetna(4)
   Freestone (3)
   Merck (2)
   Oracle(2)
   Target (2)
   Schlumberger(2)
   Dataguise(1)
   Inmobi(1)
   JPMC(1)
   MPR(1)
   SAP(1)
   SparklineData(1)
   Others(1)

1660 commits on master
80 contributors across all branches
Dev list averaged ~500-800 msgs/month in 2017
User list was created recently in April 2017. Hence not considering stats
for this.
1827 issues created

Committer¹s affiliation:
===
* Hortonworks (13)
   IBM (3)
   Aetna(4)
   Freestone (3)
   Merck (2)
   Oracle(2)
   Target (2)
   Schlumberger(2)
   Dataguise(1)
   Inmobi(1)
   JPMC(1)
   MPR(1)
   SAP(1)
   SparklineData(1)
   Others(1)


Apache Atlas Top Level Project Resolution:


Establish the Apache Atlas Project

WHEREAS, the Board of Directors deems it to be in the best interests of
the Foundation and consistent with the Foundation¹s purpose to establish a
Project Management Committee charged with the creation and maintenance of
open-source software, for distribution at no charge to the public, related
to a data management platform That provides real-time, consistent access
to data-intensive applications throughout widely distributed cloud
architectures.

NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
(PMC), to be known as the "Apache Atlas Project", be and hereby is
established pursuant to Bylaws of the Foundation; and be it further
RESOLVED,that the Apache Atlas Project be and hereby is responsible for
the creation and maintenance of software related to a data management
platform that provides real-time, consistent access to data-intensive
applications throughout widely distributed cloud architectures.

RESOLVED, that the office of "Vice President, Apache Atlas" be and
hereby is created, the person holding such office to serve at the
direction of the Board of Directors as the chair of the Apache Atlas
Project, and to have primary responsibility for management of the projects
within the scope of responsibility of the Apache Atlas Project; and be it
Further.

>RESOLVED,that the persons listed immediately below be and hereby are
>appointed to serve as the initial members of the Apache Atlas Project:

Aaron Dossett adoss...@target.com 
Andrew Ahn a...@apache.org 
Anil Varma ava...@slb.com 
Arun C. Murthy acmur...@apache.org 
Apoorv Naik apoorvn...@apache.org 
Ayub Pathan ayubpat...@apache.org 
Barbara Stortz bsto...@sap.com 
Chris Hyzer chy...@aetna.com 
Daniel