[jira] [Updated] (ATLAS-2346) dataTypeName field missing in AttributeDefinition returned by V1 REST API

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-2346:

Attachment: ATLAS-2346.patch

Review board URL: https://reviews.apache.org/r/65004/

> dataTypeName field missing in AttributeDefinition returned by V1 REST API
> -
>
> Key: ATLAS-2346
> URL: https://issues.apache.org/jira/browse/ATLAS-2346
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sharmadha Sainath
>Assignee: Madhan Neethiraj
> Attachments: ATLAS-2346.patch
>
>
> Type information returned by api/atlas/types/hive_table doesn't include field 
> dataTypeName in AttributeDefinition, as shown below:
> {noformat}
> curl -f -X GET -u username:password -H Accept: application/json -H 
> Content-Type: application/json 
> http://atlas-host:21000/api/atlas/types/hive_table
> {noformat}
> {noformat}
> {
>   "requestId": "pool-2-thread-10 - e9419c0f-6fc1-4df3-8a40-c6c882ce63d7",
>   "typeName": "hive_table",
>   "definition": {
> "classTypes": [
>   {
> "attributeDefinitions": [
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "db"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "createTime"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "lastAccessTime"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "comment"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "retention"
>   },
>   {
> "isComposite": true,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "sd"
>   },
>   {
> "isComposite": true,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "partitionKeys"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "aliases"
>   },
>   {
> "isComposite": true,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "columns"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "parameters"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "viewOriginalText"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "viewExpandedText"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "tableType"
>   },
>   {
> "isComposite": false,
> "isIndexable": true,
> "isUnique": false,
> "multiplicity": "required",
> "name": "temporary"
>   }
> ],
> "hierarchicalMetaTypeName": 
> "org.apache.atlas.typesystem.types.ClassType",
> "superTypes": [
>   "DataSet"
> ],
> "typeDescription": "hive_table",
> "typeName": "hive_table",
> "typeVersion": "1.1"
>   }
> ],
> "enumTypes": null,
> "structTypes": null,
> "traitTypes": null
>   }
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 65004: ATLAS-2346: fix V1 REST API to include AttributeDefinition.dataTypeName field in type details

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65004/
---

Review request for atlas.


Bugs: ATLAS-2346
https://issues.apache.org/jira/browse/ATLAS-2346


Repository: atlas


Description
---

fixed V2=>V1 conversion of type attributes to correctly populate all fields


Diffs
-

  intg/src/main/java/org/apache/atlas/v1/model/typedef/AttributeDefinition.java 
53a008a8 
  
repository/src/main/java/org/apache/atlas/repository/converters/TypeConverterUtil.java
 33f092e4 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
 f1d90313 


Diff: https://reviews.apache.org/r/65004/diff/1/


Testing
---

Verified that V1 REST API returns correct values for dataTypeName field


Thanks,

Madhan Neethiraj

[jira] [Updated] (ATLAS-2346) dataTypeName field missing in AttributeDefinition returned by V1 REST API

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-2346:

Reporter: Sharmadha Sainath  (was: Madhan Neethiraj)

> dataTypeName field missing in AttributeDefinition returned by V1 REST API
> -
>
> Key: ATLAS-2346
> URL: https://issues.apache.org/jira/browse/ATLAS-2346
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sharmadha Sainath
>Assignee: Madhan Neethiraj
>
> Type information returned by api/atlas/types/hive_table doesn't include field 
> dataTypeName in AttributeDefinition, as shown below:
> {noformat}
> curl -f -X GET -u username:password -H Accept: application/json -H 
> Content-Type: application/json 
> http://atlas-host:21000/api/atlas/types/hive_table
> {noformat}
> {noformat}
> {
>   "requestId": "pool-2-thread-10 - e9419c0f-6fc1-4df3-8a40-c6c882ce63d7",
>   "typeName": "hive_table",
>   "definition": {
> "classTypes": [
>   {
> "attributeDefinitions": [
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "db"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "createTime"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "lastAccessTime"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "comment"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "retention"
>   },
>   {
> "isComposite": true,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "sd"
>   },
>   {
> "isComposite": true,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "partitionKeys"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "aliases"
>   },
>   {
> "isComposite": true,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "columns"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "parameters"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "viewOriginalText"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "viewExpandedText"
>   },
>   {
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": "required",
> "name": "tableType"
>   },
>   {
> "isComposite": false,
> "isIndexable": true,
> "isUnique": false,
> "multiplicity": "required",
> "name": "temporary"
>   }
> ],
> "hierarchicalMetaTypeName": 
> "org.apache.atlas.typesystem.types.ClassType",
> "superTypes": [
>   "DataSet"
> ],
> "typeDescription": "hive_table",
> "typeName": "hive_table",
> "typeVersion": "1.1"
>   }
> ],
> "enumTypes": null,
> "structTypes": null,
> "traitTypes": null
>   }
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2346) dataTypeName field missing in AttributeDefinition returned by V1 REST API

[Madhan Neethiraj (JIRA)]

Madhan Neethiraj created ATLAS-2346:
---

 Summary: dataTypeName field missing in AttributeDefinition 
returned by V1 REST API
 Key: ATLAS-2346
 URL: https://issues.apache.org/jira/browse/ATLAS-2346
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.0.0
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Type information returned by api/atlas/types/hive_table doesn't include field 
dataTypeName in AttributeDefinition, as shown below:

{noformat}
curl -f -X GET -u username:password -H Accept: application/json -H 
Content-Type: application/json 
http://atlas-host:21000/api/atlas/types/hive_table
{noformat}

{noformat}
{
  "requestId": "pool-2-thread-10 - e9419c0f-6fc1-4df3-8a40-c6c882ce63d7",
  "typeName": "hive_table",
  "definition": {
"classTypes": [
  {
"attributeDefinitions": [
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "db"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "createTime"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "lastAccessTime"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "comment"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "retention"
  },
  {
"isComposite": true,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "sd"
  },
  {
"isComposite": true,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "partitionKeys"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "aliases"
  },
  {
"isComposite": true,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "columns"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "parameters"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "viewOriginalText"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "viewExpandedText"
  },
  {
"isComposite": false,
"isIndexable": false,
"isUnique": false,
"multiplicity": "required",
"name": "tableType"
  },
  {
"isComposite": false,
"isIndexable": true,
"isUnique": false,
"multiplicity": "required",
"name": "temporary"
  }
],
"hierarchicalMetaTypeName": 
"org.apache.atlas.typesystem.types.ClassType",
"superTypes": [
  "DataSet"
],
"typeDescription": "hive_table",
"typeName": "hive_table",
"typeVersion": "1.1"
  }
],
"enumTypes": null,
"structTypes": null,
"traitTypes": null
  }
}
{noformat}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 65001: Atlas Advanced Search: Phase 3: Count, Sum, Max, Min with Group By Implementation

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65001/#review194922
---




repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java
Lines 856 (patched)


Update variable names for more clarity



repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java
Lines 861 (patched)


Same here. Rename assign2 -> assign



repository/src/main/java/org/apache/atlas/query/IdentifierHelper.java
Lines 93 (patched)


equals -> equalsIgnoreCase



repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParser.java
Line 1 (original), 1 (patched)


Remove user information from the file header



repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParserBaseVisitor.java
Line 1 (original), 1 (patched)


Same here



repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParserVisitor.java
Line 1 (original), 1 (patched)


Same here



repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java
Lines 282 (patched)


I think we should disallow such constructs, 

The query is ambiguous when used with = operator,

I'm not sure whether the user wants to do an exact match for the array of 
values or wants to match either value in that array.

We should update the grammar to have a IN clause for such cases


- Apoorv Naik


On Jan. 6, 2018, 1:26 a.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65001/
> ---
> 
> (Updated Jan. 6, 2018, 1:26 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2229
> https://issues.apache.org/jira/browse/ATLAS-2229
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background**
> This is the subsequent commit for the new Antlr-based DSL implementation.
> 
> This builds on top of what is already present.
> 
> **Approach**
> The implementation of these operators does not impact the gremlin queries 
> generated without these operators. Alternatively stated, the approach applies 
> the new implementation only when on or any of these clauses are present 
> within the DSL query.
> 
> Basic Gremlin is constructed. The 'select' clause is processed for one of 
> these operations. Additional closures are constructed and the filtered part 
> of the query is passed to the closure.
> 
> **Bug fixes**
> - Updates include pagination across results. This causes the query to fire 
> again with the appropriate values for _limit_ and _offset_ clauses.
> - Fixed handling of date and boolean (true and false) values in _where_ 
> clause.
> - Error handling during parsing.
> - Handling errors for invalid queries.
> 
> **Documentation**
> - Updated documentation, currently in draft. Separate patch will include 
> updates to TWIKI.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  cbe9a5b5 
>   repository/src/main/java/org/apache/atlas/query/AtlasDSL.java e4c81653 
>   repository/src/main/java/org/apache/atlas/query/DSLVisitor.java a21e8843 
>   repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java 
> 6af1db67 
>   repository/src/main/java/org/apache/atlas/query/IdentifierHelper.java 
> fe1c5bdc 
>   repository/src/main/java/org/apache/atlas/query/QueryParams.java 5af8bc7b 
>   repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLLexer.java 
> 77c66c99 
>   repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParser.g4 
> 72d08e19 
>   repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParser.java 
> e557c4af 
>   
> repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParserBaseVisitor.java
>  4423c0e9 
>   
> repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParserVisitor.java
>  8c59d7a6 
>   repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java 
> af4dc3e0 
>   
> repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java 
> ccf1cb29 
>   
> repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java
>  8a06d949 
> 
> 
> Diff: https://reviews.apache.org/r/65001/diff/1/
> 
> 
> Testing
> ---
> 
> **Unit test**
> - Updated _GremlinComposerTest_ to include the new operations.
> - Added 25+ new tests to _DSLQueriesTest_. See section _minMa

Review Request 65001: Atlas Advanced Search: Phase 3: Count, Sum, Max, Min with Group By Implementation

[Ashutosh Mestry]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65001/
---

Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath Subramanian.


Bugs: ATLAS-2229
https://issues.apache.org/jira/browse/ATLAS-2229


Repository: atlas


Description
---

**Background**
This is the subsequent commit for the new Antlr-based DSL implementation.

This builds on top of what is already present.

**Approach**
The implementation of these operators does not impact the gremlin queries 
generated without these operators. Alternatively stated, the approach applies 
the new implementation only when on or any of these clauses are present within 
the DSL query.

Basic Gremlin is constructed. The 'select' clause is processed for one of these 
operations. Additional closures are constructed and the filtered part of the 
query is passed to the closure.

**Bug fixes**
- Updates include pagination across results. This causes the query to fire 
again with the appropriate values for _limit_ and _offset_ clauses.
- Fixed handling of date and boolean (true and false) values in _where_ clause.
- Error handling during parsing.
- Handling errors for invalid queries.

**Documentation**
- Updated documentation, currently in draft. Separate patch will include 
updates to TWIKI.


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
cbe9a5b5 
  repository/src/main/java/org/apache/atlas/query/AtlasDSL.java e4c81653 
  repository/src/main/java/org/apache/atlas/query/DSLVisitor.java a21e8843 
  repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java 
6af1db67 
  repository/src/main/java/org/apache/atlas/query/IdentifierHelper.java 
fe1c5bdc 
  repository/src/main/java/org/apache/atlas/query/QueryParams.java 5af8bc7b 
  repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLLexer.java 
77c66c99 
  repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParser.g4 
72d08e19 
  repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParser.java 
e557c4af 
  
repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParserBaseVisitor.java
 4423c0e9 
  
repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParserVisitor.java
 8c59d7a6 
  repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java af4dc3e0 
  repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java 
ccf1cb29 
  
repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java
 8a06d949 


Diff: https://reviews.apache.org/r/65001/diff/1/


Testing
---

**Unit test**
- Updated _GremlinComposerTest_ to include the new operations.
- Added 25+ new tests to _DSLQueriesTest_. See section _minMaxCountProvider_.

**Functional testing**
- Via web.
- Verified using hdfs_path samples.
- Verified using quick_start dataset.
- Verified using typical hive_db dataset (with 110 tables, each with 87 
columns).

**REST API testing**
- V1 and V2 end points, via CURL.


Thanks,

Ashutosh Mestry

Build failed in Jenkins: Atlas-master-IntegrationTests #240

[Apache Jenkins Server]

See 


Changes:

[ssubramanian] ATLAS-2345: v1 schema API response includes partition keys in 
addition

[ssubramanian] ATLAS-2344: Creating an entity of unknown type throws 404 error 
code ,

--
[...truncated 404.99 KB...]
testOutputsGraph(org.apache.atlas.web.integration.DataSetLineageJerseyResourceIT)
  Time elapsed: 0.015 sec  <<< FAILURE!
java.lang.NullPointerException: null
at 
org.apache.atlas.web.integration.DataSetLineageJerseyResourceIT.testOutputsGraph(DataSetLineageJerseyResourceIT.java:115)

testOutputsGraphForEntity(org.apache.atlas.web.integration.DataSetLineageJerseyResourceIT)
  Time elapsed: 0.015 sec  <<< FAILURE!
java.lang.NullPointerException: null
at 
org.apache.atlas.web.integration.DataSetLineageJerseyResourceIT.testOutputsGraphForEntity(DataSetLineageJerseyResourceIT.java:136)

testSchema(org.apache.atlas.web.integration.DataSetLineageJerseyResourceIT)  
Time elapsed: 0.019 sec  <<< FAILURE!
java.lang.NullPointerException: null
at 
org.apache.atlas.web.integration.DataSetLineageJerseyResourceIT.testSchema(DataSetLineageJerseyResourceIT.java:161)

testSchemaForEntity(org.apache.atlas.web.integration.DataSetLineageJerseyResourceIT)
  Time elapsed: 0.014 sec  <<< FAILURE!
java.lang.NullPointerException: null
at 
org.apache.atlas.web.integration.DataSetLineageJerseyResourceIT.testSchemaForEntity(DataSetLineageJerseyResourceIT.java:167)

runQuickStart(org.apache.atlas.examples.QuickStartIT)  Time elapsed: 0.763 sec  
<<< FAILURE!
java.lang.AssertionError: null
at 
org.apache.atlas.examples.QuickStart.verifyTypesCreated(QuickStart.java:423)
at org.apache.atlas.examples.QuickStart.createTypes(QuickStart.java:142)
at 
org.apache.atlas.examples.QuickStart.runQuickstart(QuickStart.java:88)
at 
org.apache.atlas.examples.QuickStartIT.runQuickStart(QuickStartIT.java:43)

testSearchByDSLForUnknownType(org.apache.atlas.web.integration.EntityDiscoveryJerseyResourceIT)
  Time elapsed: 0.307 sec  <<< FAILURE!
org.testng.TestException: 
Method EntityDiscoveryJerseyResourceIT.testSearchByDSLForUnknownType()[pri:0, 
instance:org.apache.atlas.web.integration.EntityDiscoveryJerseyResourceIT@4950bba6]
 should have thrown an exception of type class 
org.apache.atlas.AtlasServiceException
at 
org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1426)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:701)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:845)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1153)
at 
org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108)
at org.testng.TestRunner.privateRun(TestRunner.java:771)
at org.testng.TestRunner.run(TestRunner.java:621)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:357)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:352)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:310)
at org.testng.SuiteRunner.run(SuiteRunner.java:259)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1199)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1124)
at org.testng.TestNG.run(TestNG.java:1032)
at 
org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
at 
org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:205)
at 
org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:108)
at 
org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray2(ReflectionUtils.java:208)
at 
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:155)
at 
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:81)
at 
org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.runSuitesInProcess(InPluginVMSurefireStarter.java:82)
at 
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:974)
at 
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChe

Build failed in Jenkins: Atlas-master-UnitTests #227

[Apache Jenkins Server]

See 


Changes:

[ssubramanian] ATLAS-2345: v1 schema API response includes partition keys in 
addition

[ssubramanian] ATLAS-2344: Creating an entity of unknown type throws 404 error 
code ,

--
[...truncated 187.45 KB...]
at 
org.apache.atlas.query.DSLVisitor.visitFromExpression(DSLVisitor.java:31)
at 
org.apache.atlas.query.antlr4.AtlasDSLParser$FromExpressionContext.accept(AtlasDSLParser.java:1711)
at 
org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at 
org.apache.atlas.query.antlr4.AtlasDSLParserBaseVisitor.visitSingleQrySrc(AtlasDSLParserBaseVisitor.java:244)
at 
org.apache.atlas.query.antlr4.AtlasDSLParser$SingleQrySrcContext.accept(AtlasDSLParser.java:1844)
at 
org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at 
org.apache.atlas.query.antlr4.AtlasDSLParserBaseVisitor.visitCommaDelimitedQueries(AtlasDSLParserBaseVisitor.java:258)
at 
org.apache.atlas.query.antlr4.AtlasDSLParser$CommaDelimitedQueriesContext.accept(AtlasDSLParser.java:1959)
at 
org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at 
org.apache.atlas.query.antlr4.AtlasDSLParserBaseVisitor.visitQuerySrc(AtlasDSLParserBaseVisitor.java:272)
at 
org.apache.atlas.query.antlr4.AtlasDSLParser$QuerySrcContext.accept(AtlasDSLParser.java:2069)
at 
org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at 
org.apache.atlas.query.antlr4.AtlasDSLParserBaseVisitor.visitQuery(AtlasDSLParserBaseVisitor.java:279)
at 
org.apache.atlas.query.antlr4.AtlasDSLParser$QueryContext.accept(AtlasDSLParser.java:2130)
at 
org.apache.atlas.query.AtlasDSL$Translator.translate(AtlasDSL.java:130)
at 
org.apache.atlas.discovery.EntityDiscoveryService.toGremlinQuery(EntityDiscoveryService.java:679)
at 
org.apache.atlas.discovery.EntityDiscoveryService.searchUsingDslQuery(EntityDiscoveryService.java:126)
at 
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:60)
at 
org.apache.atlas.services.EntityDiscoveryServiceTest.dslTest(EntityDiscoveryServiceTest.java:52)

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.899 sec - in 
org.apache.atlas.services.MetricsServiceTest
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.539 sec - 
in org.apache.atlas.repository.store.graph.v1.AtlasRelationshipDefStoreV1Test
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.883 sec - in 
org.apache.atlas.repository.store.graph.v1.AtlasEntityDefStoreV1Test
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.884 sec - in 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateSoftDeleteV1Test
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.728 sec - in 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateHardDeleteV1Test
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.784 sec - in 
org.apache.atlas.repository.store.graph.v1.AtlasRelationshipStoreHardDeleteV1Test
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 59.376 sec - in 
org.apache.atlas.repository.store.graph.v1.AtlasRelationshipStoreSoftDeleteV1Test
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 60.057 sec - 
in org.apache.atlas.repository.userprofile.UserProfileServiceTest
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 62.462 sec - 
in org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1Test
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.633 sec - 
in org.apache.atlas.repository.impexp.ExportServiceTest
Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.358 sec - 
in org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStoreTest
Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 67.251 sec <<< 
FAILURE! - in org.apache.atlas.repository.impexp.ImportServiceTest
importDB2(org.apache.atlas.repository.impexp.ImportServiceTest)  Time elapsed: 
0.555 sec  <<< FAILURE!
java.lang.AssertionError: entity:DB_v1 expected [true] but found [false]
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.failNotEquals(Assert.java:496)
at org.testng.Assert.assertTrue(Assert.java:42)
at 
org.apache.atlas.repository.impexp.ZipFileResourceTestUtils.verifyImportedMetrics(ZipFileResourceTestUtils.java:134)
at 
org.apache.atlas.repository.impexp.ZipFileResourceTestUtils.runAndVerifyQuickStart_v1_Import(ZipFileResourceTestUtils.java:217)
at 
org.apache.atlas.repository.impexp.ImportServiceTest.importDB2(ImportServiceTest.java:104)

Tests run: 212, Failures: 2, Errors:

[jira] [Updated] (ATLAS-2345) api/atlas/lineage//schema response["results"] ["rows"] has partition keys also in addition to columns

[Sarath Subramanian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2345:
--
Attachment: ATLAS-2345.1.patch

>  api/atlas/lineage//schema response["results"] ["rows"] has partition 
> keys also in addition to columns
> 
>
> Key: ATLAS-2345
> URL: https://issues.apache.org/jira/browse/ATLAS-2345
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
> Fix For: 1.0.0
>
> Attachments: ATLAS-2345.1.patch
>
>
> Response of /api/atlas/lineage//schema,
> response["results"]["rows"] had only columns and no partition keys.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2344) Creating an entity of unknown type throws 404 error code , Expected is 400

[Sarath Subramanian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2344:
--
Attachment: ATLAS-2344.1.patch

> Creating an entity of unknown type throws 404 error code , Expected is 400
> --
>
> Key: ATLAS-2344
> URL: https://issues.apache.org/jira/browse/ATLAS-2344
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
> Fix For: 1.0.0
>
> Attachments: ATLAS-2344.1.patch
>
>
> Creating an entity of a non-existing type throws 404 Error code . Expected is 
> 400 which has been the response code till the last release.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2345) api/atlas/lineage//schema response["results"] ["rows"] has partition keys also in addition to columns

[Sarath Subramanian (JIRA)]

Sarath Subramanian created ATLAS-2345:
-

 Summary:  api/atlas/lineage//schema response["results"] 
["rows"] has partition keys also in addition to columns
 Key: ATLAS-2345
 URL: https://issues.apache.org/jira/browse/ATLAS-2345
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.0.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.0.0


Response of /api/atlas/lineage//schema,
response["results"]["rows"] had only columns and no partition keys.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2344) Creating an entity of unknown type throws 404 error code , Expected is 400

[Sarath Subramanian (JIRA)]

Sarath Subramanian created ATLAS-2344:
-

 Summary: Creating an entity of unknown type throws 404 error code 
, Expected is 400
 Key: ATLAS-2344
 URL: https://issues.apache.org/jira/browse/ATLAS-2344
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.0.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.0.0


Creating an entity of a non-existing type throws 404 Error code . Expected is 
400 which has been the response code till the last release.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64982: ATLAS2341 amend the Area 1 models so the relationship names are unique

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64982/#review194889
---


Ship it!




Ship It!

- Madhan Neethiraj


On Jan. 5, 2018, 5:43 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64982/
> ---
> 
> (Updated Jan. 5, 2018, 5:43 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Mandy Chessell.
> 
> 
> Bugs: 2341
> https://issues.apache.org/jira/browse/2341
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> While working with the model files - I found that some of the relationship 
> end names are not unique. I think it is simpler to have these names unique. 
> This Jira is also used to remove some duplicated relationshipDef 
> "ProjectScope" and to add an empty attributeDef array for entityDef 
> "PrivateTag"
> 
> 
> Diffs
> -
> 
>   addons/models/0100-Area1-Collaboration/0135-Meetings.json 59998be4 
>   addons/models/0100-Area1-Collaboration/0140-Communities.json e6d7b868 
>   addons/models/0100-Area1-Collaboration/0150-Feedback.json 0115c64d 
> 
> 
> Diff: https://reviews.apache.org/r/64982/diff/1/
> 
> 
> Testing
> ---
> 
> start atlas and make sure the types are as expected using rest calls.
> 
> 
> Thanks,
> 
> David Radley
> 
>

[jira] [Created] (ATLAS-2343) [DSL] Where keyword has become mandatory

[Apoorv Naik (JIRA)]

Apoorv Naik created ATLAS-2343:
--

 Summary: [DSL] Where keyword has become mandatory 
 Key: ATLAS-2343
 URL: https://issues.apache.org/jira/browse/ATLAS-2343
 Project: Atlas
  Issue Type: Bug
Reporter: Apoorv Naik


eg. DB name=a fails






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64982: ATLAS2341 amend the Area 1 models so the relationship names are unique

[Mandy Chessell]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64982/#review194882
---



The relationship AttachedFeedback should be between a Referenceable and a 
Comment.  There should be no relationships to Project in model 0150.

- Mandy Chessell


On Jan. 5, 2018, 5:43 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64982/
> ---
> 
> (Updated Jan. 5, 2018, 5:43 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Mandy Chessell.
> 
> 
> Bugs: 2341
> https://issues.apache.org/jira/browse/2341
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> While working with the model files - I found that some of the relationship 
> end names are not unique. I think it is simpler to have these names unique. 
> This Jira is also used to remove some duplicated relationshipDef 
> "ProjectScope" and to add an empty attributeDef array for entityDef 
> "PrivateTag"
> 
> 
> Diffs
> -
> 
>   addons/models/0100-Area1-Collaboration/0135-Meetings.json 59998be4 
>   addons/models/0100-Area1-Collaboration/0140-Communities.json e6d7b868 
>   addons/models/0100-Area1-Collaboration/0150-Feedback.json 0115c64d 
> 
> 
> Diff: https://reviews.apache.org/r/64982/diff/1/
> 
> 
> Testing
> ---
> 
> start atlas and make sure the types are as expected using rest calls.
> 
> 
> Thanks,
> 
> David Radley
> 
>

[jira] [Updated] (ATLAS-2341) Amend Atlas Area 1 model files so names are unique.

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-2341:

Attachment: ATLAS2341.patch

> Amend Atlas Area 1 model files so names are unique.
> ---
>
> Key: ATLAS-2341
> URL: https://issues.apache.org/jira/browse/ATLAS-2341
> Project: Atlas
>  Issue Type: Bug
>Reporter: David Radley
>Assignee: David Radley
> Attachments: ATLAS2341.patch
>
>
> While working with the model files - I found that some of the relationship 
> end names are not unique. I think it is simpler to have these names unique. 
> This Jira is also used to remove some duplicated relationshipDef 
> "ProjectScope" and to add an empty attributeDef array for entityDef 
> "PrivateTag"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 64982: ATLAS2341 amend the Area 1 models so the relationship names are unique

[David Radley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64982/
---

Review request for atlas, Madhan Neethiraj and Mandy Chessell.


Bugs: 2341
https://issues.apache.org/jira/browse/2341


Repository: atlas


Description
---

While working with the model files - I found that some of the relationship end 
names are not unique. I think it is simpler to have these names unique. This 
Jira is also used to remove some duplicated relationshipDef "ProjectScope" and 
to add an empty attributeDef array for entityDef "PrivateTag"


Diffs
-

  addons/models/0100-Area1-Collaboration/0135-Meetings.json 59998be4 
  addons/models/0100-Area1-Collaboration/0140-Communities.json e6d7b868 
  addons/models/0100-Area1-Collaboration/0150-Feedback.json 0115c64d 


Diff: https://reviews.apache.org/r/64982/diff/1/


Testing
---

start atlas and make sure the types are as expected using rest calls.


Thanks,

David Radley

[jira] [Created] (ATLAS-2342) [DSL] Limit is not honored when used with groupBy

[Apoorv Naik (JIRA)]

Apoorv Naik created ATLAS-2342:
--

 Summary: [DSL] Limit is not honored when used with groupBy
 Key: ATLAS-2342
 URL: https://issues.apache.org/jira/browse/ATLAS-2342
 Project: Atlas
  Issue Type: Bug
Reporter: Apoorv Naik
Assignee: Apoorv Naik






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2341) Amend Atlas Area 1 model files so names are unique.

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-2341:

Description: While working with the model files - I found that some of the 
relationship end names are not unique. I think it is simpler to have these 
names unique. This Jira is also used to remove some duplicated relationshipDef 
"ProjectScope" and to add an empty attributeDef array for entityDef "PrivateTag"

> Amend Atlas Area 1 model files so names are unique.
> ---
>
> Key: ATLAS-2341
> URL: https://issues.apache.org/jira/browse/ATLAS-2341
> Project: Atlas
>  Issue Type: Bug
>Reporter: David Radley
>Assignee: David Radley
>
> While working with the model files - I found that some of the relationship 
> end names are not unique. I think it is simpler to have these names unique. 
> This Jira is also used to remove some duplicated relationshipDef 
> "ProjectScope" and to add an empty attributeDef array for entityDef 
> "PrivateTag"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2341) Amend Atlas Area 1 model files so names are unique.

[David Radley (JIRA)]

David Radley created ATLAS-2341:
---

 Summary: Amend Atlas Area 1 model files so names are unique.
 Key: ATLAS-2341
 URL: https://issues.apache.org/jira/browse/ATLAS-2341
 Project: Atlas
  Issue Type: Bug
Reporter: David Radley
Assignee: David Radley






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (ATLAS-2241) Import-hive.sh script to have option to import selected databases or tables

[Nixon Rodrigues (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues reassigned ATLAS-2241:
--

Assignee: Nixon Rodrigues

> Import-hive.sh script to have option to import selected databases or tables
> ---
>
> Key: ATLAS-2241
> URL: https://issues.apache.org/jira/browse/ATLAS-2241
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>
> Import-hive.sh script try to import all databases and tables from hive. But 
> sometime we just need to import few databases or table so is there any way to 
> do this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2340) java.lang.IncompatibleClassChangeError while running import-hive.sh script

[Nixon Rodrigues (JIRA)]

Nixon Rodrigues created ATLAS-2340:
--

 Summary: java.lang.IncompatibleClassChangeError while running 
import-hive.sh script
 Key: ATLAS-2340
 URL: https://issues.apache.org/jira/browse/ATLAS-2340
 Project: Atlas
  Issue Type: Bug
Reporter: Nixon Rodrigues


{noformat}
[root@ctr-e137-1514896590304-3990-01-02 hook-bin]# ./import-hive.sh
Using Hive configuration directory [/etc/hive/conf]
Log file for import is /usr/hdp/current/atlas-server/logs/import-hive.log
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/grid/0/hdp/3.0.0.0-669/hive/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/grid/0/hdp/3.0.0.0-669/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2018-01-05T12:52:03,485 INFO [main] org.apache.atlas.ApplicationProperties - 
Looking for atlas-application.properties in classpath
2018-01-05T12:52:03,492 INFO [main] org.apache.atlas.ApplicationProperties - 
Loading atlas-application.properties from 
file:/etc/hive/3.0.0.0-669/0/atlas-application.properties
2018-01-05T12:52:03,534 INFO [main] org.apache.atlas.ApplicationProperties - No 
graphdb backend specified. Will use 'janus'
2018-01-05T12:52:03,534 INFO [main] org.apache.atlas.ApplicationProperties - 
Using storage backend 'hbase2'
2018-01-05T12:52:03,534 INFO [main] org.apache.atlas.ApplicationProperties - 
Using index backend 'solr'
2018-01-05T12:52:03,534 INFO [main] org.apache.atlas.ApplicationProperties - 
Setting solr-wait-searcher property 'true'
Enter username for atlas :- admin
Enter password for atlas :-
Exception in thread "main" java.lang.IncompatibleClassChangeError: 
com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider and 
com.sun.jersey.json.impl.provider.entity.JSONRootElementProvider$Wadl disagree 
on InnerClasses attribute
at java.lang.Class.getDeclaringClass0(Native Method)
at java.lang.Class.getDeclaringClass(Class.java:1235)
at java.lang.Class.getEnclosingClass(Class.java:1277)
at 
com.sun.jersey.core.spi.component.ComponentConstructor.getInstance(ComponentConstructor.java:170)
at 
com.sun.jersey.core.spi.component.ProviderFactory.__getComponentProvider(ProviderFactory.java:166)
at 
com.sun.jersey.core.spi.component.ProviderFactory.getComponentProvider(ProviderFactory.java:137)
at 
com.sun.jersey.core.spi.component.ProviderServices.getComponent(ProviderServices.java:283)
at 
com.sun.jersey.core.spi.component.ProviderServices.getServices(ProviderServices.java:163)
at 
com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:176)
at 
com.sun.jersey.core.spi.factory.MessageBodyFactory.init(MessageBodyFactory.java:162)
at com.sun.jersey.api.client.Client.init(Client.java:342)
at com.sun.jersey.api.client.Client.access$000(Client.java:118)
at com.sun.jersey.api.client.Client$1.f(Client.java:191)
at com.sun.jersey.api.client.Client$1.f(Client.java:187)
at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
at com.sun.jersey.api.client.Client.(Client.java:187)
at com.sun.jersey.api.client.Client.(Client.java:170)
at org.apache.atlas.AtlasBaseClient.getClient(AtlasBaseClient.java:275)
at 
org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:415)
at 
org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:410)
at org.apache.atlas.AtlasBaseClient.(AtlasBaseClient.java:113)
at org.apache.atlas.AtlasClient.(AtlasClient.java:136)
at 
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:642)
Failed to import Hive Data Model!!!

{noformat}





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2241) Import-hive.sh script to have option to import selected databases or tables

[Nixon Rodrigues (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-2241:
---
Affects Version/s: trunk

> Import-hive.sh script to have option to import selected databases or tables
> ---
>
> Key: ATLAS-2241
> URL: https://issues.apache.org/jira/browse/ATLAS-2241
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Nixon Rodrigues
>
> Import-hive.sh script try to import all databases and tables from hive. But 
> sometime we just need to import few databases or table so is there any way to 
> do this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 64975: ATLAS-2339 : UI : Modifications in "columns" in Basic search result view affects DSL also.

[pratik pandey]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64975/
---

Review request for atlas, keval bhatt, Madhan Neethiraj, Nixon Rodrigues, and 
Sharmadha Sainath.


Bugs: ATLAS-2339
https://issues.apache.org/jira/browse/ATLAS-2339


Repository: atlas


Description
---

1.Fire a basic query , ex : type = hive_table .
2.Switch to DSL , fire the same type=hive_table.
3.Switch back to basic. Deselect "Type" from columns.
4.Switch to DSL. DSL query is fired again. This time "Type" column is not found 
in UI of DSL search results.


Diffs
-

  dashboardv2/public/js/views/search/SearchResultLayoutView.js 81816f3 


Diff: https://reviews.apache.org/r/64975/diff/1/


Testing
---

Done one round of sanity testing.


Thanks,

pratik pandey

[jira] [Updated] (ATLAS-2339) UI : Modifications in "columns" in Basic search result view affects DSL also.

[pratik pandey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

pratik pandey updated ATLAS-2339:
-
Attachment: ATLAS-2339.patch

> UI : Modifications in "columns" in Basic search result view affects DSL also.
> -
>
> Key: ATLAS-2339
> URL: https://issues.apache.org/jira/browse/ATLAS-2339
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: trunk
>Reporter: pratik pandey
>Assignee: pratik pandey
> Fix For: 1.0.0, 0.8.2
>
> Attachments: ATLAS-2339.patch
>
>
> 1.Fire a basic query , ex : type = hive_table .
> 2.Switch to DSL , fire the same type=hive_table.
> 3.Switch back to basic. Deselect "Type" from columns.
> 4.Switch to DSL. DSL query is fired again. This time "Type" column is not 
> found in UI of DSL search results.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2339) UI : Modifications in "columns" in Basic search result view affects DSL also.

[pratik pandey (JIRA)]

pratik pandey created ATLAS-2339:


 Summary: UI : Modifications in "columns" in Basic search result 
view affects DSL also.
 Key: ATLAS-2339
 URL: https://issues.apache.org/jira/browse/ATLAS-2339
 Project: Atlas
  Issue Type: Bug
  Components: atlas-webui
Affects Versions: trunk
Reporter: pratik pandey
Assignee: pratik pandey
 Fix For: 1.0.0, 0.8.2


1.Fire a basic query , ex : type = hive_table .
2.Switch to DSL , fire the same type=hive_table.
3.Switch back to basic. Deselect "Type" from columns.
4.Switch to DSL. DSL query is fired again. This time "Type" column is not found 
in UI of DSL search results.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64943: ATLAS-2332: support for attributes having nested collection datatype

[Graham Wallis]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64943/#review194849
---



Regarding serialization with JanusGraph:

I think JanusGraph's StandardSerializer has support for Lists - in the 
xxxArraySerializer classes (registered as 22 thru 30). But I don't see anything 
in StandardSerializer that handles Mapswhich seems to contradict your email 
so is slightly confusing. Presumably we (Atlas) work our way through the Map 
delegating serialization of each entry to the appopriate serializer?

I don't think there is anything (as standard) in JanusGraph's 
StandardSerializer that would help with BigDecimal, BigInteger or 
TypeCategorySerializer - but for any Atlas specific classes we could register 
Custom Serializers with JanusGraph - i.e. located above offset 100. 
TypeCategory is only a skinny layer over EnumSerializer, so not sure it is 
worth the effort? I think we could just keep our existing Atlas 
BigDecimalSerializer, BigIntegerSerializer and TypeCategorySerializer.



In terms of Atlas classes/methods:

Why do we have both GraphHelper:getMapValueProperty() and 
EntityGraphMapper:getMapValueProperty()? Should one delegate to the other?
Also, for anything that is not a Reference, one of the above methods explicitly 
handles List and Map property values and returns List.class and Map.class 
objects, whereas the other returns an Object.class. Should this be consistent 
across both methods?

- Graham Wallis


On Jan. 4, 2018, 3:24 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64943/
> ---
> 
> (Updated Jan. 4, 2018, 3:24 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2332
> https://issues.apache.org/jira/browse/ATLAS-2332
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Updated type-def and instance modules to enable attributes with nested 
> collection datatypes, like map>, 
> array>
> 
> 
> Diffs
> -
> 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java
>  e507a8ad 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/serializer/StringListSerializer.java
>  fa6f5fda 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java bbccf77c 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 3e602431 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  f6a15b69 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  706e7373 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
>  e1047220 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  1f8b9fdb 
> 
> 
> Diff: https://reviews.apache.org/r/64943/diff/1/
> 
> 
> Testing
> ---
> 
> added unit tests to validate handling of attributes with nested collection 
> datatypes
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

[jira] [Comment Edited] (ATLAS-2298) OCF JDBC Connector for GaianDB

[Yao Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312804#comment-16312804
 ] 

Yao Li edited comment on ATLAS-2298 at 1/5/18 9:57 AM:
---

1) I would find it useful to understand the use cases for this connector.I 
assume the 2 main use cases are to query and to define new Logical tables (the 
new views), is adding GaianDB data sources in scope? 

This JDBC Connector is only used by now for querying data from GaianDB. It is a 
sub class of OCF connector. I will add a use case diagram to the document.

2) It would be useful to see how the DB_ parameters map to the actual JDBC url 
that will be used.

I will later upload the patch for review.

3) I am wondering how the security works
   - are you using impersonation - if so how does this work?
   - if we are using certificates - does this connector need to know 
anything about them.  

Nigel used the impersonation solution. The detailed information can be found in 
Ranger-1850 (https://issues.apache.org/jira/browse/RANGER-1850)

4) It would be useful to understand and document what is assumed to be running 
and setup for the connector to work. 
 - I assume we need a running GaianDB instance.  
 - I assume the OCF needs to have the GaianDB connector defined. 

When we use the GaianDB JDBC Connector, we need the running GaianDB instance 
and the GaianDBJDBCConnector(the name will be discussed) is a sub class of OCF. 
It has all the functions of OCF but for data query.

Thank you very much for your comments. [~davidrad]



was (Author: yao 22):
1) I would find it useful to understand the use cases for this connector.I 
assume the 2 main use cases are to query and to define new Logical tables (the 
new views), is adding GaianDB data sources in scope? 

This JDBC Connector is only used by now for querying data from GaianDB. It is a 
sub class of OCF connector. I will add a use case diagram to the document.

2) It would be useful to see how the DB_ parameters map to the actual JDBC url 
that will be used.

I will later upload the patch for review. [~davidrad]

3) I am wondering how the security works
   - are you using impersonation - if so how does this work?
   - if we are using certificates - does this connector need to know 
anything about them.  

Nigel used the impersonation solution. The detailed information can be found in 
Ranger-1850 (https://issues.apache.org/jira/browse/RANGER-1850)

4) It would be useful to understand and document what is assumed to be running 
and setup for the connector to work. 
 - I assume we need a running GaianDB instance.  
 - I assume the OCF needs to have the GaianDB connector defined. 

When we use the GaianDB JDBC Connector, we need the running GaianDB instance 
and the GaianDBJDBCConnector(the name will be discussed) is a sub class of OCF. 
It has all the functions of OCF but for data query.

Thank you very much for your comments. [#Dav]


> OCF JDBC Connector for GaianDB
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
> Attachments: JDBC Connector Java Doc.zip, OCF JDBC Connector.pdf
>
>
> This Jira is focused on development of the OCF JDBC Connector for the GaianDB 
> which will be used to access data from GaianDB.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (ATLAS-2298) OCF JDBC Connector for GaianDB

[Yao Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312788#comment-16312788
 ] 

Yao Li edited comment on ATLAS-2298 at 1/5/18 9:57 AM:
---

1) I notice in the javadoc that JDBCConnectorErrorCode contains an extra /**

Fixed this. Thank you for pointing it out.

2) I notice public class ConnectionCheckedException extends 
JDBCConnectorCheckedExceptionBase. From the naming,  it is not obvious that 
ConnectionCheckedException has anything to do with JDBC from the name. 
3) The same for ExecutionCheckedException. 
4) to ease readability in 2 and 3 I suggest removing the checked from the names 
- as the runtime already indicated those exceptions are unchecked. 
 - renaming JDBCConnectorCheckedExceptionBase to JDBCConnectorException
 - renaming ConnectionCheckedException to ConnectionJDBCConnectorException
 - renaming ExecutionCheckedException to ExecutionJDBCConnectorException
5) In JDBCConnectorRuntimeException getReportedCaughtException() returns a 
Throwable - should this method be getReportedThrowable
6)  In JDBCConnectorRuntimeException getReportedSystemAction() - is this system 
action reported? I suggest removing the "reported" from the name. Same for 
"getReportedUserAction()" and "getReportedHTTPCode()"

>From 2) to 6) these codes I want to keep consistent with Mandy’s OCF and OMRS 
>Connector’s code. After she has the new patch available I will check out the 
>code and update the code, if the name are still the same, we can discuss these 
>again.

7) Maybe rename GaianDBJDBCConnector to GaianJDBCConnector for readability

I will set it as a small discuss during the workshop

8)  For setDBUrlCreateFalse(java.lang.String realUserId) set up the proper 
database url, create is false. 
 - I suggest we pass a second boolean flag parameter rather than have 2 
methods.
 - what does create true and false mean?
 - what is the proper database?
 - what is the realUserId?
 - what does this mean "realUserId - used by the tempory GaianDB db url for 
authentication". What is temporary about this and what is real about it?

-   Changed it to a method with a Boolean flag parameter
-   Create true and false is used in the database url.
When you use “create=true” to connect the database:
•   If the database cannot be created, the error appears in the error log 
and the connection attempt fails with an SQLException indicating that the 
database cannot be found.
•   If the database already exists, creates a connection to the existing 
database and an SQLWarning is issued.
•   JDBC does not remove the database on failure to connect at create time 
if failure occurs after the database call occurs. If a database connection URL 
used create=true and the connection fails to be created, check for the database 
directory. If it exists, remove it and its contents before the next attempt to 
create the database.
-   Proper here only means that you offer the correct connection 
information to form a url. Already change this to “set up the database url”
-   Here we use NPA to connect to GaianDB, Nigle has developed the code to 
use impersonation, and here the “realUserId” is used to differentiate from 
“proxy-user”. Already changed it to “userId”
-   It is temporary method to deal with the impersonation. Detailed 
information is listed in 
Ranger-1850(https://issues.apache.org/jira/browse/RANGER-1850)


9) for ConnectionCheckedException - it says "there may be the odd bug that 
surfaces here." I suggest removing this. When should we use  
"JDBCConnectorErrorCode" and when should be use " messages defined uniquely for 
a ConnectorProvider/Connector implementation"

Fixed it and change to :
ConnectionCheckedException provides a checked exception for reporting errors 
found when using JDBC connectors to connect with database.
Typically these errors are either configuration or operational errors that can 
be fixed by an administrator
or power user. The JDBCConnectorErrorCode can be used within this exception to 
populate it with standard messages. Otherwise messages
defined uniquely for JDBC Connector implementation can be used(such as 
exceptions in database Connection).
The aim is to be able to uniquely identify the cause
and remedy for the error.

Thank you very much for the comments [~davidrad]


was (Author: yao 22):
1) I notice in the javadoc that JDBCConnectorErrorCode contains an extra /**

Fixed this. Thank you for pointing it out.

2) I notice public class ConnectionCheckedException extends 
JDBCConnectorCheckedExceptionBase. From the naming,  it is not obvious that 
ConnectionCheckedException has anything to do with JDBC from the name. 
3) The same for ExecutionCheckedException. 
4) to ease readability in 2 and 3 I suggest removing the checked from the names 
- as the runtime already indicated those exceptions a

[jira] [Commented] (ATLAS-2298) OCF JDBC Connector for GaianDB

[Yao Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312804#comment-16312804
 ] 

Yao Li commented on ATLAS-2298:
---

1) I would find it useful to understand the use cases for this connector.I 
assume the 2 main use cases are to query and to define new Logical tables (the 
new views), is adding GaianDB data sources in scope? 

This JDBC Connector is only used by now for querying data from GaianDB. It is a 
sub class of OCF connector. I will add a use case diagram to the document.

2) It would be useful to see how the DB_ parameters map to the actual JDBC url 
that will be used.

I will later upload the patch for review. [~davidrad]

3) I am wondering how the security works
   - are you using impersonation - if so how does this work?
   - if we are using certificates - does this connector need to know 
anything about them.  

Nigel used the impersonation solution. The detailed information can be found in 
Ranger-1850 (https://issues.apache.org/jira/browse/RANGER-1850)

4) It would be useful to understand and document what is assumed to be running 
and setup for the connector to work. 
 - I assume we need a running GaianDB instance.  
 - I assume the OCF needs to have the GaianDB connector defined. 

When we use the GaianDB JDBC Connector, we need the running GaianDB instance 
and the GaianDBJDBCConnector(the name will be discussed) is a sub class of OCF. 
It has all the functions of OCF but for data query.

Thank you very much for your comments. [#Dav]


> OCF JDBC Connector for GaianDB
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
> Attachments: JDBC Connector Java Doc.zip, OCF JDBC Connector.pdf
>
>
> This Jira is focused on development of the OCF JDBC Connector for the GaianDB 
> which will be used to access data from GaianDB.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ATLAS-2298) OCF JDBC Connector for GaianDB

[Yao Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312788#comment-16312788
 ] 

Yao Li commented on ATLAS-2298:
---

1) I notice in the javadoc that JDBCConnectorErrorCode contains an extra /**

Fixed this. Thank you for pointing it out.

2) I notice public class ConnectionCheckedException extends 
JDBCConnectorCheckedExceptionBase. From the naming,  it is not obvious that 
ConnectionCheckedException has anything to do with JDBC from the name. 
3) The same for ExecutionCheckedException. 
4) to ease readability in 2 and 3 I suggest removing the checked from the names 
- as the runtime already indicated those exceptions are unchecked. 
 - renaming JDBCConnectorCheckedExceptionBase to JDBCConnectorException
 - renaming ConnectionCheckedException to ConnectionJDBCConnectorException
 - renaming ExecutionCheckedException to ExecutionJDBCConnectorException
5) In JDBCConnectorRuntimeException getReportedCaughtException() returns a 
Throwable - should this method be getReportedThrowable
6)  In JDBCConnectorRuntimeException getReportedSystemAction() - is this system 
action reported? I suggest removing the "reported" from the name. Same for 
"getReportedUserAction()" and "getReportedHTTPCode()"

>From 2) to 6) these codes I want to keep consistent with Mandy’s OCF and OMRS 
>Connector’s code. After she has the new patch available I will check out the 
>code and update the code, if the name are still the same, we can discuss these 
>again.

7) Maybe rename GaianDBJDBCConnector to GaianJDBCConnector for readability

I will set it as a small discuss during the workshop

8)  For setDBUrlCreateFalse(java.lang.String realUserId) set up the proper 
database url, create is false. 
 - I suggest we pass a second boolean flag parameter rather than have 2 
methods.
 - what does create true and false mean?
 - what is the proper database?
 - what is the realUserId?
 - what does this mean "realUserId - used by the tempory GaianDB db url for 
authentication". What is temporary about this and what is real about it?

-   Changed it to a method with a Boolean flag parameter
-   Create true and false is used in the database url.
When you use “create=true” to connect the database:
•   If the database cannot be created, the error appears in the error log 
and the connection attempt fails with an SQLException indicating that the 
database cannot be found.
•   If the database already exists, creates a connection to the existing 
database and an SQLWarning is issued.
•   JDBC does not remove the database on failure to connect at create time 
if failure occurs after the database call occurs. If a database connection URL 
used create=true and the connection fails to be created, check for the database 
directory. If it exists, remove it and its contents before the next attempt to 
create the database.
-   Proper here only means that you offer the correct connection 
information to form a url. Already change this to “set up the database url”
-   Here we use NPA to connect to GaianDB, Nigle has developed the code to 
use impersonation, and here the “realUserId” is used to differentiate from 
“proxy-user”. Already changed it to “userId”
-   It is temporary method to deal with the impersonation. Detailed 
information is listed in 
Ranger-1850(https://issues.apache.org/jira/browse/RANGER-1850)


9) for ConnectionCheckedException - it says "there may be the odd bug that 
surfaces here." I suggest removing this. When should we use  
"JDBCConnectorErrorCode" and when should be use " messages defined uniquely for 
a ConnectorProvider/Connector implementation"

Fixed it and change to :
ConnectionCheckedException provides a checked exception for reporting errors 
found when using JDBC connectors to connect with database.
Typically these errors are either configuration or operational errors that can 
be fixed by an administrator
or power user. The JDBCConnectorErrorCode can be used within this exception to 
populate it with standard messages. Otherwise messages
defined uniquely for JDBC Connector implementation can be used(such as 
exceptions in database Connection).
The aim is to be able to uniquely identify the cause
and remedy for the error.


> OCF JDBC Connector for GaianDB
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
> Attachments: JDBC Connector Java Doc.zip, OCF JDBC Connector.pdf
>
>
> This Jira is focused on development of the OCF JDBC Connector for the GaianDB 
> which will be used to access data from GaianDB.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)