date:20200928

Re: Review Request 72912: ATLAS-3962 : Include business metadata def header in typdefs headers API

2020-09-28 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72912/#review221972
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 29, 2020, 5:03 a.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72912/
> ---
> 
> (Updated Sept. 29, 2020, 5:03 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3962
> https://issues.apache.org/jira/browse/ATLAS-3962
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Bug details:- While trying to fetch headers of all types, data about business 
> metadata is not fetched via "atlas/v2/types/typedefs/headers" .We could 
> verify is there is business metadata in 
> "api/atlas/v2/types/typedefs?type=business_metadata" though
> 
> The patch provided fix to add business metadata def header in typdefs headers.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 9abcd3449 
> 
> 
> Diff: https://reviews.apache.org/r/72912/diff/1/
> 
> 
> Testing
> ---
> 
> Tested output of atlas/v2/types/typedefs/headers and 
> api/atlas/v2/types/typedefs?type=business_metadata API
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

[jira] [Created] (ATLAS-3963) Atlas api 'atlas/v2/types/typedefs/headers' does not give information about business metadata

2020-09-28 Thread Dharshana M Krishnamoorthy (Jira)

Dharshana M Krishnamoorthy created ATLAS-3963:
-

 Summary: Atlas api 'atlas/v2/types/typedefs/headers' does not give 
information about business metadata
 Key: ATLAS-3963
 URL: https://issues.apache.org/jira/browse/ATLAS-3963
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Reporter: Dharshana M Krishnamoorthy
Assignee: Nixon Rodrigues


While trying to fetch headers of all types, data about business metadata is not 
fetched via "*atlas/v2/types/typedefs/headers*"

We could verify is there is business metadata in 
"api/atlas/v2/types/typedefs?type=business_metadata" though

 
This holds true for 
'[api/atlas/v2/types/typedefs/headers?type=business_metadata'|https://quasar-oczlgo-1.quasar-oczlgo.root.hwx.site:31443/api/atlas/v2/types/typedefs/headers?type=business_metadata%27]
 as well
 * [|https://jira.cloudera.com/secure/AddComment!default.jspa?id=917333]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 72912: ATLAS-3962 : Include business metadata def header in typdefs headers API

2020-09-28 Thread Nixon Rodrigues


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72912/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, and 
Sarath Subramanian.


Bugs: ATLAS-3962
https://issues.apache.org/jira/browse/ATLAS-3962


Repository: atlas


Description
---

Bug details:- While trying to fetch headers of all types, data about business 
metadata is not fetched via "atlas/v2/types/typedefs/headers" .We could verify 
is there is business metadata in 
"api/atlas/v2/types/typedefs?type=business_metadata" though

The patch provided fix to add business metadata def header in typdefs headers.


Diffs
-

  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 9abcd3449 


Diff: https://reviews.apache.org/r/72912/diff/1/


Testing
---


Thanks,

Nixon Rodrigues

[jira] [Created] (ATLAS-3962) Include business metadata def header in typdefs headers API

2020-09-28 Thread Nixon Rodrigues (Jira)

Nixon Rodrigues created ATLAS-3962:
--

 Summary: Include business metadata def header in typdefs headers 
API
 Key: ATLAS-3962
 URL: https://issues.apache.org/jira/browse/ATLAS-3962
 Project: Atlas
  Issue Type: Bug
Reporter: Nixon Rodrigues
Assignee: Nixon Rodrigues


While trying to fetch headers of all types, data about business metadata is not 
fetched via "*atlas/v2/types/typedefs/headers*"

We could verify is there is business metadata in 
"api/atlas/v2/types/typedefs?type=business_metadata" though



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

2020-09-28 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72895/#review221970
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityDefStoreV2.java
Lines 95 (patched)


- Attribute of 
struct-def/classification-def/relationship-def/business-metadata-def also need 
to be checked similarly. It will be useful to add another 
verifyTypeReadAccess() method, with Collection(AttributeDef attributeDefs), and 
call from relevant places
- in addition, please review handling the same in update() call as well.


- Madhan Neethiraj


On Sept. 28, 2020, 6:04 p.m., chaitali wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72895/
> ---
> 
> (Updated Sept. 28, 2020, 6:04 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3950
> https://issues.apache.org/jira/browse/ATLAS-3950
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> User is able to create an attribute for 
> businessmetadata/classification/entitydef of typename which doesn't have read 
> access in the ranger policy. UI works fine but but through REST , User is 
> able to add any type as attribute typename.
> 
> Steps to reproduce:
> 
> User has following rights in ranger policy :
> 
> CRUD permissions on hive_table type only
> 
> CRUD permissions on all business_metadata type.
> 
> hrt_qa creates a business metadata bm1
> 
> hrt_qa is able to create an attribute for bm1 - say attrib1 which allows 
> Applicable types to be anything. UI displays only hive_table but through REST 
> , hrt_qa is able to add any type as Applicable type.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
>  2cb2b4789 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasBusinessMetadataDefStoreV2.java
>  6b4fa65ae 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasClassificationDefStoreV2.java
>  9ffede4e3 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityDefStoreV2.java
>  e5153de0e 
> 
> 
> Diff: https://reviews.apache.org/r/72895/diff/10/
> 
> 
> Testing
> ---
> 
> Tested with following JSON : 
> 
> Entity typedef creation with an attribute of typename which admin doesn't 
> have type-read rights for:
> {
> "enumDefs": [],
> "structDefs": [],
> "classificationDefs": [],
> "entityDefs": [
>   {
> "category": "ENTITY",
> "createdBy": "admin",
> "updatedBy": "admin",
> "createTime": 1537261952180,
> "updateTime": 1537262097732,
> "version": 3,
> "name": "testtab",
> "description": "sport",
> "typeVersion": "1.4",
> "attributeDefs": [
>   {
> "name": "test_name",
> "typeName": "string",
> "isOptional": false,
> "cardinality": "SINGLE",
> "valuesMinCount": 1,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": false,
> "includeInNotification": false
> },
> {
> "name": "year",
> "typeName": "int",
> "isOptional": false,
> "cardinality": "SINGLE",
> "valuesMinCount": 1,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": false,
> "includeInNotification": false
> },
> {
> "name": "place",
> "typeName": "array",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": false,
> "includeInNotification": false
> }
> ],
> "superTypes": [
>   "DataSet"
> ],
> "subTypes": []
> }
> ],
> "relationshipDefs": []
> }
> 
> Classification typedef creation with entitytypes for which doesn't have 
> type-read rights for:
> 
> {
>   "classificationDefs": [
> {
>   "name": "testclass",
>   "description": "oif",
>   "superTypes": [],
>   "attributeDefs": [],
>   "entityTypes": [
> "hdfs_path",
> "hive_table"
>   ],
>   "category": "CLASSIFICATION",
>   "guid": "-123456789"
> }
>   ],
>   "entityDefs": [],
>   "enumDefs": [],
>   "structDefs": []
> }
> 
> Businessmetadata typedef creation with entitytypes for which admin doesn't 
> have type-read rights for:
> 
> {
>   "enumDefs": [],
>   "structDefs": [],
>   "classificationDefs": [],
>   "entityDefs": [],
>   "businessMetadataDefs": [
> {
>   "category": "BUSINESS_METADATA",
>   "createdBy": "admin",
>   "updatedBy": "admin",
>   "version": 1,
>   "typeVersion": "1.1",
>   "name": "testBM",
>   "description": "",
>   "attributeDefs": [
> {
>   "name": "attrtestBM",
>   "typeName": "string",
>   "isOptional": true,
>   "cardinality": "SINGLE",
>   "values

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

2020-09-28 Thread chaitali


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72895/
---

(Updated Sept. 28, 2020, 6:04 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3950
https://issues.apache.org/jira/browse/ATLAS-3950


Repository: atlas


Description
---

User is able to create an attribute for 
businessmetadata/classification/entitydef of typename which doesn't have read 
access in the ranger policy. UI works fine but but through REST , User is able 
to add any type as attribute typename.

Steps to reproduce:

User has following rights in ranger policy :

CRUD permissions on hive_table type only

CRUD permissions on all business_metadata type.

hrt_qa creates a business metadata bm1

hrt_qa is able to create an attribute for bm1 - say attrib1 which allows 
Applicable types to be anything. UI displays only hive_table but through REST , 
hrt_qa is able to add any type as Applicable type.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
 2cb2b4789 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasBusinessMetadataDefStoreV2.java
 6b4fa65ae 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasClassificationDefStoreV2.java
 9ffede4e3 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityDefStoreV2.java
 e5153de0e 


Diff: https://reviews.apache.org/r/72895/diff/10/

Changes: https://reviews.apache.org/r/72895/diff/9-10/


Testing
---

Tested with following JSON : 

Entity typedef creation with an attribute of typename which admin doesn't have 
type-read rights for:
{
"enumDefs": [],
"structDefs": [],
"classificationDefs": [],
"entityDefs": [
  {
"category": "ENTITY",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1537261952180,
"updateTime": 1537262097732,
"version": 3,
"name": "testtab",
"description": "sport",
"typeVersion": "1.4",
"attributeDefs": [
  {
"name": "test_name",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
},
{
"name": "year",
"typeName": "int",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
},
{
"name": "place",
"typeName": "array",
"isOptional": true,
"cardinality": "SINGLE",
"valuesMinCount": 0,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
}
],
"superTypes": [
  "DataSet"
],
"subTypes": []
}
],
"relationshipDefs": []
}

Classification typedef creation with entitytypes for which doesn't have 
type-read rights for:

{
  "classificationDefs": [
{
  "name": "testclass",
  "description": "oif",
  "superTypes": [],
  "attributeDefs": [],
  "entityTypes": [
"hdfs_path",
"hive_table"
  ],
  "category": "CLASSIFICATION",
  "guid": "-123456789"
}
  ],
  "entityDefs": [],
  "enumDefs": [],
  "structDefs": []
}

Businessmetadata typedef creation with entitytypes for which admin doesn't have 
type-read rights for:

{
  "enumDefs": [],
  "structDefs": [],
  "classificationDefs": [],
  "entityDefs": [],
  "businessMetadataDefs": [
{
  "category": "BUSINESS_METADATA",
  "createdBy": "admin",
  "updatedBy": "admin",
  "version": 1,
  "typeVersion": "1.1",
  "name": "testBM",
  "description": "",
  "attributeDefs": [
{
  "name": "attrtestBM",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": true,
  "options": {
"maxStrLength": "50",
"applicableEntityTypes": "[\"hive_table\"]"
  },
  "searchWeight": "5"
}
  ]
}
  ]
}

Expected Output :

{
"errorCode": "ATLAS-403-00-001",
"errorMessage": "admin is not authorized to perform read type hive_table"
}


Thanks,

chaitali

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

2020-09-28 Thread chaitali


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72895/
---

(Updated Sept. 28, 2020, 5:28 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3950
https://issues.apache.org/jira/browse/ATLAS-3950


Repository: atlas


Description
---

User is able to create an attribute for 
businessmetadata/classification/entitydef of typename which doesn't have read 
access in the ranger policy. UI works fine but but through REST , User is able 
to add any type as attribute typename.

Steps to reproduce:

User has following rights in ranger policy :

CRUD permissions on hive_table type only

CRUD permissions on all business_metadata type.

hrt_qa creates a business metadata bm1

hrt_qa is able to create an attribute for bm1 - say attrib1 which allows 
Applicable types to be anything. UI displays only hive_table but through REST , 
hrt_qa is able to add any type as Applicable type.


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasBusinessMetadataType.java 
f17300140 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
 2cb2b4789 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasBusinessMetadataDefStoreV2.java
 6b4fa65ae 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasClassificationDefStoreV2.java
 9ffede4e3 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityDefStoreV2.java
 e5153de0e 


Diff: https://reviews.apache.org/r/72895/diff/9/

Changes: https://reviews.apache.org/r/72895/diff/8-9/


Testing
---

Tested with following JSON : 

Entity typedef creation with an attribute of typename which admin doesn't have 
type-read rights for:
{
"enumDefs": [],
"structDefs": [],
"classificationDefs": [],
"entityDefs": [
  {
"category": "ENTITY",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1537261952180,
"updateTime": 1537262097732,
"version": 3,
"name": "testtab",
"description": "sport",
"typeVersion": "1.4",
"attributeDefs": [
  {
"name": "test_name",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
},
{
"name": "year",
"typeName": "int",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
},
{
"name": "place",
"typeName": "array",
"isOptional": true,
"cardinality": "SINGLE",
"valuesMinCount": 0,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
}
],
"superTypes": [
  "DataSet"
],
"subTypes": []
}
],
"relationshipDefs": []
}

Classification typedef creation with entitytypes for which doesn't have 
type-read rights for:

{
  "classificationDefs": [
{
  "name": "testclass",
  "description": "oif",
  "superTypes": [],
  "attributeDefs": [],
  "entityTypes": [
"hdfs_path",
"hive_table"
  ],
  "category": "CLASSIFICATION",
  "guid": "-123456789"
}
  ],
  "entityDefs": [],
  "enumDefs": [],
  "structDefs": []
}

Businessmetadata typedef creation with entitytypes for which admin doesn't have 
type-read rights for:

{
  "enumDefs": [],
  "structDefs": [],
  "classificationDefs": [],
  "entityDefs": [],
  "businessMetadataDefs": [
{
  "category": "BUSINESS_METADATA",
  "createdBy": "admin",
  "updatedBy": "admin",
  "version": 1,
  "typeVersion": "1.1",
  "name": "testBM",
  "description": "",
  "attributeDefs": [
{
  "name": "attrtestBM",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": true,
  "options": {
"maxStrLength": "50",
"applicableEntityTypes": "[\"hive_table\"]"
  },
  "searchWeight": "5"
}
  ]
}
  ]
}

Expected Output :

{
"errorCode": "ATLAS-403-00-001",
"errorMessage": "admin is not authorized to perform read type hive_table"
}


Thanks,

chaitali

[jira] [Updated] (ATLAS-3961) Atlas should encrypt LDAP/AD bind user password

2020-09-28 Thread Saurabh Lambe (Jira)



 [ 
https://issues.apache.org/jira/browse/ATLAS-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Saurabh Lambe updated ATLAS-3961:
-
Component/s: (was:  atlas-core)
 atlas-webui

> Atlas should encrypt LDAP/AD bind user password
> ---
>
> Key: ATLAS-3961
> URL: https://issues.apache.org/jira/browse/ATLAS-3961
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: 2.0.0
>Reporter: Saurabh Lambe
>Priority: Major
>  Labels: LDAP, password
>
> Atlas should encrypt the LDAP/AD bind user password stored in 
> atlas-application.properties file.
> By default the LDAP bind user password is shown in plaintext in the 
> above-said config file:
> {noformat}
> atlas.authentication.method.ldap.bind.password=hadoop{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3961) Atlas should encrypt LDAP/AD bind user password

2020-09-28 Thread Saurabh Lambe (Jira)

Saurabh Lambe created ATLAS-3961:


 Summary: Atlas should encrypt LDAP/AD bind user password
 Key: ATLAS-3961
 URL: https://issues.apache.org/jira/browse/ATLAS-3961
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 2.0.0
Reporter: Saurabh Lambe


Atlas should encrypt the LDAP/AD bind user password stored in 
atlas-application.properties file.

By default the LDAP bind user password is shown in plaintext in the above-said 
config file:
{noformat}
atlas.authentication.method.ldap.bind.password=hadoop{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

2020-09-28 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72895/#review221966
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
Lines 63 (patched)


#63: consider adding null check for 'types'
#65: consider adding null check for 'def'



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasClassificationDefStoreV2.java
Lines 102 (patched)


Instead of creating "Collection entityTypes" here, 
consider adding method:
  class AtlasAbstractDefStoreV2 {
public void verifyTypeReadAccess(Collection typeNames) {
  if (typeNames != null) {
for (String typeName : typeNames) {
  AtlasBaseTypeDef def = typeRegistry.getTypeDefByName(typeName);
  
  if (def != null) {
AtlasAuthorizationUtils.verifyAccess(new 
AtlasTypeAccessRequest(AtlasPrivilege.TYPE_READ, def), "read type-def of 
category ", def.getCategory(), " ", def.getName());
  }
}
  }
}

and replace #96, #97, #102 - #110 with a call to 
verifyTypeReadAccess(classificationDef.getEntityTypes());

Same applies for AtlasEntityDefStoreV2.create() as well.


- Madhan Neethiraj


On Sept. 28, 2020, 8:11 a.m., chaitali wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72895/
> ---
> 
> (Updated Sept. 28, 2020, 8:11 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3950
> https://issues.apache.org/jira/browse/ATLAS-3950
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> User is able to create an attribute for 
> businessmetadata/classification/entitydef of typename which doesn't have read 
> access in the ranger policy. UI works fine but but through REST , User is 
> able to add any type as attribute typename.
> 
> Steps to reproduce:
> 
> User has following rights in ranger policy :
> 
> CRUD permissions on hive_table type only
> 
> CRUD permissions on all business_metadata type.
> 
> hrt_qa creates a business metadata bm1
> 
> hrt_qa is able to create an attribute for bm1 - say attrib1 which allows 
> Applicable types to be anything. UI displays only hive_table but through REST 
> , hrt_qa is able to add any type as Applicable type.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
>  2cb2b4789 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasBusinessMetadataDefStoreV2.java
>  6b4fa65ae 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasClassificationDefStoreV2.java
>  9ffede4e3 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityDefStoreV2.java
>  e5153de0e 
> 
> 
> Diff: https://reviews.apache.org/r/72895/diff/8/
> 
> 
> Testing
> ---
> 
> Tested with following JSON : 
> 
> Entity typedef creation with an attribute of typename which admin doesn't 
> have type-read rights for:
> {
> "enumDefs": [],
> "structDefs": [],
> "classificationDefs": [],
> "entityDefs": [
>   {
> "category": "ENTITY",
> "createdBy": "admin",
> "updatedBy": "admin",
> "createTime": 1537261952180,
> "updateTime": 1537262097732,
> "version": 3,
> "name": "testtab",
> "description": "sport",
> "typeVersion": "1.4",
> "attributeDefs": [
>   {
> "name": "test_name",
> "typeName": "string",
> "isOptional": false,
> "cardinality": "SINGLE",
> "valuesMinCount": 1,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": false,
> "includeInNotification": false
> },
> {
> "name": "year",
> "typeName": "int",
> "isOptional": false,
> "cardinality": "SINGLE",
> "valuesMinCount": 1,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": false,
> "includeInNotification": false
> },
> {
> "name": "place",
> "typeName": "array",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": false,
> "includeInNotification": false
> }
> ],
> "superTypes": [
>   "DataSet"
> ],
> "subTypes": []
> }
> ],
> "relationshipDefs": []
> }
> 
> Classification typedef creation with entitytypes for which doesn't have 
> type-read rights for:
> 
> {
>   "classificationDefs": [
> {
>   "name": "testclass",
>   "description": "oif",
>   "superTypes": [],
>   "attributeDefs": [],
>   "entityTypes": [
> "hdfs_path",
> "hive_table"
>   ],
>   "category": "CLASSIFICATION",
>   "guid":

[jira] [Commented] (ATLAS-3959) Upgrade Atlas to Spring Framework version 4.3.16.release

2020-09-28 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/ATLAS-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203196#comment-17203196
 ] 

ASF subversion and git services commented on ATLAS-3959:


Commit 3a6c6bd085d71a4301c6184ad2478267de0f5a96 in atlas's branch 
refs/heads/branch-0.8 from chaitali borole
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=3a6c6bd ]

ATLAS-3959 : Upgrade Atlas to Spring Framework version 4.3.16.release

Signed-off-by: nixonrodrigues 


> Upgrade Atlas to Spring Framework version 4.3.16.release
> 
>
> Key: ATLAS-3959
> URL: https://issues.apache.org/jira/browse/ATLAS-3959
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.8.1
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 0.8.1
>
> Attachments: ATLAS-3959.patch
>
>
> Existing Spring version 4.3.8.release is vulnerable to CVE-2018-1270



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3960) Testcase failing [testGetDefinition(org.apache.atlas.web.integration.TypedefsJerseyResourceIT)]

2020-09-28 Thread Nixon Rodrigues (Jira)

Nixon Rodrigues created ATLAS-3960:
--

 Summary: Testcase failing 
[testGetDefinition(org.apache.atlas.web.integration.TypedefsJerseyResourceIT)] 
 Key: ATLAS-3960
 URL: https://issues.apache.org/jira/browse/ATLAS-3960
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.8.4
Reporter: Nixon Rodrigues


Testcase failing with mvn clean install

 
{noformat}
testGetDefinition(org.apache.atlas.web.integration.TypedefsJerseyResourceIT)  
Time elapsed: 0.004 sec  <<< FAILURE!
java.lang.AssertionError: Get byName should've succeeded
at 
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:420)
at 
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:350)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:254)
at 
org.apache.atlas.AtlasClientV2.getTypeDefByName(AtlasClientV2.java:428)
at 
org.apache.atlas.AtlasClientV2.getClassificationDefByName(AtlasClientV2.java:159)
at 
org.apache.atlas.web.integration.TypedefsJerseyResourceIT.verifyByNameAndGUID(TypedefsJerseyResourceIT.java:339)
at 
org.apache.atlas.web.integration.TypedefsJerseyResourceIT.testGetDefinition(TypedefsJerseyResourceIT.java:185)
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3608) Hive Bridge: Hive Metastore: Alter Table Query Not Handled Correctly

2020-09-28 Thread Palash Das (Jira)



[ 
https://issues.apache.org/jira/browse/ATLAS-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203131#comment-17203131
 ] 

Palash Das commented on ATLAS-3608:
---

This implementation is breaking a lot of things on presto. May be it works for 
impala. 
Presto fires an alter table event with exactly the same _transient_lastDdlTime_ 
 value. So checking on that is not helping. And unfortunately it is causing an 
NPE later since the hiveOperation is not getting set.

I'm using the following implementation to get it done. 
{code:java}
private boolean isAlterTableProperty(AlterTableEvent tableEvent) {
final MapDifference differences = Maps.difference(
tableEvent.getOldTable().getParameters(),
tableEvent.getNewTable().getParameters());

if (LOG.isDebugEnabled()) {
LOG.debug("HiveMetastoreHook.isAlterTableProperty({}): truely alter 
table event.", differences.toString());
}
return !differences.areEqual();
}

{code}

We found the new fields on new table (known as right, in guava MapDifference)
{noformat}
not equal: only on right={numRows=0, rawDataSize=0, 
STATS_GENERATED_VIA_STATS_TASK=workaround for potential lack of 
HIVE-12730}{noformat}

> Hive Bridge: Hive Metastore: Alter Table Query Not Handled Correctly
> 
>
> Key: ATLAS-3608
> URL: https://issues.apache.org/jira/browse/ATLAS-3608
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: 2.0.0, trunk
>
> Attachments: ATLAS-3608-Incorrect-processing-of-alter-table.patch
>
>
> *Background*
> DDL queries in Impala are processed via _Hive Metastore_ bridge.
>  
> *Steps to Duplicate*
> Keep HMS logs in view. Depending on installation, they can be found at this 
> location: _/var/log/hive/hadoop-cmf-HIVE-1-HIVEMETASTORE-.log.out_
> From Impala:
> - Run _impala-shell_
>  * Run _create database stocks; use stocks; create table daily (dt string, 
> open string, high string); create view daily_rpt as select * from daily; 
> create external table weekly (dt string, open string, high string);_
>  * Note within Atlas that the new entities for _stocks, daily, daily_rpt_ and 
> _weekly_ have been created. Note the columns in _weekly_ table.
>  * From _impala-shell,_ run _alter table weekly add columns ( newCol string_);
> _Expected results_
>  * HMS logs should not show _NullPointerException_.
>  * Atlas should show the table weekly with the newCol column.
>  
> _Observed results_:
> HMS logs show _NullPointerExcetion_ from Atlas hook.
> New entity _newCol_ is not seen within Atlas.
> *Root cause*
> When assessing the incoming event to determine the type of alter, Atlas uses 
> table parameter. The recent build has a new parameter for timestamp: 
> _last_modified_time_ _transient_lastDdlTime_. This results in incorrect 
> assessment. Hence the alter event is incorrectly processed, thereby causing 
> an exception.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

2020-09-28 Thread chaitali


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72895/
---

(Updated Sept. 28, 2020, 8:11 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3950
https://issues.apache.org/jira/browse/ATLAS-3950


Repository: atlas


Description
---

User is able to create an attribute for 
businessmetadata/classification/entitydef of typename which doesn't have read 
access in the ranger policy. UI works fine but but through REST , User is able 
to add any type as attribute typename.

Steps to reproduce:

User has following rights in ranger policy :

CRUD permissions on hive_table type only

CRUD permissions on all business_metadata type.

hrt_qa creates a business metadata bm1

hrt_qa is able to create an attribute for bm1 - say attrib1 which allows 
Applicable types to be anything. UI displays only hive_table but through REST , 
hrt_qa is able to add any type as Applicable type.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java
 2cb2b4789 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasBusinessMetadataDefStoreV2.java
 6b4fa65ae 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasClassificationDefStoreV2.java
 9ffede4e3 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityDefStoreV2.java
 e5153de0e 


Diff: https://reviews.apache.org/r/72895/diff/8/

Changes: https://reviews.apache.org/r/72895/diff/7-8/


Testing
---

Tested with following JSON : 

Entity typedef creation with an attribute of typename which admin doesn't have 
type-read rights for:
{
"enumDefs": [],
"structDefs": [],
"classificationDefs": [],
"entityDefs": [
  {
"category": "ENTITY",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1537261952180,
"updateTime": 1537262097732,
"version": 3,
"name": "testtab",
"description": "sport",
"typeVersion": "1.4",
"attributeDefs": [
  {
"name": "test_name",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
},
{
"name": "year",
"typeName": "int",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
},
{
"name": "place",
"typeName": "array",
"isOptional": true,
"cardinality": "SINGLE",
"valuesMinCount": 0,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false,
"includeInNotification": false
}
],
"superTypes": [
  "DataSet"
],
"subTypes": []
}
],
"relationshipDefs": []
}

Classification typedef creation with entitytypes for which doesn't have 
type-read rights for:

{
  "classificationDefs": [
{
  "name": "testclass",
  "description": "oif",
  "superTypes": [],
  "attributeDefs": [],
  "entityTypes": [
"hdfs_path",
"hive_table"
  ],
  "category": "CLASSIFICATION",
  "guid": "-123456789"
}
  ],
  "entityDefs": [],
  "enumDefs": [],
  "structDefs": []
}

Businessmetadata typedef creation with entitytypes for which admin doesn't have 
type-read rights for:

{
  "enumDefs": [],
  "structDefs": [],
  "classificationDefs": [],
  "entityDefs": [],
  "businessMetadataDefs": [
{
  "category": "BUSINESS_METADATA",
  "createdBy": "admin",
  "updatedBy": "admin",
  "version": 1,
  "typeVersion": "1.1",
  "name": "testBM",
  "description": "",
  "attributeDefs": [
{
  "name": "attrtestBM",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": true,
  "options": {
"maxStrLength": "50",
"applicableEntityTypes": "[\"hive_table\"]"
  },
  "searchWeight": "5"
}
  ]
}
  ]
}

Expected Output :

{
"errorCode": "ATLAS-403-00-001",
"errorMessage": "admin is not authorized to perform read type hive_table"
}


Thanks,

chaitali

[jira] [Updated] (ATLAS-3957) Use Audit framework to capture audit entries for "Server Start" and "Server In Active mode" (HA)

2020-09-28 Thread Mandar Ambawane (Jira)



 [ 
https://issues.apache.org/jira/browse/ATLAS-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandar Ambawane updated ATLAS-3957:
---
Attachment: ATLAS-3957.patch

> Use Audit framework to capture audit entries for "Server Start" and "Server 
> In Active mode" (HA)
> 
>
> Key: ATLAS-3957
> URL: https://issues.apache.org/jira/browse/ATLAS-3957
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Attachments: ATLAS-3957.patch
>
>
> By using Audit Framework, capture audit entries for "Server Start" and 
> "Server In Active mode" (HA)
> These audit entries can be seen under the "Audits" tab of Admin section.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72912: ATLAS-3962 : Include business metadata def header in typdefs headers API

[jira] [Created] (ATLAS-3963) Atlas api 'atlas/v2/types/typedefs/headers' does not give information about business metadata

Review Request 72912: ATLAS-3962 : Include business metadata def header in typdefs headers API

[jira] [Created] (ATLAS-3962) Include business metadata def header in typdefs headers API

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

[jira] [Updated] (ATLAS-3961) Atlas should encrypt LDAP/AD bind user password

[jira] [Created] (ATLAS-3961) Atlas should encrypt LDAP/AD bind user password

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

[jira] [Commented] (ATLAS-3959) Upgrade Atlas to Spring Framework version 4.3.16.release

[jira] [Created] (ATLAS-3960) Testcase failing [testGetDefinition(org.apache.atlas.web.integration.TypedefsJerseyResourceIT)]

[jira] [Commented] (ATLAS-3608) Hive Bridge: Hive Metastore: Alter Table Query Not Handled Correctly

Re: Review Request 72895: ATLAS-3950 : Read Type Auth : Classification, Business metadata , Entity types are able to have attributes of type which are not permissible to read

[jira] [Updated] (ATLAS-3957) Use Audit framework to capture audit entries for "Server Start" and "Server In Active mode" (HA)

15 matches

Site Navigation

Mail list logo

Footer information