[GitHub] [atlas] heary-cao opened a new pull request #41: ATLAS-3135 delete null data from typedefs results in messy data

[GitBox]

heary-cao opened a new pull request #41: ATLAS-3135 delete null data from 
typedefs results in messy data
URL: https://github.com/apache/atlas/pull/41
 
 
   Currently, when we use http://IP:21000/api/atlas/v2/types/typedefs/, 
deleting the data of typedefs is empty 
({"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
 it will lead to messy data, and basic search error, so the atlas system must 
be restarted in order to recover the data.
   
   issue: https://issues.apache.org/jira/browse/ATLAS-3135
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

caoxuewen updated ATLAS-3135:
-
Attachment: image-2019-04-13-11-46-23-779.png

> delete null data from typedefs results in messy data
> 
>
> Key: ATLAS-3135
> URL: https://issues.apache.org/jira/browse/ATLAS-3135
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: caoxuewen
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: image-2019-04-13-11-45-12-868.png, 
> image-2019-04-13-11-45-20-381.png, image-2019-04-13-11-45-32-642.png, 
> image-2019-04-13-11-45-39-847.png, image-2019-04-13-11-45-47-698.png, 
> image-2019-04-13-11-46-23-779.png
>
>
> Currently, when we use 
> [http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
>  deleting the data of typedefs is empty 
> (\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
>  it will lead to messy data, and basic search error, so the atlas system must 
> be restarted in order to recover the data.
> Correct data:
> !image-2019-04-13-11-45-12-868.png!
> !image-2019-04-13-11-45-20-381.png!
>  
> exec 
> [http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
>  json data: 
> \{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}
> atlas entity data is: （basic search is error）
> !image-2019-04-13-11-45-32-642.png!
> messy data:
> !image-2019-04-13-11-45-39-847.png!
> !image-2019-04-13-11-45-47-698.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

caoxuewen updated ATLAS-3135:
-
Description: 
Currently, when we use 
[http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
 deleting the data of typedefs is empty 
(\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
 it will lead to messy data, and basic search error, so the atlas system must 
be restarted in order to recover the data.

Correct data:

!image-2019-04-13-11-45-12-868.png!

!image-2019-04-13-11-45-20-381.png!

 

exec 
[http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
 json data: 
\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}

atlas entity data is: （basic search is error）

!image-2019-04-13-11-45-32-642.png!

messy data:

!image-2019-04-13-11-45-39-847.png!

!image-2019-04-13-11-46-23-779.png!

  was:
Currently, when we use 
[http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
 deleting the data of typedefs is empty 
(\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
 it will lead to messy data, and basic search error, so the atlas system must 
be restarted in order to recover the data.

Correct data:

!image-2019-04-13-11-45-12-868.png!

!image-2019-04-13-11-45-20-381.png!

 

exec 
[http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
 json data: 
\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}

atlas entity data is: （basic search is error）

!image-2019-04-13-11-45-32-642.png!

messy data:

!image-2019-04-13-11-45-39-847.png!

!image-2019-04-13-11-45-47-698.png!


> delete null data from typedefs results in messy data
> 
>
> Key: ATLAS-3135
> URL: https://issues.apache.org/jira/browse/ATLAS-3135
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: caoxuewen
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: image-2019-04-13-11-45-12-868.png, 
> image-2019-04-13-11-45-20-381.png, image-2019-04-13-11-45-32-642.png, 
> image-2019-04-13-11-45-39-847.png, image-2019-04-13-11-45-47-698.png, 
> image-2019-04-13-11-46-23-779.png
>
>
> Currently, when we use 
> [http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
>  deleting the data of typedefs is empty 
> (\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
>  it will lead to messy data, and basic search error, so the atlas system must 
> be restarted in order to recover the data.
> Correct data:
> !image-2019-04-13-11-45-12-868.png!
> !image-2019-04-13-11-45-20-381.png!
>  
> exec 
> [http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
>  json data: 
> \{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}
> atlas entity data is: （basic search is error）
> !image-2019-04-13-11-45-32-642.png!
> messy data:
> !image-2019-04-13-11-45-39-847.png!
> !image-2019-04-13-11-46-23-779.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

caoxuewen updated ATLAS-3135:
-
Attachment: image-2019-04-13-11-45-12-868.png

> delete null data from typedefs results in messy data
> 
>
> Key: ATLAS-3135
> URL: https://issues.apache.org/jira/browse/ATLAS-3135
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: caoxuewen
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: image-2019-04-13-11-45-12-868.png, 
> image-2019-04-13-11-45-20-381.png, image-2019-04-13-11-45-32-642.png, 
> image-2019-04-13-11-45-39-847.png, image-2019-04-13-11-45-47-698.png
>
>
> Currently, when we use http://IP:21000/api/atlas/v2/types/typedefs/, deleting 
> the data of typedefs is empty 
> (\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
>  it will lead to messy data, and basic search error, so the atlas system must 
> be restarted in order to recover the data.
> Correct data:
> !image-2019-04-13-11-40-14-653.png!
> !image-2019-04-13-11-40-26-552.png!
>  
> exec http://IP:21000/api/atlas/v2/types/typedefs/, json data: 
> \{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}
> atlas entity data is: （basic search is error）
> !image-2019-04-13-11-42-28-808.png!
> messy data:
> !image-2019-04-13-11-42-36-593.png!
> !image-2019-04-13-11-43-27-551.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

caoxuewen updated ATLAS-3135:
-
Attachment: image-2019-04-13-11-45-20-381.png

> delete null data from typedefs results in messy data
> 
>
> Key: ATLAS-3135
> URL: https://issues.apache.org/jira/browse/ATLAS-3135
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: caoxuewen
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: image-2019-04-13-11-45-12-868.png, 
> image-2019-04-13-11-45-20-381.png, image-2019-04-13-11-45-32-642.png, 
> image-2019-04-13-11-45-39-847.png, image-2019-04-13-11-45-47-698.png
>
>
> Currently, when we use http://IP:21000/api/atlas/v2/types/typedefs/, deleting 
> the data of typedefs is empty 
> (\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
>  it will lead to messy data, and basic search error, so the atlas system must 
> be restarted in order to recover the data.
> Correct data:
> !image-2019-04-13-11-40-14-653.png!
> !image-2019-04-13-11-40-26-552.png!
>  
> exec http://IP:21000/api/atlas/v2/types/typedefs/, json data: 
> \{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}
> atlas entity data is: （basic search is error）
> !image-2019-04-13-11-42-28-808.png!
> messy data:
> !image-2019-04-13-11-42-36-593.png!
> !image-2019-04-13-11-43-27-551.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

caoxuewen updated ATLAS-3135:
-
Description: 
Currently, when we use 
[http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
 deleting the data of typedefs is empty 
(\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
 it will lead to messy data, and basic search error, so the atlas system must 
be restarted in order to recover the data.

Correct data:

!image-2019-04-13-11-45-12-868.png!

!image-2019-04-13-11-45-20-381.png!

 

exec 
[http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
 json data: 
\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}

atlas entity data is: （basic search is error）

!image-2019-04-13-11-45-32-642.png!

messy data:

!image-2019-04-13-11-45-39-847.png!

!image-2019-04-13-11-45-47-698.png!

  was:
Currently, when we use http://IP:21000/api/atlas/v2/types/typedefs/, deleting 
the data of typedefs is empty 
(\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
 it will lead to messy data, and basic search error, so the atlas system must 
be restarted in order to recover the data.

Correct data:

!image-2019-04-13-11-40-14-653.png!

!image-2019-04-13-11-40-26-552.png!

 

exec http://IP:21000/api/atlas/v2/types/typedefs/, json data: 
\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}

atlas entity data is: （basic search is error）

!image-2019-04-13-11-42-28-808.png!

messy data:

!image-2019-04-13-11-42-36-593.png!

!image-2019-04-13-11-43-27-551.png!


> delete null data from typedefs results in messy data
> 
>
> Key: ATLAS-3135
> URL: https://issues.apache.org/jira/browse/ATLAS-3135
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: caoxuewen
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: image-2019-04-13-11-45-12-868.png, 
> image-2019-04-13-11-45-20-381.png, image-2019-04-13-11-45-32-642.png, 
> image-2019-04-13-11-45-39-847.png, image-2019-04-13-11-45-47-698.png
>
>
> Currently, when we use 
> [http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
>  deleting the data of typedefs is empty 
> (\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
>  it will lead to messy data, and basic search error, so the atlas system must 
> be restarted in order to recover the data.
> Correct data:
> !image-2019-04-13-11-45-12-868.png!
> !image-2019-04-13-11-45-20-381.png!
>  
> exec 
> [http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
>  json data: 
> \{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}
> atlas entity data is: （basic search is error）
> !image-2019-04-13-11-45-32-642.png!
> messy data:
> !image-2019-04-13-11-45-39-847.png!
> !image-2019-04-13-11-45-47-698.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

caoxuewen updated ATLAS-3135:
-
Attachment: image-2019-04-13-11-45-39-847.png

> delete null data from typedefs results in messy data
> 
>
> Key: ATLAS-3135
> URL: https://issues.apache.org/jira/browse/ATLAS-3135
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: caoxuewen
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: image-2019-04-13-11-45-12-868.png, 
> image-2019-04-13-11-45-20-381.png, image-2019-04-13-11-45-32-642.png, 
> image-2019-04-13-11-45-39-847.png, image-2019-04-13-11-45-47-698.png
>
>
> Currently, when we use http://IP:21000/api/atlas/v2/types/typedefs/, deleting 
> the data of typedefs is empty 
> (\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
>  it will lead to messy data, and basic search error, so the atlas system must 
> be restarted in order to recover the data.
> Correct data:
> !image-2019-04-13-11-40-14-653.png!
> !image-2019-04-13-11-40-26-552.png!
>  
> exec http://IP:21000/api/atlas/v2/types/typedefs/, json data: 
> \{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}
> atlas entity data is: （basic search is error）
> !image-2019-04-13-11-42-28-808.png!
> messy data:
> !image-2019-04-13-11-42-36-593.png!
> !image-2019-04-13-11-43-27-551.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

caoxuewen updated ATLAS-3135:
-
Attachment: image-2019-04-13-11-45-32-642.png

> delete null data from typedefs results in messy data
> 
>
> Key: ATLAS-3135
> URL: https://issues.apache.org/jira/browse/ATLAS-3135
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: caoxuewen
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: image-2019-04-13-11-45-12-868.png, 
> image-2019-04-13-11-45-20-381.png, image-2019-04-13-11-45-32-642.png, 
> image-2019-04-13-11-45-39-847.png, image-2019-04-13-11-45-47-698.png
>
>
> Currently, when we use http://IP:21000/api/atlas/v2/types/typedefs/, deleting 
> the data of typedefs is empty 
> (\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
>  it will lead to messy data, and basic search error, so the atlas system must 
> be restarted in order to recover the data.
> Correct data:
> !image-2019-04-13-11-40-14-653.png!
> !image-2019-04-13-11-40-26-552.png!
>  
> exec http://IP:21000/api/atlas/v2/types/typedefs/, json data: 
> \{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}
> atlas entity data is: （basic search is error）
> !image-2019-04-13-11-42-28-808.png!
> messy data:
> !image-2019-04-13-11-42-36-593.png!
> !image-2019-04-13-11-43-27-551.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

caoxuewen updated ATLAS-3135:
-
Attachment: image-2019-04-13-11-45-47-698.png

> delete null data from typedefs results in messy data
> 
>
> Key: ATLAS-3135
> URL: https://issues.apache.org/jira/browse/ATLAS-3135
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: caoxuewen
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: image-2019-04-13-11-45-12-868.png, 
> image-2019-04-13-11-45-20-381.png, image-2019-04-13-11-45-32-642.png, 
> image-2019-04-13-11-45-39-847.png, image-2019-04-13-11-45-47-698.png
>
>
> Currently, when we use 
> [http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
>  deleting the data of typedefs is empty 
> (\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
>  it will lead to messy data, and basic search error, so the atlas system must 
> be restarted in order to recover the data.
> Correct data:
> !image-2019-04-13-11-45-12-868.png!
> !image-2019-04-13-11-45-20-381.png!
>  
> exec 
> [http://IP:21000/api/atlas/v2/types/typedefs/|http://ip:21000/api/atlas/v2/types/typedefs/],
>  json data: 
> \{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}
> atlas entity data is: （basic search is error）
> !image-2019-04-13-11-45-32-642.png!
> messy data:
> !image-2019-04-13-11-45-39-847.png!
> !image-2019-04-13-11-45-47-698.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-3135) delete null data from typedefs results in messy data

[caoxuewen (JIRA)]

caoxuewen created ATLAS-3135:


 Summary: delete null data from typedefs results in messy data
 Key: ATLAS-3135
 URL: https://issues.apache.org/jira/browse/ATLAS-3135
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0
Reporter: caoxuewen
 Fix For: 1.1.0


Currently, when we use http://IP:21000/api/atlas/v2/types/typedefs/, deleting 
the data of typedefs is empty 
(\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}),
 it will lead to messy data, and basic search error, so the atlas system must 
be restarted in order to recover the data.

Correct data:

!image-2019-04-13-11-40-14-653.png!

!image-2019-04-13-11-40-26-552.png!

 

exec http://IP:21000/api/atlas/v2/types/typedefs/, json data: 
\{"enumDefs":[],"structDefs":[],"classificationDefs":[],"entityDefs":[],"relationshipDefs":[]}

atlas entity data is: （basic search is error）

!image-2019-04-13-11-42-28-808.png!

messy data:

!image-2019-04-13-11-42-36-593.png!

!image-2019-04-13-11-43-27-551.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70463: ATLAS-3132: Improvements for Structure and Performance

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70463/#review214622
---




graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java
Lines 28 (patched)


this 'import' is the only change in the file. Please revert.



graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/DataPatchGraphDBHandler.java
Lines 27 (patched)


Its not clear what 'apply()' would do with 'typeNameAttributeMap'. Consider 
a better name that coveys the operation performed.



graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/VertexIterator.java
Lines 42 (patched)


next() method is creating a vertex?

This class, 'VertexIterator', seems to be to covert Gremlin 'Vertext' to 
'AtlasVertex'. Perhaps a better naming or abstraction?

And this class doesn't seem to be used at all. If not needed, please remove.



repository/src/main/java/org/apache/atlas/repository/patches/DataPatchHandler.java
Lines 24 (patched)


I think JavaPatchHandler is better, as the patch can be used to make 
changes to types and data as well. The name JavaPatch is to differentiate 
typedef-patching mechanism used during bootstrap.

I suggest to revert this rename of classes.



repository/src/main/java/org/apache/atlas/repository/patches/DataPatchService.java
Lines 30 (patched)


Use of @Component for DataPatchService seems unnecessary. This is needed 
only during startup; we should keep these references around for the life of the 
server. Please review and update.



repository/src/main/java/org/apache/atlas/repository/patches/UniqueAttributePatch.java
Lines 39 (patched)


Use of @Component for patches seems unnecessary. These patch handlers are 
needed only during startup, that too only if the corresponding patch hasn't 
already been applied. Please review and revert.



repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
Line 160 (original)


Why was PatchContext abstraction removed?



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
Lines 393 (patched)


findAllByTypeAndUniquePropertyName => 
findActiveEntityVerticesByTypeAndPropertyName


- Madhan Neethiraj


On April 12, 2019, 5:43 a.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70463/
> ---
> 
> (Updated April 12, 2019, 5:43 a.m.)
> 
> 
> Review request for atlas, Kapildeo Nayak, Madhan Neethiraj, Nikhil Bonte, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3132
> https://issues.apache.org/jira/browse/ATLAS-3132
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> - Refactored existing implementation for new design.
> - Renamed 'Java Patch Framework' to 'Data Patch Framework', rationale being 
> that this is essentially to modify structure of existing data.
> - New _DataPatchService_: Modified order in which services are called. 
> _DataPatchService_ will be called before other services are invoked, thereby 
> giving chance for it to complete before entertaining new data.
> - New _DataPatchRegistry_: Data access (CRUD) operation for data patches.
> - New _UniqueAttributePatchHandler_: Current implementation for adding the 
> new property to data vertices. Implemented rudimentary caching to precent 
> repetitive look-ups.
> - New REST Endpoint to query status of patches.
> 
> **Performance**
> Since the data patching operation is high-volume operation, it has been 
> treated with priority. 
> - New _NewPropertyDataHandler_ uses database in bulk loading mode for rapid 
> processing. This scales with resources. Additional properties:
> - _atlas.processing.batchSize_: Size of batch.
> - _atlas.processing.numWorkers_: Number of worker threads to be employed. 
> - Leverages existing PC framework.
> 
> Processing speed:
> - 300K vertices: ~5 mins (8 threads, batch size: 3000)
> - 4.2 M entities: ~45 mins (from: 2019-04-12 04:44:50 to 2019-04-12 
> 05:29:04), (4 threads, batch size: 300)
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
> d282c9966 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/DataPatchGraphDBHandler.java
>  PRE-CREATION 
>   
> 

[jira] [Commented] (ATLAS-3131) UI : Stats Popup Improvements

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816506#comment-16816506
 ] 

ASF subversion and git services commented on ATLAS-3131:


Commit 5a29491c4f4a99a30ddcedd26d2d228a5daa72a6 in atlas's branch 
refs/heads/branch-2.0 from Keval Bhatt
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=5a29491 ]

ATLAS-3131: stats UI improvements

Signed-off-by: Madhan Neethiraj 
(cherry picked from commit d61bfcfb03a116203df6280924eb9df94194026a)


> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3131-3.patch, ATLAS-3131-4.patch, 
> ATLAS-3131.patch, ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3131) UI : Stats Popup Improvements

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816505#comment-16816505
 ] 

ASF subversion and git services commented on ATLAS-3131:


Commit d61bfcfb03a116203df6280924eb9df94194026a in atlas's branch 
refs/heads/master from Keval Bhatt
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=d61bfcf ]

ATLAS-3131: stats UI improvements

Signed-off-by: Madhan Neethiraj 


> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3131-3.patch, ATLAS-3131-4.patch, 
> ATLAS-3131.patch, ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3131) UI : Stats Popup Improvements

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3131:
---
Attachment: ATLAS-3131-4.patch

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3131-3.patch, ATLAS-3131-4.patch, 
> ATLAS-3131.patch, ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (ATLAS-3131) UI : Stats Popup Improvements

[Srikanth Venkat (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816462#comment-16816462
 ] 

Srikanth Venkat edited comment on ATLAS-3131 at 4/12/19 4:57 PM:
-

Please add before and after screenshots so we can understand what the 
improvements are especially since there isn't a lot of description


was (Author: srikvenk):
Please add before and after screenshots so we can understand what the 
improvements are.

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3131-3.patch, ATLAS-3131.patch, 
> ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3131) UI : Stats Popup Improvements

[Srikanth Venkat (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816462#comment-16816462
 ] 

Srikanth Venkat commented on ATLAS-3131:


Please add before and after screenshots so we can understand what the 
improvements are.

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3131-3.patch, ATLAS-3131.patch, 
> ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3131) UI : Stats Popup Improvements

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3131:
---
Attachment: ATLAS-3131-3.patch

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3131-3.patch, ATLAS-3131.patch, 
> ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (ATLAS-3131) UI : Stats Popup Improvements

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt reassigned ATLAS-3131:
--

Assignee: Keval Bhatt  (was: Binit Gutka)

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3131.patch, ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (ATLAS-3131) UI : Stats Popup Improvements

[Keval Bhatt (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816241#comment-16816241
 ] 

Keval Bhatt edited comment on ATLAS-3131 at 4/12/19 1:08 PM:
-

[~binitgutka] I have rebased your patch [^ATLAS-3131_2.patch]


was (Author: kevalbhatt18):
[~binitgutka] I have rebased your patch 
[^ATLAS-3131_2.patch]^[^ATLAS-3131_2.patch]^

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Binit Gutka
>Priority: Major
> Attachments: ATLAS-3131.patch, ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3131) UI : Stats Popup Improvements

[Keval Bhatt (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816241#comment-16816241
 ] 

Keval Bhatt commented on ATLAS-3131:


[~binitgutka] I have rebased your patch 
[^ATLAS-3131_2.patch]^[^ATLAS-3131_2.patch]^

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Binit Gutka
>Priority: Major
> Attachments: ATLAS-3131.patch, ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3131) UI : Stats Popup Improvements

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3131:
---
Attachment: ATLAS-3131_2.patch

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Binit Gutka
>Priority: Major
> Attachments: ATLAS-3131.patch, ATLAS-3131_1.patch, ATLAS-3131_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3128) UI: Create entity is not working after new relationship attribute introduce

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3128:
---
Attachment: ATLAS-3128-new-improvment_1.patch

> UI: Create entity is not working after new relationship attribute introduce
> ---
>
> Key: ATLAS-3128
> URL: https://issues.apache.org/jira/browse/ATLAS-3128
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3128-1.patch, ATLAS-3128-2.patch, 
> ATLAS-3128-new-improvment.patch, ATLAS-3128-new-improvment_1.patch, 
> ATLAS-3128.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3134) Change Date.getTime() to System.currentTimeMillis() to improve performance

[bd2019us (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bd2019us updated ATLAS-3134:

Description: 
new Date() is a thin wrapper of method System.currentTimeMillis(). If it is 
intensively invoked in the program, the performance will be greatly damaged.  
 According to my local testing, when these two methods are invoked 5,000,000 
times at the same environment, System.currentTimeMillis() can achieve a speedup 
to 5 times (435ms vs 2073ms). 
 Therefore, if only getTime() is used for Date object, the light method 
System.currentTimeMillis() is highly recommended, which can also avoid creating 
the temporary Date object.
  

  was:
{color:#33}Location: 
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java:527{color}
 
new Date() is a thin wrapper of method System.currentTimeMillis(). If it is 
intensively invoked in the program, the performance will be greatly damaged.  
According to my local testing, when these two methods are invoked 5,000,000 
times at the same environment, System.currentTimeMillis() can achieve a speedup 
to 5 times (435ms vs 2073ms). 
Therefore, if only getTime() is used for Date object, the light method 
System.currentTimeMillis() is highly recommended, which can also avoid creating 
the temporary Date object.
 


> Change Date.getTime() to System.currentTimeMillis() to improve performance
> --
>
> Key: ATLAS-3134
> URL: https://issues.apache.org/jira/browse/ATLAS-3134
> Project: Atlas
>  Issue Type: Bug
>Reporter: bd2019us
>Priority: Major
> Attachments: 1.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> new Date() is a thin wrapper of method System.currentTimeMillis(). If it is 
> intensively invoked in the program, the performance will be greatly damaged.  
>  According to my local testing, when these two methods are invoked 5,000,000 
> times at the same environment, System.currentTimeMillis() can achieve a 
> speedup to 5 times (435ms vs 2073ms). 
>  Therefore, if only getTime() is used for Date object, the light method 
> System.currentTimeMillis() is highly recommended, which can also avoid 
> creating the temporary Date object.
>   



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70457: ATLAS-3131 UI : Stats Popup Improvements

[Binit Gutka]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70457/
---

(Updated April 12, 2019, 12:08 p.m.)


Review request for atlas, keval bhatt, Madhan Neethiraj, Nikhil Bonte, and 
Sarath Subramanian.


Changes
---

Changes in the Notification table structure


Bugs: ATLAS-3131
https://issues.apache.org/jira/browse/ATLAS-3131


Repository: atlas


Description
---

Sepreation of server stats, notification stats and connections


Diffs (updated)
-

  dashboardv2/public/css/scss/theme.scss 25ca86806 
  dashboardv2/public/js/templates/site/Statistics_Notification_table_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/templates/site/Statistics_tmpl.html 71643c7b0 
  dashboardv2/public/js/utils/Enums.js 8df22af1a 
  dashboardv2/public/js/utils/Utils.js 56bd844f1 
  dashboardv2/public/js/views/site/Statistics.js 9d3478483 


Diff: https://reviews.apache.org/r/70457/diff/2/

Changes: https://reviews.apache.org/r/70457/diff/1-2/


Testing
---

Done one round of sanity test


Thanks,

Binit Gutka

[jira] [Updated] (ATLAS-3131) UI : Stats Popup Improvements

[Binit Gutka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Binit Gutka updated ATLAS-3131:
---
Attachment: ATLAS-3131_1.patch

> UI : Stats Popup Improvements
> -
>
> Key: ATLAS-3131
> URL: https://issues.apache.org/jira/browse/ATLAS-3131
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Binit Gutka
>Assignee: Binit Gutka
>Priority: Major
> Attachments: ATLAS-3131.patch, ATLAS-3131_1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)