Re: Review Request 47658: ATLAS-499: Update reverse references

2017-02-10 Thread David Kantor 


> On Feb. 10, 2017, 7:50 p.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java,
> >  line 749
> > 
> >
> > Isn't there a need for "removeReverseReference()" - to update the 
> > reverseAttribute when a referred entity is deleted? For example: when a 
> > hive_column entity is deleted, hive_table.columns needs to be updated.
> > 
> > Perhaps this is already handled in the current implementation. If so, 
> > please ignore this comment.

This is already handled in DeleteHandler.deleteEdge().  Also this review 
request had been discarded a few days ago, so that I could separate the V1 and 
V2 changes.  If you have any further review comments on the V1 changes, please 
post them to https://reviews.apache.org/r/56287/


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review165156
---


On June 17, 2016, 2:44 a.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 17, 2016, 2:44 a.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14 
>   addons/hive-bridge/src/test/resources/je.properties PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2017-02-10 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review165156
---




repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 (line 748)


Isn't there a need for "removeReverseReference()" - to update the 
reverseAttribute when a referred entity is deleted? For example: when a 
hive_column entity is deleted, hive_table.columns needs to be updated.

Perhaps this is already handled in the current implementation. If so, 
please ignore this comment.


- Madhan Neethiraj


On June 17, 2016, 2:44 a.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 17, 2016, 2:44 a.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14 
>   addons/hive-bridge/src/test/resources/je.properties PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-16 Thread David Kantor 


> On June 9, 2016, 11:30 a.m., Shwetha GS wrote:
> > The following tests failed for me with this patch. Did they work for you?
> > Failed tests:
> >   
> > HiveHookIT.testAlterTablePartitionColumnType:911->assertColumnIsRegistered:276->assertEntityIsRegistered:1498->waitFor:1591
> >  Assertions failed. Failing after waiting for timeout 8 msecs
> >   
> > HiveHookIT.testAlterTableRename:659->assertColumnIsRegistered:271->assertColumnIsRegistered:276->assertEntityIsRegistered:1498->waitFor:1591
> >  Assertions failed. Failing after waiting for timeout 8 msecs
> > 
> > Tests run: 43, Failures: 2, Errors: 0, Skipped: 0
> 
> David Kantor wrote:
> I am getting this test failure, will investigate.
> 
> David Kantor wrote:
> I have investigated the test failures, and they are being caused by lock 
> timeouts that are causing entity updates applied by the 
> NotificationHookConsumer to fail, so the test fails to find the expected 
> updated entity within 8 msec.  This looks similar to the issue raised in 
> https://issues.apache.org/jira/browse/ATLAS-759.  It appears that the 
> addition of updating the reverse reference is causing more contention on 
> berkeley db locks between Jersey threads reading the data on behalf of the 
> test vs NotificationHookConsumer threads writing the data.  The default lock 
> timeout for berkeley db is only 500ms which seems pretty small.  The question 
> is whether the increased berkeley  db lock contention is exposing a problem 
> with my changes or with the test design.  Another question is whether this is 
> an issue when running on HBase.  Thoughts?
> 
> Shwetha GS wrote:
> In these tests, you can try if replacing runCommand() with 
> runCommandWithDelay() will help
> 
> David Kantor wrote:
> Increased Berkeley DB lock timeout setting and that resolved all test 
> failures.

Disregard previous comment, I am still seeing the test failures even with 
increasing the lock timeout, needs further investigation.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136791
---


On June 17, 2016, 2:44 a.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 17, 2016, 2:44 a.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14 
>   addons/hive-bridge/src/test/resources/je.properties PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-16 Thread David Kantor 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/
---

(Updated June 17, 2016, 2:44 a.m.)


Review request for atlas and Shwetha GS.


Changes
---

Resolve test failures


Bugs: ATLAS-499
https://issues.apache.org/jira/browse/ATLAS-499


Repository: atlas


Description
---

ATLAS-499: Automatically update inverse references to prevent repository 
corruption from unbalanced references.


Diffs (updated)
-

  addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14 
  addons/hive-bridge/src/test/resources/je.properties PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
91f9bd008a6939dfe78656f5c1845637a30ee9eb 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
4f6d0118072380342f5ea302d0aaf7902784849a 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 4c1f5591f4adead41c3336e64b0bc956836f7edb 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
 449e066a167ba4546b118a77c8e3de5fd99f077b 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47658/diff/


Testing
---

Ran all unit and integration tests with no regressions.


Thanks,

David Kantor

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-16 Thread David Kantor 


> On June 9, 2016, 11:30 a.m., Shwetha GS wrote:
> > The following tests failed for me with this patch. Did they work for you?
> > Failed tests:
> >   
> > HiveHookIT.testAlterTablePartitionColumnType:911->assertColumnIsRegistered:276->assertEntityIsRegistered:1498->waitFor:1591
> >  Assertions failed. Failing after waiting for timeout 8 msecs
> >   
> > HiveHookIT.testAlterTableRename:659->assertColumnIsRegistered:271->assertColumnIsRegistered:276->assertEntityIsRegistered:1498->waitFor:1591
> >  Assertions failed. Failing after waiting for timeout 8 msecs
> > 
> > Tests run: 43, Failures: 2, Errors: 0, Skipped: 0
> 
> David Kantor wrote:
> I am getting this test failure, will investigate.
> 
> David Kantor wrote:
> I have investigated the test failures, and they are being caused by lock 
> timeouts that are causing entity updates applied by the 
> NotificationHookConsumer to fail, so the test fails to find the expected 
> updated entity within 8 msec.  This looks similar to the issue raised in 
> https://issues.apache.org/jira/browse/ATLAS-759.  It appears that the 
> addition of updating the reverse reference is causing more contention on 
> berkeley db locks between Jersey threads reading the data on behalf of the 
> test vs NotificationHookConsumer threads writing the data.  The default lock 
> timeout for berkeley db is only 500ms which seems pretty small.  The question 
> is whether the increased berkeley  db lock contention is exposing a problem 
> with my changes or with the test design.  Another question is whether this is 
> an issue when running on HBase.  Thoughts?
> 
> Shwetha GS wrote:
> In these tests, you can try if replacing runCommand() with 
> runCommandWithDelay() will help

Increased Berkeley DB lock timeout setting and that resolved all test failures.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136791
---


On June 17, 2016, 1:32 a.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 17, 2016, 1:32 a.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14 
>   addons/hive-bridge/src/test/resources/je.properties PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-16 Thread David Kantor 


> On June 3, 2016, 11:29 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java,
> >  line 443
> > 
> >
> > for map types, can we check if the reverseAttributeName is set on the 
> > map type and remove it if set else ignore?
> 
> David Kantor wrote:
> I'm thinking it may be better to remove the reverse attribute name when 
> the type is registered and possibly issue a warning that bi-directional 
> references are not supported for map types, rather then doing it every time 
> there is an update operation involving that type.  Thoughts?
> 
> Suma Shivaprasad wrote:
> yeah that seems like a better option
> 
> David Kantor wrote:
> In thinking more about this, perhaps it is beyond the scope of this JIRA 
> to make such a significant change to the type system functionality.  You 
> could argue that having a bi-directional reference where one of the ends is a 
> map reference is OK, it's just that in the specific case where the user just 
> updates the non-map end of the reference, we don't know what to use for the 
> map key when updating the reverse map reference so we can't support 
> auto-updating in that use case.  It doesn't necessarily mean that the type 
> system should ignore/remove the reverseAttributeName on a map reference at 
> registration time.  Modelers may have a valid use case for bi-directional map 
> references and if we suddenly change the type system to ignore or remove the 
> reverseAttributeName, that could be a breaking change to existing 
> applications.  Perhaps your point in the original comment is that if we don't 
> support auto-updating when just the non-map reference is updated, we should 
> be consistent and not 
 support automatic reverse reference updating if *either* end of the reference 
is a map.

Added logic to avoid automatic reverse reference updating if either end of a 
bi-directional reference is a map.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136133
---


On June 17, 2016, 1:32 a.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 17, 2016, 1:32 a.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14 
>   addons/hive-bridge/src/test/resources/je.properties PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-16 Thread David Kantor 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/
---

(Updated June 17, 2016, 1:32 a.m.)


Review request for atlas and Shwetha GS.


Changes
---

Override default Berkeley DB lock timeout to address HiveIT test failures.  
Optimize graph I/O to avoid unnecessary edge queries.


Bugs: ATLAS-499
https://issues.apache.org/jira/browse/ATLAS-499


Repository: atlas


Description
---

ATLAS-499: Automatically update inverse references to prevent repository 
corruption from unbalanced references.


Diffs (updated)
-

  addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14 
  addons/hive-bridge/src/test/resources/je.properties PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
91f9bd008a6939dfe78656f5c1845637a30ee9eb 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
4f6d0118072380342f5ea302d0aaf7902784849a 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 4c1f5591f4adead41c3336e64b0bc956836f7edb 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
 449e066a167ba4546b118a77c8e3de5fd99f077b 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47658/diff/


Testing
---

Ran all unit and integration tests with no regressions.


Thanks,

David Kantor

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-09 Thread David Kantor 


> On June 9, 2016, 11:30 a.m., Shwetha GS wrote:
> > The following tests failed for me with this patch. Did they work for you?
> > Failed tests:
> >   
> > HiveHookIT.testAlterTablePartitionColumnType:911->assertColumnIsRegistered:276->assertEntityIsRegistered:1498->waitFor:1591
> >  Assertions failed. Failing after waiting for timeout 8 msecs
> >   
> > HiveHookIT.testAlterTableRename:659->assertColumnIsRegistered:271->assertColumnIsRegistered:276->assertEntityIsRegistered:1498->waitFor:1591
> >  Assertions failed. Failing after waiting for timeout 8 msecs
> > 
> > Tests run: 43, Failures: 2, Errors: 0, Skipped: 0

I am getting this test failure, will investigate.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136791
---


On June 6, 2016, 10:53 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 6, 2016, 10:53 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-09 Thread Shwetha GS 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136791
---



The following tests failed for me with this patch. Did they work for you?
Failed tests:
  
HiveHookIT.testAlterTablePartitionColumnType:911->assertColumnIsRegistered:276->assertEntityIsRegistered:1498->waitFor:1591
 Assertions failed. Failing after waiting for timeout 8 msecs
  
HiveHookIT.testAlterTableRename:659->assertColumnIsRegistered:271->assertColumnIsRegistered:276->assertEntityIsRegistered:1498->waitFor:1591
 Assertions failed. Failing after waiting for timeout 8 msecs

Tests run: 43, Failures: 2, Errors: 0, Skipped: 0

- Shwetha GS


On June 6, 2016, 10:53 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 6, 2016, 10:53 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-09 Thread Shwetha GS 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136790
---



Looks good to me. Suma, do you have any other comments

- Shwetha GS


On June 6, 2016, 10:53 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 6, 2016, 10:53 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-06 Thread David Kantor 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/
---

(Updated June 6, 2016, 10:53 p.m.)


Review request for atlas and Shwetha GS.


Changes
---

Addressed Suma's comment about HashSet usage.


Bugs: ATLAS-499
https://issues.apache.org/jira/browse/ATLAS-499


Repository: atlas


Description
---

ATLAS-499: Automatically update inverse references to prevent repository 
corruption from unbalanced references.


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
91f9bd008a6939dfe78656f5c1845637a30ee9eb 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
4f6d0118072380342f5ea302d0aaf7902784849a 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 4c1f5591f4adead41c3336e64b0bc956836f7edb 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
 449e066a167ba4546b118a77c8e3de5fd99f077b 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47658/diff/


Testing
---

Ran all unit and integration tests with no regressions.


Thanks,

David Kantor

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-06 Thread David Kantor 


> On June 3, 2016, 11:25 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java,
> >  line 785
> > 
> >
> > do we need the HashSet since contains and all will both O(n)
> 
> David Kantor wrote:
> HashSet.contains is O(1).  I am using the HashSet to avoid the O(n) 
> performance of ArrayList.contains.  For a very small data set, the difference 
> between HashSet.contains and ArrayList.contains is probably negligible.  But 
> in my experience doing Java performance analysis on our legacy metadata 
> repository, ArrayList.contains() often ended up being a major hot spot when 
> enforcing uniqueness for high cardinality references.
> 
> Suma Shivaprasad wrote:
> I was just saying that adding all the list elemnts to the Hashset is 
> itself O(n) so it might be better to skip creating the hashset. Leave it up 
> to you not a major issue to fix.

Thanks very much for clarifying that, I agree and have addressed the comment.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136132
---


On June 6, 2016, 10:53 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 6, 2016, 10:53 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-06 Thread David Kantor 


> On June 3, 2016, 11:29 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java,
> >  line 443
> > 
> >
> > for map types, can we check if the reverseAttributeName is set on the 
> > map type and remove it if set else ignore?
> 
> David Kantor wrote:
> I'm thinking it may be better to remove the reverse attribute name when 
> the type is registered and possibly issue a warning that bi-directional 
> references are not supported for map types, rather then doing it every time 
> there is an update operation involving that type.  Thoughts?
> 
> Suma Shivaprasad wrote:
> yeah that seems like a better option

In thinking more about this, perhaps it is beyond the scope of this JIRA to 
make such a significant change to the type system functionality.  You could 
argue that having a bi-directional reference where one of the ends is a map 
reference is OK, it's just that in the specific case where the user just 
updates the non-map end of the reference, we don't know what to use for the map 
key when updating the reverse map reference so we can't support auto-updating 
in that use case.  It doesn't necessarily mean that the type system should 
ignore/remove the reverseAttributeName on a map reference at registration time. 
 Modelers may have a valid use case for bi-directional map references and if we 
suddenly change the type system to ignore or remove the reverseAttributeName, 
that could be a breaking change to existing applications.  Perhaps your point 
in the original comment is that if we don't support auto-updating when just the 
non-map reference is updated, we should be consistent and not suppor
 t automatic reverse reference updating if *either* end of the reference is a 
map.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136133
---


On June 3, 2016, 1:55 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 3, 2016, 1:55 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-06 Thread Suma Shivaprasad 


> On June 3, 2016, 11:25 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java,
> >  line 785
> > 
> >
> > do we need the HashSet since contains and all will both O(n)
> 
> David Kantor wrote:
> HashSet.contains is O(1).  I am using the HashSet to avoid the O(n) 
> performance of ArrayList.contains.  For a very small data set, the difference 
> between HashSet.contains and ArrayList.contains is probably negligible.  But 
> in my experience doing Java performance analysis on our legacy metadata 
> repository, ArrayList.contains() often ended up being a major hot spot when 
> enforcing uniqueness for high cardinality references.

I was just saying that adding all the list elemnts to the Hashset is itself 
O(n) so it might be better to skip creating the hashset. Leave it up to you not 
a major issue to fix.


- Suma


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136132
---


On June 3, 2016, 1:55 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 3, 2016, 1:55 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-06 Thread Suma Shivaprasad 


> On June 3, 2016, 11:29 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java,
> >  line 443
> > 
> >
> > for map types, can we check if the reverseAttributeName is set on the 
> > map type and remove it if set else ignore?
> 
> David Kantor wrote:
> I'm thinking it may be better to remove the reverse attribute name when 
> the type is registered and possibly issue a warning that bi-directional 
> references are not supported for map types, rather then doing it every time 
> there is an update operation involving that type.  Thoughts?

yeah that seems like a better option


- Suma


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136133
---


On June 3, 2016, 1:55 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 3, 2016, 1:55 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-06 Thread David Kantor 


> On June 3, 2016, 11:29 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java,
> >  line 443
> > 
> >
> > for map types, can we check if the reverseAttributeName is set on the 
> > map type and remove it if set else ignore?

I'm thinking it may be better to remove the reverse attribute name when the 
type is registered and possibly issue a warning that bi-directional references 
are not supported for map types, rather then doing it every time there is an 
update operation involving that type.  Thoughts?


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136133
---


On June 3, 2016, 1:55 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 3, 2016, 1:55 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-06 Thread David Kantor 


> On June 3, 2016, 11:25 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java,
> >  line 785
> > 
> >
> > do we need the HashSet since contains and all will both O(n)

HashSet.contains is O(1).  I am using the HashSet to avoid the O(n) performance 
of ArrayList.contains.  For a very small data set, the difference between 
HashSet.contains and ArrayList.contains is probably negligible.  But in my 
experience doing Java performance analysis on our legacy metadata repository, 
ArrayList.contains() often ended up being a major hot spot when enforcing 
uniqueness for high cardinality references.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136132
---


On June 3, 2016, 1:55 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 3, 2016, 1:55 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-03 Thread Suma Shivaprasad 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136133
---




repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 (line 441)


for map types, can we check if the reverseAttributeName is set on the map 
type and remove it if set else ignore?


- Suma Shivaprasad


On June 3, 2016, 1:55 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 3, 2016, 1:55 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-03 Thread Suma Shivaprasad 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/#review136132
---




repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 (line 783)


do we need the HashSet since contains and all will both O(n)


- Suma Shivaprasad


On June 3, 2016, 1:55 p.m., David Kantor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47658/
> ---
> 
> (Updated June 3, 2016, 1:55 p.m.)
> 
> 
> Review request for atlas and Shwetha GS.
> 
> 
> Bugs: ATLAS-499
> https://issues.apache.org/jira/browse/ATLAS-499
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-499: Automatically update inverse references to prevent repository 
> corruption from unbalanced references.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
> 91f9bd008a6939dfe78656f5c1845637a30ee9eb 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 4f6d0118072380342f5ea302d0aaf7902784849a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  4c1f5591f4adead41c3336e64b0bc956836f7edb 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  449e066a167ba4546b118a77c8e3de5fd99f077b 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47658/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit and integration tests with no regressions.
> 
> 
> Thanks,
> 
> David Kantor
> 
>

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-03 Thread David Kantor 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/
---

(Updated June 3, 2016, 1:55 p.m.)


Review request for atlas and Shwetha GS.


Changes
---

Addressed Suma's review comment.


Bugs: ATLAS-499
https://issues.apache.org/jira/browse/ATLAS-499


Repository: atlas


Description
---

ATLAS-499: Automatically update inverse references to prevent repository 
corruption from unbalanced references.


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
91f9bd008a6939dfe78656f5c1845637a30ee9eb 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
4f6d0118072380342f5ea302d0aaf7902784849a 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 4c1f5591f4adead41c3336e64b0bc956836f7edb 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
 449e066a167ba4546b118a77c8e3de5fd99f077b 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47658/diff/


Testing
---

Ran all unit and integration tests with no regressions.


Thanks,

David Kantor

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-06-02 Thread David Kantor 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/
---

(Updated June 2, 2016, 1:04 p.m.)


Review request for atlas.


Changes
---

Rebased changes with trunk, added tests


Bugs: ATLAS-499
https://issues.apache.org/jira/browse/ATLAS-499


Repository: atlas


Description
---

ATLAS-499: Automatically update inverse references to prevent repository 
corruption from unbalanced references.


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
91f9bd008a6939dfe78656f5c1845637a30ee9eb 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
4f6d0118072380342f5ea302d0aaf7902784849a 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 4c1f5591f4adead41c3336e64b0bc956836f7edb 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
 449e066a167ba4546b118a77c8e3de5fd99f077b 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47658/diff/


Testing
---

Ran all unit and integration tests with no regressions.


Thanks,

David Kantor

Re: Review Request 47658: ATLAS-499: Update reverse references

2016-05-23 Thread David Kantor 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47658/
---

(Updated May 23, 2016, 3:41 p.m.)


Review request for atlas.


Bugs: ATLAS-499
https://issues.apache.org/jira/browse/ATLAS-499


Repository: atlas


Description
---

ATLAS-499: Automatically update inverse references to prevent repository 
corruption from unbalanced references.


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 
a9e4f3944030a28f5f555f939957e254bac63c60 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
cccafc2040560e095f792719e70cd4924a3c6e6d 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 a0175363eb5d613c97fe0f83c04dac0d7333ff55 
  
repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47658/diff/


Testing
---

Ran all unit and integration tests with no regressions.


Thanks,

David Kantor