----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/75223/#review227163 -----------------------------------------------------------
repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationStatusService.java Line 62 (original), 62 (patched) <https://reviews.apache.org/r/75223/#comment315422> Consider evaluating the MD5 only once and reusing it on line no.57 and 62 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/MigrationImport.java Lines 86 (patched) <https://reviews.apache.org/r/75223/#comment315423> Is this change required? - Jayendra Parab On Oct. 8, 2024, 5:38 a.m., chaitali wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/75223/ > ----------------------------------------------------------- > > (Updated Oct. 8, 2024, 5:38 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, and Pinal Shah. > > > Bugs: ATLAS-4907 > https://issues.apache.org/jira/browse/ATLAS-4907 > > > Repository: atlas > > > Description > ------- > > Whenever Migration starts and vertex is created for the respective file in > progress > > whenever Migration status is attempted to update it fails with nullpointer > exception > > Due to this because the position was not updated in db hence when migration > was interrupted and restarted it would start from position 0 > > 2024-09-18 10:29:04,153 WARN - [zipFileBasedMigrationImporter:] > ~ Error updating status. Please rely on log messages. > (DataMigrationStatusService$MigrationStatusVertexManagement:202) > java.lang.NullPointerException > at > org.apache.atlas.repository.store.graph.v2.AtlasGraphUtilsV2.setProperty(AtlasGraphUtilsV2.java:250) > at > org.apache.atlas.repository.store.graph.v2.AtlasGraphUtilsV2.setEncodedProperty(AtlasGraphUtilsV2.java:238) > at > org.apache.atlas.repository.migration.DataMigrationStatusService$MigrationStatusVertexManagement.updateVertexPartialStatus(DataMigrationStatusService.java:200) > at > org.apache.atlas.repository.migration.DataMigrationStatusService.setStatus(DataMigrationStatusService.java:119) > at > org.apache.atlas.repository.store.graph.v2.bulkimport.pc.EntityCreationManager.read(EntityCreationManager.java:57) > at > org.apache.atlas.repository.store.graph.v2.bulkimport.MigrationImport.run(MigrationImport.java:79) > at > org.apache.atlas.repository.store.graph.v2.BulkImporterImpl.bulkImport(BulkImporterImpl.java:77) > at > org.apache.atlas.repository.impexp.ImportService.processEntities(ImportService.java:232) > at > org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:119) > at > org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:96) > at > org.apache.atlas.repository.migration.ZipFileMigrationImporter.performImport(ZipFileMigrationImporter.java:255) > at > org.apache.atlas.repository.migration.ZipFileMigrationImporter.run(ZipFileMigrationImporter.java:97) > at java.lang.Thread.run(Thread.java:748) > > With this patch adds file hash instead of filename whenever it will try to > update vertex with status and position it will use file hash > > > Diffs > ----- > > > repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationStatusService.java > 5b22f9cd5 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/MigrationImport.java > d34d9b658 > > > Diff: https://reviews.apache.org/r/75223/diff/2/ > > > Testing > ------- > > Before patch even after migration was completed position was never updated > due to above nullpointer exception > gremlin> g.V().has("__guid","b4fa0dcf2b0acac65e75dfda4f6dedd9").valueMap() > ==>[__guid:[b4fa0dcf2b0acac65e75dfda4f6dedd9],__migration.startTime:[1726486589197],__migration.size:[1],__migration.position:[0],__migration.status:[DONE]] > > 2024-10-02 14:16:53,204 INFO - [zipFileBasedMigrationImporter:] ~ Migration > Import: Size: 0: Done! (MigrationImport:87) > > > After applying patch when interrupted in between it fetches last position of > entities processed: > ==>[__guid:[b4fa0dcf2b0acac65e75dfda4f6dedd9],__migration.startTime:[1727887269568],__migration.size:[1],__migration.position:[6],__migration.status:[IN_PROGRESS]] > > > After completing migration: > gremlin> g.V().has("__guid","b4fa0dcf2b0acac65e75dfda4f6dedd9").valueMap() > ==>[__guid:[b4fa0dcf2b0acac65e75dfda4f6dedd9],__migration.startTime:[1727950058639],__migration.size:[1],__migration.position:[3553],__migration.status:[DONE]] > > It fetches correct position > 2024-10-03 15:44:08,163 INFO - [zipFileBasedMigrationImporter:] ~ Migration > Import: Size: 3555: Done! (MigrationImport:88) > > > Thanks, > > chaitali > >
