[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Attachment: NullPointerException_stackTrace.txt > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4847.patch, NullPointerException_stackTrace.txt > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[
https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4847:
--
Description:
Steps to reproduce:
1. Create a hive_table entity and assign a term to it.
2. Perform 1st incremental export of that entity without using changeMarker.
3. Without any modification on that entity perform 2nd consecutive incremental
export with changeMarker from previous export.
Problem is it throws NullPointerException and export is failed.
was:
Steps to reproduce:
1. Create a hive_table entity and assign a term to it.
2. Perform 1st incremental export of that entity without using changeMarker.
3. Without any modification on that entity perform 2nd consecutive incremental
export with changeMarker from previous export.
Problem is it throws NullPointerException and export is failed.
Application.log
2024-04-10 17:49:44,148 INFO - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ ==> export(user=admin, from=127.0.0.1)
(ExportService:95)
2024-04-10 17:49:44,219 INFO - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~
export(item=AtlasObjectId\{guid='null', typeName='hive_table',
uniqueAttributes={qualifiedName:test_db.abc__hive_impala_table_1@cm}};
matchType=null, fetchType=INCREMENTAL): found 1 entities: options:
\{"itemsToExport":[{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"test_db.abc__hive_impala_table_1@cm"}}],"options":\{"fetchType":"INCREMENTAL","changeMarker":1712751459762},"fetchTypeOptionValue":"INCREMENTAL","skipLineageOptionValue":false,"omitZipResponseForEmptyExport":false,"changeTokenFromOptions":1712751459762}
(StartEntityFetchByExportRequest:123)
2024-04-10 17:49:44,242 ERROR - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ Operation failed: (ExportService:103)
java.lang.{color:#de350b}NullPointerException{color}
at
org.apache.atlas.repository.impexp.ExportService$ExportContext.getEntitiesWithModifiedTimestamp(ExportService.java:445)
at
org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:332)
at
org.apache.atlas.repository.impexp.ExportService.addGlossaryEntities(ExportService.java:299)
at
org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:274)
> Export/Import : Atlas export fails and throws NullPointerException.
> ---
>
> Key: ATLAS-4847
> URL: https://issues.apache.org/jira/browse/ATLAS-4847
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Attachments: ATLAS-4847.patch, NullPointerException_stackTrace.txt
>
>
> Steps to reproduce:
> 1. Create a hive_table entity and assign a term to it.
> 2. Perform 1st incremental export of that entity without using changeMarker.
> 3. Without any modification on that entity perform 2nd consecutive
> incremental export with changeMarker from previous export.
> Problem is it throws NullPointerException and export is failed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[
https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4847:
--
Description:
Steps to reproduce:
1. Create a hive_table entity and assign a term to it.
2. Perform 1st incremental export of that entity without using changeMarker.
3. Without any modification on that entity perform 2nd consecutive incremental
export with changeMarker from previous export.
Problem is it throws NullPointerException and export is failed.
Application.log
2024-04-10 17:49:44,148 INFO - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ ==> export(user=admin, from=127.0.0.1)
(ExportService:95)
2024-04-10 17:49:44,219 INFO - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~
export(item=AtlasObjectId\{guid='null', typeName='hive_table',
uniqueAttributes={qualifiedName:test_db.abc__hive_impala_table_1@cm}};
matchType=null, fetchType=INCREMENTAL): found 1 entities: options:
\{"itemsToExport":[{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"test_db.abc__hive_impala_table_1@cm"}}],"options":\{"fetchType":"INCREMENTAL","changeMarker":1712751459762},"fetchTypeOptionValue":"INCREMENTAL","skipLineageOptionValue":false,"omitZipResponseForEmptyExport":false,"changeTokenFromOptions":1712751459762}
(StartEntityFetchByExportRequest:123)
2024-04-10 17:49:44,242 ERROR - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ Operation failed: (ExportService:103)
java.lang.{color:#de350b}NullPointerException{color}
at
org.apache.atlas.repository.impexp.ExportService$ExportContext.getEntitiesWithModifiedTimestamp(ExportService.java:445)
at
org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:332)
at
org.apache.atlas.repository.impexp.ExportService.addGlossaryEntities(ExportService.java:299)
at
org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:274)
was:
Steps to reproduce:
1. Create a hive_table entity and assign a term to it.
2. Perform 1st incremental export of that entity without using changeMarker.
3. Without any modification on that entity perform 2nd consecutive incremental
export with changeMarker from previous export.
Problem is it throws NullPointerException and export is failed.
> Export/Import : Atlas export fails and throws NullPointerException.
> ---
>
> Key: ATLAS-4847
> URL: https://issues.apache.org/jira/browse/ATLAS-4847
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Attachments: ATLAS-4847.patch
>
>
> Steps to reproduce:
> 1. Create a hive_table entity and assign a term to it.
> 2. Perform 1st incremental export of that entity without using changeMarker.
> 3. Without any modification on that entity perform 2nd consecutive
> incremental export with changeMarker from previous export.
> Problem is it throws NullPointerException and export is failed.
>
> Application.log
> 2024-04-10 17:49:44,148 INFO - [etp370869802-458 -
> 76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ ==> export(user=admin,
> from=127.0.0.1) (ExportService:95)
> 2024-04-10 17:49:44,219 INFO - [etp370869802-458 -
> 76b2eeb5-22fb-4631-a99a-2567c307152a:] ~
> export(item=AtlasObjectId\{guid='null', typeName='hive_table',
> uniqueAttributes={qualifiedName:test_db.abc__hive_impala_table_1@cm}};
> matchType=null, fetchType=INCREMENTAL): found 1 entities: options:
> \{"itemsToExport":[{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"test_db.abc__hive_impala_table_1@cm"}}],"options":\{"fetchType":"INCREMENTAL","changeMarker":1712751459762},"fetchTypeOptionValue":"INCREMENTAL","skipLineageOptionValue":false,"omitZipResponseForEmptyExport":false,"changeTokenFromOptions":1712751459762}
> (StartEntityFetchByExportRequest:123)
> 2024-04-10 17:49:44,242 ERROR - [etp370869802-458 -
> 76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ Operation failed:
> (ExportService:103)
> java.lang.{color:#de350b}NullPointerException{color}
> at
> org.apache.atlas.repository.impexp.ExportService$ExportContext.getEntitiesWithModifiedTimestamp(ExportService.java:445)
> at
> org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:332)
> at
> org.apache.atlas.repository.impexp.ExportService.addGlossaryEntities(ExportService.java:299)
> at
> org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:274)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Attachment: (was: image-2024-04-11-15-15-59-551.png) > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4847.patch > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Attachment: ATLAS-4847.patch > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4847.patch, image-2024-04-11-15-15-59-551.png > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836457#comment-17836457 ] ASF subversion and git services commented on ATLAS-4847: Commit ec44700a318d17f6fb78c588e0d3a4a4c0f8904b in atlas's branch refs/heads/branch-2.0 from priyanshi-shah26 [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=ec44700a3 ] ATLAS-4847 : Export/Import : Atlas export fails and throws NullPointerException Signed-off-by: Pinal Shah > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: image-2024-04-11-15-15-59-551.png > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836456#comment-17836456 ] ASF subversion and git services commented on ATLAS-4847: Commit 94083a3802dfe6f712e2c2d8cf777311ad5ca34b in atlas's branch refs/heads/master from priyanshi-shah26 [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=94083a380 ] ATLAS-4847 : Export/Import : Atlas export fails and throws NullPointerException Signed-off-by: Pinal Shah > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: image-2024-04-11-15-15-59-551.png > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74956: ATLAS-4847 : Export/Import : Atlas export fails and throws NullPointerException.
> On April 12, 2024, 7:38 a.m., Pinal Shah wrote: > > Ship It! Glossary Updates will be handled in other jira - Pinal --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74956/#review226388 --- On April 12, 2024, 7:22 a.m., Priyanshi Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74956/ > --- > > (Updated April 12, 2024, 7:22 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and > Sheetal Shah. > > > Bugs: ATLAS-4847 > https://issues.apache.org/jira/browse/ATLAS-4847 > > > Repository: atlas > > > Description > --- > > Steps to reproduce: > > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > > Result is it that it throws NullPointerException and export is failed. > > Problem: > > When an entity which has termed assigned was exported with changeMarker value > greater than value of lastModified time of that entity it resulted in > throwing NullPointerException. > > Solution: > > 1. NullPointerException occured because it was trying to fetch referred > entities of AtlasGlossary Object while performing export operation and as > AtlasGlossary didn't have any referred entities it gave raise this exception. > 2. This problem is solved by handling NullPointer before trying to fetch > referred entities. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java > 4615c6c2f > > > Diff: https://reviews.apache.org/r/74956/diff/1/ > > > Testing > --- > > Manual testing has been done for the same. > > PC link: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1608/ > > > Thanks, > > Priyanshi Shah > >
Re: Review Request 74956: ATLAS-4847 : Export/Import : Atlas export fails and throws NullPointerException.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74956/#review226388 --- Ship it! Ship It! - Pinal Shah On April 12, 2024, 7:22 a.m., Priyanshi Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74956/ > --- > > (Updated April 12, 2024, 7:22 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and > Sheetal Shah. > > > Bugs: ATLAS-4847 > https://issues.apache.org/jira/browse/ATLAS-4847 > > > Repository: atlas > > > Description > --- > > Steps to reproduce: > > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > > Result is it that it throws NullPointerException and export is failed. > > Problem: > > When an entity which has termed assigned was exported with changeMarker value > greater than value of lastModified time of that entity it resulted in > throwing NullPointerException. > > Solution: > > 1. NullPointerException occured because it was trying to fetch referred > entities of AtlasGlossary Object while performing export operation and as > AtlasGlossary didn't have any referred entities it gave raise this exception. > 2. This problem is solved by handling NullPointer before trying to fetch > referred entities. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java > 4615c6c2f > > > Diff: https://reviews.apache.org/r/74956/diff/1/ > > > Testing > --- > > Manual testing has been done for the same. > > PC link: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1608/ > > > Thanks, > > Priyanshi Shah > >
Re: Review Request 74956: ATLAS-4847 : Export/Import : Atlas export fails and throws NullPointerException.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74956/ --- (Updated April 12, 2024, 7:22 a.m.) Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and Sheetal Shah. Bugs: ATLAS-4847 https://issues.apache.org/jira/browse/ATLAS-4847 Repository: atlas Description --- Steps to reproduce: 1. Create a hive_table entity and assign a term to it. 2. Perform 1st incremental export of that entity without using changeMarker. 3. Without any modification on that entity perform 2nd consecutive incremental export with changeMarker from previous export. Result is it that it throws NullPointerException and export is failed. Problem: When an entity which has termed assigned was exported with changeMarker value greater than value of lastModified time of that entity it resulted in throwing NullPointerException. Solution: 1. NullPointerException occured because it was trying to fetch referred entities of AtlasGlossary Object while performing export operation and as AtlasGlossary didn't have any referred entities it gave raise this exception. 2. This problem is solved by handling NullPointer before trying to fetch referred entities. Diffs - repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 4615c6c2f Diff: https://reviews.apache.org/r/74956/diff/1/ Testing (updated) --- Manual testing has been done for the same. PC link: https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1608/ Thanks, Priyanshi Shah
[jira] [Created] (ATLAS-4848) Atlas 'updateTime' parameter is not updated when term is added.
Priyanshi Shah created ATLAS-4848: - Summary: Atlas 'updateTime' parameter is not updated when term is added. Key: ATLAS-4848 URL: https://issues.apache.org/jira/browse/ATLAS-4848 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Priyanshi Shah Assignee: Priyanshi Shah -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74954: ATLAS-4846 : Source the atlas-env file in ATLAS import script to get all the argument
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74954/ --- (Updated April 12, 2024, 6:01 a.m.) Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and Sheetal Shah. Bugs: ATLAS-4846 https://issues.apache.org/jira/browse/ATLAS-4846 Repository: atlas Description (updated) --- User has to run manually export command to get all the argument which are set, to run the Atlas import script. The problem is not with the Atlas service itself but with these scripts: /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-kafka.sh /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hbase.sh /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-impala.sh Need to set argument before this script are being called. Diffs - addons/hbase-bridge/src/bin/import-hbase.sh 8ae735034 addons/hive-bridge/src/bin/import-hive.sh 951be5ba2 addons/impala-bridge/src/main/resources/import-impala.sh b440f2d05 addons/kafka-bridge/src/bin/import-kafka.sh 8fe27748a Diff: https://reviews.apache.org/r/74954/diff/3/ Testing --- Testing:- Manually create the cluster and ran the import script. Thanks, Paresh Devalia
