[jira] [Resolved] (ATLAS-4854) Atlas - Upgrade Spring Security to 5.8.11
[ https://issues.apache.org/jira/browse/ATLAS-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah resolved ATLAS-4854. --- Resolution: Fixed > Atlas - Upgrade Spring Security to 5.8.11 > - > > Key: ATLAS-4854 > URL: https://issues.apache.org/jira/browse/ATLAS-4854 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Upgrading spring security from 5.8.5 to 5.8.11 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4854) Atlas - Upgrade Spring Security to 5.8.11
[ https://issues.apache.org/jira/browse/ATLAS-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4854: -- Fix Version/s: 2.3.0 > Atlas - Upgrade Spring Security to 5.8.11 > - > > Key: ATLAS-4854 > URL: https://issues.apache.org/jira/browse/ATLAS-4854 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Upgrading spring security from 5.8.5 to 5.8.11 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ATLAS-4844) Atlas - Upgrade Common Configuration2 to 2.10.1
[ https://issues.apache.org/jira/browse/ATLAS-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah resolved ATLAS-4844. --- Fix Version/s: 3.0.0 2.3.0 Resolution: Fixed > Atlas - Upgrade Common Configuration2 to 2.10.1 > --- > > Key: ATLAS-4844 > URL: https://issues.apache.org/jira/browse/ATLAS-4844 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 3.0.0 >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Upgrading common configurations2 from 2.8.0 to 2.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4892) Export/Import APIs : Unable to import shell entities
[
https://issues.apache.org/jira/browse/ATLAS-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4892:
--
Fix Version/s: 3.0.0
2.3.0
> Export/Import APIs : Unable to import shell entities
>
>
> Key: ATLAS-4892
> URL: https://issues.apache.org/jira/browse/ATLAS-4892
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Exporting shell entities causes failure at Import.
> For Hive tables created using spark, there is a chance of shell entities that
> can be created in Atlas.
> Before the entities could get resolved , if export zip has those entities,
> import fails with
> {code:java}
> 2024-04-29 17:12:16,981 ERROR - [etp640294829-283 -
> 84da4eb4-f1b8-4723-953c-6b86bcf60647:] ~ import(user=hrt_qa,
> from=10.19.24.189): failed (ImportService:124)
> org.apache.atlas.exception.AtlasBaseException: Invalid instance
> creation/updation parameters passed : hive_table.name: mandatory attribute
> value missing in type Asset {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4892) Export/Import APIs : Unable to import shell entities
[
https://issues.apache.org/jira/browse/ATLAS-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4892:
--
Affects Version/s: (was: 3.0.0)
(was: 2.3.0)
> Export/Import APIs : Unable to import shell entities
>
>
> Key: ATLAS-4892
> URL: https://issues.apache.org/jira/browse/ATLAS-4892
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
>
> Exporting shell entities causes failure at Import.
> For Hive tables created using spark, there is a chance of shell entities that
> can be created in Atlas.
> Before the entities could get resolved , if export zip has those entities,
> import fails with
> {code:java}
> 2024-04-29 17:12:16,981 ERROR - [etp640294829-283 -
> 84da4eb4-f1b8-4723-953c-6b86bcf60647:] ~ import(user=hrt_qa,
> from=10.19.24.189): failed (ImportService:124)
> org.apache.atlas.exception.AtlasBaseException: Invalid instance
> creation/updation parameters passed : hive_table.name: mandatory attribute
> value missing in type Asset {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4848) Atlas 'updateTime' parameter is not updated when term is added.
[ https://issues.apache.org/jira/browse/ATLAS-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4848: -- Description: *Steps to reproduce:* 1. Create hive_table testhive1 and perform incremental export for the same with changeMarker 0. 2. Now, before 2nd run create and assign term t1 to testhive1 and perform incremental export with changeMarker that we got after 1st run. 3. Result is export works but the term assigned to the hive_table entity didn't got exported. *Problem:* When term was getting assigned to hive_table entity, it did not updated the value of 'updateTime' attribute of that entity and hence,during 2nd run with incremental export it didn't exported the term as for incremental export we fetch data according to the changeMarker provided. *Fix:* Attribute 'updateTime' value gets updated after the term is assigned to an entity. > Atlas 'updateTime' parameter is not updated when term is added. > --- > > Key: ATLAS-4848 > URL: https://issues.apache.org/jira/browse/ATLAS-4848 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > > *Steps to reproduce:* > 1. Create hive_table testhive1 and perform incremental export for the same > with changeMarker 0. > 2. Now, before 2nd run create and assign term t1 to testhive1 and perform > incremental export with changeMarker that we got after 1st run. > 3. Result is export works but the term assigned to the hive_table entity > didn't got exported. > *Problem:* > When term was getting assigned to hive_table entity, it did not updated the > value of 'updateTime' attribute of that entity and hence,during 2nd run with > incremental export it didn't exported the term as for incremental export we > fetch data according to the changeMarker provided. > *Fix:* > Attribute 'updateTime' value gets updated after the term is assigned to an > entity. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4892) Export/Import APIs : Unable to import shell entities
[
https://issues.apache.org/jira/browse/ATLAS-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4892:
--
Description:
Exporting shell entities causes failure at Import.
For Hive tables created using spark, there is a chance of shell entities that
can be created in Atlas.
Before the entities could get resolved , if export zip has those entities,
import fails with
{code:java}
2024-04-29 17:12:16,981 ERROR - [etp640294829-283 -
84da4eb4-f1b8-4723-953c-6b86bcf60647:] ~ import(user=hrt_qa,
from=10.19.24.189): failed (ImportService:124)
org.apache.atlas.exception.AtlasBaseException: Invalid instance
creation/updation parameters passed : hive_table.name: mandatory attribute
value missing in type Asset {code}
> Export/Import APIs : Unable to import shell entities
>
>
> Key: ATLAS-4892
> URL: https://issues.apache.org/jira/browse/ATLAS-4892
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
>
> Exporting shell entities causes failure at Import.
> For Hive tables created using spark, there is a chance of shell entities that
> can be created in Atlas.
> Before the entities could get resolved , if export zip has those entities,
> import fails with
> {code:java}
> 2024-04-29 17:12:16,981 ERROR - [etp640294829-283 -
> 84da4eb4-f1b8-4723-953c-6b86bcf60647:] ~ import(user=hrt_qa,
> from=10.19.24.189): failed (ImportService:124)
> org.apache.atlas.exception.AtlasBaseException: Invalid instance
> creation/updation parameters passed : hive_table.name: mandatory attribute
> value missing in type Asset {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ATLAS-4892) Export/Import APIs : Unable to import shell entities
Priyanshi Shah created ATLAS-4892: - Summary: Export/Import APIs : Unable to import shell entities Key: ATLAS-4892 URL: https://issues.apache.org/jira/browse/ATLAS-4892 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.3.0, 3.0.0 Reporter: Priyanshi Shah Assignee: Priyanshi Shah -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4859) NPE during Atlas Import
[
https://issues.apache.org/jira/browse/ATLAS-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4859:
--
Attachment: (was: ATLAS-4859.patch)
> NPE during Atlas Import
> ---
>
> Key: ATLAS-4859
> URL: https://issues.apache.org/jira/browse/ATLAS-4859
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> While doing an incremental import with multiple classifications associated
> with entities it was giving NPE and import was getting failed.
> {code:java}
> 2024-07-23 15:40:31,244 ERROR - [etp370869802-217 -
> cd17f4b8-d164-4bd6-9458-97d3585ae093:] ~ import(user=admin, from=127.0.0.1):
> failed (ImportService:125)
> org.apache.atlas.exception.AtlasBaseException: java.lang.NullPointerException
> at
> org.apache.atlas.repository.store.graph.v2.bulkimport.RegularImport.run(RegularImport.java:137)
> at
> org.apache.atlas.repository.store.graph.v2.BulkImporterImpl.bulkImport(BulkImporterImpl.java:77)
> at
> org.apache.atlas.repository.impexp.ImportService.processEntities(ImportService.java:233)
> at
> org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:120)
> at
> org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:97)
> at
> org.apache.atlas.web.resources.AdminResource.importData(AdminResource.java:698)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4859) NPE during Atlas Import
[
https://issues.apache.org/jira/browse/ATLAS-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4859:
--
Attachment: ATLAS-4859.patch
> NPE during Atlas Import
> ---
>
> Key: ATLAS-4859
> URL: https://issues.apache.org/jira/browse/ATLAS-4859
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4859.patch
>
>
> While doing an incremental import with multiple classifications associated
> with entities it was giving NPE and import was getting failed.
> {code:java}
> 2024-07-23 15:40:31,244 ERROR - [etp370869802-217 -
> cd17f4b8-d164-4bd6-9458-97d3585ae093:] ~ import(user=admin, from=127.0.0.1):
> failed (ImportService:125)
> org.apache.atlas.exception.AtlasBaseException: java.lang.NullPointerException
> at
> org.apache.atlas.repository.store.graph.v2.bulkimport.RegularImport.run(RegularImport.java:137)
> at
> org.apache.atlas.repository.store.graph.v2.BulkImporterImpl.bulkImport(BulkImporterImpl.java:77)
> at
> org.apache.atlas.repository.impexp.ImportService.processEntities(ImportService.java:233)
> at
> org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:120)
> at
> org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:97)
> at
> org.apache.atlas.web.resources.AdminResource.importData(AdminResource.java:698)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4859) NPE during Atlas Import
[
https://issues.apache.org/jira/browse/ATLAS-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4859:
--
Attachment: ATLAS-4859.patch
> NPE during Atlas Import
> ---
>
> Key: ATLAS-4859
> URL: https://issues.apache.org/jira/browse/ATLAS-4859
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4859.patch
>
>
> While doing an incremental import with multiple classifications associated
> with entities it was giving NPE and import was getting failed.
> {code:java}
> 2024-07-23 15:40:31,244 ERROR - [etp370869802-217 -
> cd17f4b8-d164-4bd6-9458-97d3585ae093:] ~ import(user=admin, from=127.0.0.1):
> failed (ImportService:125)
> org.apache.atlas.exception.AtlasBaseException: java.lang.NullPointerException
> at
> org.apache.atlas.repository.store.graph.v2.bulkimport.RegularImport.run(RegularImport.java:137)
> at
> org.apache.atlas.repository.store.graph.v2.BulkImporterImpl.bulkImport(BulkImporterImpl.java:77)
> at
> org.apache.atlas.repository.impexp.ImportService.processEntities(ImportService.java:233)
> at
> org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:120)
> at
> org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:97)
> at
> org.apache.atlas.web.resources.AdminResource.importData(AdminResource.java:698)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4859) NPE during Atlas Import
[
https://issues.apache.org/jira/browse/ATLAS-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4859:
--
Description:
While doing an incremental import with multiple classifications associated with
entities it was giving NPE and import was getting failed.
{code:java}
2024-07-23 15:40:31,244 ERROR - [etp370869802-217 -
cd17f4b8-d164-4bd6-9458-97d3585ae093:] ~ import(user=admin, from=127.0.0.1):
failed (ImportService:125)
org.apache.atlas.exception.AtlasBaseException: java.lang.NullPointerException
at
org.apache.atlas.repository.store.graph.v2.bulkimport.RegularImport.run(RegularImport.java:137)
at
org.apache.atlas.repository.store.graph.v2.BulkImporterImpl.bulkImport(BulkImporterImpl.java:77)
at
org.apache.atlas.repository.impexp.ImportService.processEntities(ImportService.java:233)
at
org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:120)
at
org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:97)
at
org.apache.atlas.web.resources.AdminResource.importData(AdminResource.java:698)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
{code}
> NPE during Atlas Import
> ---
>
> Key: ATLAS-4859
> URL: https://issues.apache.org/jira/browse/ATLAS-4859
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> While doing an incremental import with multiple classifications associated
> with entities it was giving NPE and import was getting failed.
> {code:java}
> 2024-07-23 15:40:31,244 ERROR - [etp370869802-217 -
> cd17f4b8-d164-4bd6-9458-97d3585ae093:] ~ import(user=admin, from=127.0.0.1):
> failed (ImportService:125)
> org.apache.atlas.exception.AtlasBaseException: java.lang.NullPointerException
> at
> org.apache.atlas.repository.store.graph.v2.bulkimport.RegularImport.run(RegularImport.java:137)
> at
> org.apache.atlas.repository.store.graph.v2.BulkImporterImpl.bulkImport(BulkImporterImpl.java:77)
> at
> org.apache.atlas.repository.impexp.ImportService.processEntities(ImportService.java:233)
> at
> org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:120)
> at
> org.apache.atlas.repository.impexp.ImportService.run(ImportService.java:97)
> at
> org.apache.atlas.web.resources.AdminResource.importData(AdminResource.java:698)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ATLAS-4859) NPE during Atlas Import
Priyanshi Shah created ATLAS-4859: - Summary: NPE during Atlas Import Key: ATLAS-4859 URL: https://issues.apache.org/jira/browse/ATLAS-4859 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Priyanshi Shah Assignee: Priyanshi Shah Fix For: 3.0.0, 2.3.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4856) Parallel Atlas Import is failing with unexpected errors
[ https://issues.apache.org/jira/browse/ATLAS-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4856: -- Description: PermanentLockingException was thrown while running parallel imports. > Parallel Atlas Import is failing with unexpected errors > --- > > Key: ATLAS-4856 > URL: https://issues.apache.org/jira/browse/ATLAS-4856 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > PermanentLockingException was thrown while running parallel imports. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ATLAS-4856) Parallel Atlas Import is failing with unexpected errors
[ https://issues.apache.org/jira/browse/ATLAS-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah reassigned ATLAS-4856: - Assignee: Priyanshi Shah > Parallel Atlas Import is failing with unexpected errors > --- > > Key: ATLAS-4856 > URL: https://issues.apache.org/jira/browse/ATLAS-4856 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ATLAS-4856) Parallel Atlas Import is failing with unexpected errors
Priyanshi Shah created ATLAS-4856: - Summary: Parallel Atlas Import is failing with unexpected errors Key: ATLAS-4856 URL: https://issues.apache.org/jira/browse/ATLAS-4856 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Priyanshi Shah Fix For: 3.0.0, 2.3.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4854) Atlas - Upgrade Spring Security to 5.8.11
[ https://issues.apache.org/jira/browse/ATLAS-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4854: -- Description: Upgrading spring security from 5.8.5 to 5.8.11 > Atlas - Upgrade Spring Security to 5.8.11 > - > > Key: ATLAS-4854 > URL: https://issues.apache.org/jira/browse/ATLAS-4854 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0 > > > Upgrading spring security from 5.8.5 to 5.8.11 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ATLAS-4854) Atlas - Upgrade Spring Security to 5.8.11
Priyanshi Shah created ATLAS-4854: - Summary: Atlas - Upgrade Spring Security to 5.8.11 Key: ATLAS-4854 URL: https://issues.apache.org/jira/browse/ATLAS-4854 Project: Atlas Issue Type: Improvement Components: atlas-core Reporter: Priyanshi Shah Fix For: 3.0.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ATLAS-4854) Atlas - Upgrade Spring Security to 5.8.11
[ https://issues.apache.org/jira/browse/ATLAS-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah reassigned ATLAS-4854: - Assignee: Priyanshi Shah > Atlas - Upgrade Spring Security to 5.8.11 > - > > Key: ATLAS-4854 > URL: https://issues.apache.org/jira/browse/ATLAS-4854 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4844) Atlas - Upgrade Common Configuration2 to 2.10.1
[ https://issues.apache.org/jira/browse/ATLAS-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4844: -- Due Date: (was: 2/Apr/24) > Atlas - Upgrade Common Configuration2 to 2.10.1 > --- > > Key: ATLAS-4844 > URL: https://issues.apache.org/jira/browse/ATLAS-4844 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 3.0.0 >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > > Upgrading common configurations2 from 2.8.0 to 2.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Fix Version/s: 3.0.0 2.3.0 > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: ATLAS-4847.patch, NullPointerException_stackTrace.txt > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4842) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4842: -- Fix Version/s: 3.0.0 2.3.0 > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4842 > URL: https://issues.apache.org/jira/browse/ATLAS-4842 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sheetal Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: ATLAS-4842.patch > > > Create two hive_tables in same database: > # create table hive1(id int); > # create table hive1_ctas as select * from hive1; > # create table hive2(name string); > Perform incremental export of hive1 with changeMarker as 0 and skipLineage > value as false. > Import the zip. > Expectation is only the requested hive_table hive1 and its related entities > should be exported, but it does full export and hive2 is also imported. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4845) Atlas Import is failing with fetchType: Incremental if there are no changes between two consecutive runs
[ https://issues.apache.org/jira/browse/ATLAS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4845: -- Fix Version/s: 3.0.0 2.3.0 > Atlas Import is failing with fetchType: Incremental if there are no changes > between two consecutive runs > > > Key: ATLAS-4845 > URL: https://issues.apache.org/jira/browse/ATLAS-4845 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: ATLAS-4845.patch > > > Repro steps: > # First export + import with changeMarker “0” - worked as expected. > # I tried to run 2nd export + import without any changes in atlas, 2nd > export also worked and gave different changeMarker in output as expected. > *But 2nd import failed* > In other fetch Types this will work. So we also require similar behaviour in > case of Incremental type also because this is very normal scenario and > customer will hit this very frequently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4838) Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
[
https://issues.apache.org/jira/browse/ATLAS-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4838:
--
Fix Version/s: 3.0.0
2.3.0
> Export/Import : changeMarker is not set to entity's lastupdatetime or its
> closer timestamp value
>
>
> Key: ATLAS-4838
> URL: https://issues.apache.org/jira/browse/ATLAS-4838
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Sharmadha S
>Assignee: Priyanshi Shah
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4838-V2.patch
>
>
> When an hive table entity is exported using fetchType incremental with
> changeMarker 0 , after exporting , the changeMarker in the export response is
> not set to recent timestamp.
> For example , on exporting iceberg table with fetchType incremental and
> changeMarker as 0 :
> {code:java}
> 2024-02-08 13:28:43,838 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~
> export(item=AtlasObjectId{guid='null', typeName='iceberg_table',
> uniqueAttributes={qualifiedName:default.iceberg_table2@cm}}; matchType=null,
> fetchType=incremental): found 1 entities: options:
> {"itemsToExport":[{"typeName":"iceberg_table","uniqueAttributes":{"qualifiedName":"default.iceberg_table2@cm"}}],"options":{"fetchType":"incremental","changeMarker":0},"fetchTypeOptionValue":"incremental","skipLineageOptionValue":false,"changeTokenFromOptions":0}
> (StartEntityFetchByExportRequest:123)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ Assigned guid =
> f5e6342a-dfdf-4d54-a1b3-b24f987c4ecb to newly created entity (DataAccess:85)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ addAuditEntry: user: admin, source:
> cm, target: , operation: EXPORT (ExportImportAuditService:168)
> 2024-02-08 13:28:48,699 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ <== export(user=admin,
> from=10.19.28.124): status SUCCESS: changeMarker: 1707225183928
> (ExportService:107) {code}
> changeMarker from response is 1707225183928 => Tuesday, February 6, 2024
> 6:43:03.928 PM
>
> But lastUpdatetime of iceberg table is
> {code:java}
> "createTime": 1707398866970 => Thursday, February 8, 2024 6:57:46.970 PM
> "updateTime": 1707398867375. => Thursday, February 8, 2024 6:57:47.375 PM
> {code}
> Value is set to 2 days back. This is not then an incremental export , as it
> will fetch all updates from Tuesday, February 6, 2024 6:43:03.928 PM in next
> export
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Attachment: NullPointerException_stackTrace.txt > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4847.patch, NullPointerException_stackTrace.txt > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[
https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4847:
--
Description:
Steps to reproduce:
1. Create a hive_table entity and assign a term to it.
2. Perform 1st incremental export of that entity without using changeMarker.
3. Without any modification on that entity perform 2nd consecutive incremental
export with changeMarker from previous export.
Problem is it throws NullPointerException and export is failed.
was:
Steps to reproduce:
1. Create a hive_table entity and assign a term to it.
2. Perform 1st incremental export of that entity without using changeMarker.
3. Without any modification on that entity perform 2nd consecutive incremental
export with changeMarker from previous export.
Problem is it throws NullPointerException and export is failed.
Application.log
2024-04-10 17:49:44,148 INFO - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ ==> export(user=admin, from=127.0.0.1)
(ExportService:95)
2024-04-10 17:49:44,219 INFO - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~
export(item=AtlasObjectId\{guid='null', typeName='hive_table',
uniqueAttributes={qualifiedName:test_db.abc__hive_impala_table_1@cm}};
matchType=null, fetchType=INCREMENTAL): found 1 entities: options:
\{"itemsToExport":[{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"test_db.abc__hive_impala_table_1@cm"}}],"options":\{"fetchType":"INCREMENTAL","changeMarker":1712751459762},"fetchTypeOptionValue":"INCREMENTAL","skipLineageOptionValue":false,"omitZipResponseForEmptyExport":false,"changeTokenFromOptions":1712751459762}
(StartEntityFetchByExportRequest:123)
2024-04-10 17:49:44,242 ERROR - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ Operation failed: (ExportService:103)
java.lang.{color:#de350b}NullPointerException{color}
at
org.apache.atlas.repository.impexp.ExportService$ExportContext.getEntitiesWithModifiedTimestamp(ExportService.java:445)
at
org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:332)
at
org.apache.atlas.repository.impexp.ExportService.addGlossaryEntities(ExportService.java:299)
at
org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:274)
> Export/Import : Atlas export fails and throws NullPointerException.
> ---
>
> Key: ATLAS-4847
> URL: https://issues.apache.org/jira/browse/ATLAS-4847
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Attachments: ATLAS-4847.patch, NullPointerException_stackTrace.txt
>
>
> Steps to reproduce:
> 1. Create a hive_table entity and assign a term to it.
> 2. Perform 1st incremental export of that entity without using changeMarker.
> 3. Without any modification on that entity perform 2nd consecutive
> incremental export with changeMarker from previous export.
> Problem is it throws NullPointerException and export is failed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[
https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4847:
--
Description:
Steps to reproduce:
1. Create a hive_table entity and assign a term to it.
2. Perform 1st incremental export of that entity without using changeMarker.
3. Without any modification on that entity perform 2nd consecutive incremental
export with changeMarker from previous export.
Problem is it throws NullPointerException and export is failed.
Application.log
2024-04-10 17:49:44,148 INFO - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ ==> export(user=admin, from=127.0.0.1)
(ExportService:95)
2024-04-10 17:49:44,219 INFO - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~
export(item=AtlasObjectId\{guid='null', typeName='hive_table',
uniqueAttributes={qualifiedName:test_db.abc__hive_impala_table_1@cm}};
matchType=null, fetchType=INCREMENTAL): found 1 entities: options:
\{"itemsToExport":[{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"test_db.abc__hive_impala_table_1@cm"}}],"options":\{"fetchType":"INCREMENTAL","changeMarker":1712751459762},"fetchTypeOptionValue":"INCREMENTAL","skipLineageOptionValue":false,"omitZipResponseForEmptyExport":false,"changeTokenFromOptions":1712751459762}
(StartEntityFetchByExportRequest:123)
2024-04-10 17:49:44,242 ERROR - [etp370869802-458 -
76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ Operation failed: (ExportService:103)
java.lang.{color:#de350b}NullPointerException{color}
at
org.apache.atlas.repository.impexp.ExportService$ExportContext.getEntitiesWithModifiedTimestamp(ExportService.java:445)
at
org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:332)
at
org.apache.atlas.repository.impexp.ExportService.addGlossaryEntities(ExportService.java:299)
at
org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:274)
was:
Steps to reproduce:
1. Create a hive_table entity and assign a term to it.
2. Perform 1st incremental export of that entity without using changeMarker.
3. Without any modification on that entity perform 2nd consecutive incremental
export with changeMarker from previous export.
Problem is it throws NullPointerException and export is failed.
> Export/Import : Atlas export fails and throws NullPointerException.
> ---
>
> Key: ATLAS-4847
> URL: https://issues.apache.org/jira/browse/ATLAS-4847
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Priyanshi Shah
>Assignee: Priyanshi Shah
>Priority: Major
> Attachments: ATLAS-4847.patch
>
>
> Steps to reproduce:
> 1. Create a hive_table entity and assign a term to it.
> 2. Perform 1st incremental export of that entity without using changeMarker.
> 3. Without any modification on that entity perform 2nd consecutive
> incremental export with changeMarker from previous export.
> Problem is it throws NullPointerException and export is failed.
>
> Application.log
> 2024-04-10 17:49:44,148 INFO - [etp370869802-458 -
> 76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ ==> export(user=admin,
> from=127.0.0.1) (ExportService:95)
> 2024-04-10 17:49:44,219 INFO - [etp370869802-458 -
> 76b2eeb5-22fb-4631-a99a-2567c307152a:] ~
> export(item=AtlasObjectId\{guid='null', typeName='hive_table',
> uniqueAttributes={qualifiedName:test_db.abc__hive_impala_table_1@cm}};
> matchType=null, fetchType=INCREMENTAL): found 1 entities: options:
> \{"itemsToExport":[{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"test_db.abc__hive_impala_table_1@cm"}}],"options":\{"fetchType":"INCREMENTAL","changeMarker":1712751459762},"fetchTypeOptionValue":"INCREMENTAL","skipLineageOptionValue":false,"omitZipResponseForEmptyExport":false,"changeTokenFromOptions":1712751459762}
> (StartEntityFetchByExportRequest:123)
> 2024-04-10 17:49:44,242 ERROR - [etp370869802-458 -
> 76b2eeb5-22fb-4631-a99a-2567c307152a:] ~ Operation failed:
> (ExportService:103)
> java.lang.{color:#de350b}NullPointerException{color}
> at
> org.apache.atlas.repository.impexp.ExportService$ExportContext.getEntitiesWithModifiedTimestamp(ExportService.java:445)
> at
> org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:332)
> at
> org.apache.atlas.repository.impexp.ExportService.addGlossaryEntities(ExportService.java:299)
> at
> org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:274)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Attachment: (was: image-2024-04-11-15-15-59-551.png) > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4847.patch > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Attachment: ATLAS-4847.patch > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4847.patch, image-2024-04-11-15-15-59-551.png > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ATLAS-4848) Atlas 'updateTime' parameter is not updated when term is added.
Priyanshi Shah created ATLAS-4848: - Summary: Atlas 'updateTime' parameter is not updated when term is added. Key: ATLAS-4848 URL: https://issues.apache.org/jira/browse/ATLAS-4848 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Priyanshi Shah Assignee: Priyanshi Shah -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Description: Steps to reproduce: 1. Create a hive_table entity and assign a term to it. 2. Perform 1st incremental export of that entity without using changeMarker. 3. Without any modification on that entity perform 2nd consecutive incremental export with changeMarker from previous export. Problem is it throws NullPointerException and export is failed. was: Steps to reproduce: 1. Create a hive_table entity and assign a term to it. 2. Perform 1st incremental export of that entity without using changeMarker. 3. Without any modification on that entity perform 2nd consecutive incremental export with changeMarker from previous export. > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: image-2024-04-11-15-15-59-551.png > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Description: Steps to reproduce: 1. Create a hive_table entity and assign a term to it. 2. Perform 1st incremental export of that entity without using changeMarker. 3. Without any modification on that entity perform 2nd consecutive incremental export with changeMarker from previous export. was: Steps to reproduce: 1. Create a hive_table entity and assign a term to it. 2. Perform 1st incremental export of that entity without using changeMarker. 3. Without any modification on that entity perform 2nd consecutive incremental export with changeMarker from previous export. !image-2024-04-11-15-15-59-551.png! > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: image-2024-04-11-15-15-59-551.png > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Attachment: image-2024-04-11-15-15-59-551.png > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: image-2024-04-11-15-15-59-551.png > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it fails export and throws NullPointerException. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Description: Steps to reproduce: 1. Create a hive_table entity and assign a term to it. 2. Perform 1st incremental export of that entity without using changeMarker. 3. Without any modification on that entity perform 2nd consecutive incremental export with changeMarker from previous export. !image-2024-04-11-15-15-59-551.png! was: Steps to reproduce: 1. Create a hive_table entity and assign a term to it. 2. Perform 1st incremental export of that entity without using changeMarker. 3. Without any modification on that entity perform 2nd consecutive incremental export with changeMarker from previous export. Problem is it fails export and throws NullPointerException. > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: image-2024-04-11-15-15-59-551.png > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > !image-2024-04-11-15-15-59-551.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Description: Steps to reproduce: 1. Create a hive_table entity and assign a term to it. 2. Perform 1st incremental export of that entity without using changeMarker. 3. Without any modification on that entity perform 2nd consecutive incremental export with changeMarker from previous export. Problem is it fails export and throws NullPointerException. > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it fails export and throws NullPointerException. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4845) Atlas Import is failing with fetchType: Incremental if there are no changes between two consecutive runs
[ https://issues.apache.org/jira/browse/ATLAS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4845: -- Description: Repro steps: # First export + import with changeMarker “0” - worked as expected. # I tried to run 2nd export + import without any changes in atlas, 2nd export also worked and gave different changeMarker in output as expected. *But 2nd import failed* In other fetch Types this will work. So we also require similar behaviour in case of Incremental type also because this is very normal scenario and customer will hit this very frequently. > Atlas Import is failing with fetchType: Incremental if there are no changes > between two consecutive runs > > > Key: ATLAS-4845 > URL: https://issues.apache.org/jira/browse/ATLAS-4845 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4845.patch > > > Repro steps: > # First export + import with changeMarker “0” - worked as expected. > # I tried to run 2nd export + import without any changes in atlas, 2nd > export also worked and gave different changeMarker in output as expected. > *But 2nd import failed* > In other fetch Types this will work. So we also require similar behaviour in > case of Incremental type also because this is very normal scenario and > customer will hit this very frequently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4845) Atlas Import is failing with fetchType: Incremental if there are no changes between two consecutive runs
[ https://issues.apache.org/jira/browse/ATLAS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4845: -- Attachment: ATLAS-4845.patch > Atlas Import is failing with fetchType: Incremental if there are no changes > between two consecutive runs > > > Key: ATLAS-4845 > URL: https://issues.apache.org/jira/browse/ATLAS-4845 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4845.patch > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Summary: Export/Import : Atlas export fails and throws NullPointerException. (was: Export/Import : Atlas export fails when glossary is assigned to exported entity with fetchtype : Incremental and changeMarker value greater than lastModified time of that entity.) > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ATLAS-4847) Export/Import : Atlas export fails when glossary is assigned to exported entity with fetchtype : Incremental and changeMarker value greater than lastModified time of that
Priyanshi Shah created ATLAS-4847: - Summary: Export/Import : Atlas export fails when glossary is assigned to exported entity with fetchtype : Incremental and changeMarker value greater than lastModified time of that entity. Key: ATLAS-4847 URL: https://issues.apache.org/jira/browse/ATLAS-4847 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Priyanshi Shah Assignee: Priyanshi Shah -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4845) Atlas Import is failing with fetchType: Incremental if there are no changes between two consecutive runs
[ https://issues.apache.org/jira/browse/ATLAS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4845: -- Attachment: (was: ATLAS-4845.patch) > Atlas Import is failing with fetchType: Incremental if there are no changes > between two consecutive runs > > > Key: ATLAS-4845 > URL: https://issues.apache.org/jira/browse/ATLAS-4845 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4842) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4842: -- Attachment: ATLAS-4842.patch > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4842 > URL: https://issues.apache.org/jira/browse/ATLAS-4842 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sheetal Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4842.patch > > > Create two hive_tables in same database: > # create table hive1(id int); > # create table hive1_ctas as select * from hive1; > # create table hive2(name string); > Perform incremental export of hive1 with changeMarker as 0 and skipLineage > value as false. > Import the zip. > Expectation is only the requested hive_table hive1 and its related entities > should be exported, but it does full export and hive2 is also imported. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4845) Atlas Import is failing with fetchType: Incremental if there are no changes between two consecutive runs
[ https://issues.apache.org/jira/browse/ATLAS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4845: -- Attachment: ATLAS-4845.patch > Atlas Import is failing with fetchType: Incremental if there are no changes > between two consecutive runs > > > Key: ATLAS-4845 > URL: https://issues.apache.org/jira/browse/ATLAS-4845 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4845.patch > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4838) Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
[
https://issues.apache.org/jira/browse/ATLAS-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4838:
--
Attachment: (was: ATLAS-4838.patch)
> Export/Import : changeMarker is not set to entity's lastupdatetime or its
> closer timestamp value
>
>
> Key: ATLAS-4838
> URL: https://issues.apache.org/jira/browse/ATLAS-4838
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Sharmadha S
>Assignee: Priyanshi Shah
>Priority: Major
> Attachments: ATLAS-4838-V2.patch
>
>
> When an hive table entity is exported using fetchType incremental with
> changeMarker 0 , after exporting , the changeMarker in the export response is
> not set to recent timestamp.
> For example , on exporting iceberg table with fetchType incremental and
> changeMarker as 0 :
> {code:java}
> 2024-02-08 13:28:43,838 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~
> export(item=AtlasObjectId{guid='null', typeName='iceberg_table',
> uniqueAttributes={qualifiedName:default.iceberg_table2@cm}}; matchType=null,
> fetchType=incremental): found 1 entities: options:
> {"itemsToExport":[{"typeName":"iceberg_table","uniqueAttributes":{"qualifiedName":"default.iceberg_table2@cm"}}],"options":{"fetchType":"incremental","changeMarker":0},"fetchTypeOptionValue":"incremental","skipLineageOptionValue":false,"changeTokenFromOptions":0}
> (StartEntityFetchByExportRequest:123)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ Assigned guid =
> f5e6342a-dfdf-4d54-a1b3-b24f987c4ecb to newly created entity (DataAccess:85)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ addAuditEntry: user: admin, source:
> cm, target: , operation: EXPORT (ExportImportAuditService:168)
> 2024-02-08 13:28:48,699 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ <== export(user=admin,
> from=10.19.28.124): status SUCCESS: changeMarker: 1707225183928
> (ExportService:107) {code}
> changeMarker from response is 1707225183928 => Tuesday, February 6, 2024
> 6:43:03.928 PM
>
> But lastUpdatetime of iceberg table is
> {code:java}
> "createTime": 1707398866970 => Thursday, February 8, 2024 6:57:46.970 PM
> "updateTime": 1707398867375. => Thursday, February 8, 2024 6:57:47.375 PM
> {code}
> Value is set to 2 days back. This is not then an incremental export , as it
> will fetch all updates from Tuesday, February 6, 2024 6:43:03.928 PM in next
> export
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4842) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4842: -- Attachment: (was: ATLAS-4842-V2.patch) > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4842 > URL: https://issues.apache.org/jira/browse/ATLAS-4842 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sheetal Shah >Assignee: Priyanshi Shah >Priority: Major > > Create two hive_tables in same database: > # create table hive1(id int); > # create table hive1_ctas as select * from hive1; > # create table hive2(name string); > Perform incremental export of hive1 with changeMarker as 0 and skipLineage > value as false. > Import the zip. > Expectation is only the requested hive_table hive1 and its related entities > should be exported, but it does full export and hive2 is also imported. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4842) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4842: -- Attachment: (was: 0001-Hive-Incremental.patch) > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4842 > URL: https://issues.apache.org/jira/browse/ATLAS-4842 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sheetal Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: ATLAS-4842-V2.patch > > > Create two hive_tables in same database: > # create table hive1(id int); > # create table hive1_ctas as select * from hive1; > # create table hive2(name string); > Perform incremental export of hive1 with changeMarker as 0 and skipLineage > value as false. > Import the zip. > Expectation is only the requested hive_table hive1 and its related entities > should be exported, but it does full export and hive2 is also imported. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ATLAS-4845) Atlas Import is failing with fetchType: Incremental if there are no changes between two consecutive runs
Priyanshi Shah created ATLAS-4845: - Summary: Atlas Import is failing with fetchType: Incremental if there are no changes between two consecutive runs Key: ATLAS-4845 URL: https://issues.apache.org/jira/browse/ATLAS-4845 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Priyanshi Shah Assignee: Priyanshi Shah -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4844) Atlas - Upgrade Common Configuration2 to 2.10.1
[ https://issues.apache.org/jira/browse/ATLAS-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4844: -- Description: Upgrading common configurations2 from 2.8.0 to 2.10.1 > Atlas - Upgrade Common Configuration2 to 2.10.1 > --- > > Key: ATLAS-4844 > URL: https://issues.apache.org/jira/browse/ATLAS-4844 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 3.0.0 >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > > Upgrading common configurations2 from 2.8.0 to 2.10.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ATLAS-4844) Atlas - Upgrade Common Configuration2 to 2.10.1
Priyanshi Shah created ATLAS-4844: - Summary: Atlas - Upgrade Common Configuration2 to 2.10.1 Key: ATLAS-4844 URL: https://issues.apache.org/jira/browse/ATLAS-4844 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 3.0.0 Reporter: Priyanshi Shah Assignee: Priyanshi Shah -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4842) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4842: -- Attachment: ATLAS-4842-V2.patch > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4842 > URL: https://issues.apache.org/jira/browse/ATLAS-4842 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sheetal Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: 0001-Hive-Incremental.patch, ATLAS-4842-V2.patch > > > Create two hive_tables in same database: > # create table hive1(id int); > # create table hive1_ctas as select * from hive1; > # create table hive2(name string); > Perform incremental export of hive1 with changeMarker as 0 and skipLineage > value as false. > Import the zip. > Expectation is only the requested hive_table hive1 and its related entities > should be exported, but it does full export and hive2 is also imported. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4842) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4842: -- Description: Create two hive_tables in same database: # create table hive1(id int); # create table hive1_ctas as select * from hive1; # create table hive2(name string); Perform incremental export of hive1 with changeMarker as 0 and skipLineage value as false. Import the zip. Expectation is only the requested hive_table hive1 and its related entities should be exported, but it does full export and hive2 is also imported. was: Create two hive_tables in same database: # create table hive1(id int); # create table hive2(name string); Perform incremental export of hive1 with changeMarker provided by previous export. Import the zip. Expectation is only hive1 to be exported and imported, but it does full export and hive2 is also imported. > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4842 > URL: https://issues.apache.org/jira/browse/ATLAS-4842 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sheetal Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: 0001-Hive-Incremental.patch, ATLAS-4842-V2.patch > > > Create two hive_tables in same database: > # create table hive1(id int); > # create table hive1_ctas as select * from hive1; > # create table hive2(name string); > Perform incremental export of hive1 with changeMarker as 0 and skipLineage > value as false. > Import the zip. > Expectation is only the requested hive_table hive1 and its related entities > should be exported, but it does full export and hive2 is also imported. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4838) Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
[
https://issues.apache.org/jira/browse/ATLAS-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4838:
--
Attachment: ATLAS-4838-V2.patch
> Export/Import : changeMarker is not set to entity's lastupdatetime or its
> closer timestamp value
>
>
> Key: ATLAS-4838
> URL: https://issues.apache.org/jira/browse/ATLAS-4838
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Sharmadha S
>Assignee: Priyanshi Shah
>Priority: Major
> Attachments: ATLAS-4838-V2.patch, ATLAS-4838.patch
>
>
> When an hive table entity is exported using fetchType incremental with
> changeMarker 0 , after exporting , the changeMarker in the export response is
> not set to recent timestamp.
> For example , on exporting iceberg table with fetchType incremental and
> changeMarker as 0 :
> {code:java}
> 2024-02-08 13:28:43,838 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~
> export(item=AtlasObjectId{guid='null', typeName='iceberg_table',
> uniqueAttributes={qualifiedName:default.iceberg_table2@cm}}; matchType=null,
> fetchType=incremental): found 1 entities: options:
> {"itemsToExport":[{"typeName":"iceberg_table","uniqueAttributes":{"qualifiedName":"default.iceberg_table2@cm"}}],"options":{"fetchType":"incremental","changeMarker":0},"fetchTypeOptionValue":"incremental","skipLineageOptionValue":false,"changeTokenFromOptions":0}
> (StartEntityFetchByExportRequest:123)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ Assigned guid =
> f5e6342a-dfdf-4d54-a1b3-b24f987c4ecb to newly created entity (DataAccess:85)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ addAuditEntry: user: admin, source:
> cm, target: , operation: EXPORT (ExportImportAuditService:168)
> 2024-02-08 13:28:48,699 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ <== export(user=admin,
> from=10.19.28.124): status SUCCESS: changeMarker: 1707225183928
> (ExportService:107) {code}
> changeMarker from response is 1707225183928 => Tuesday, February 6, 2024
> 6:43:03.928 PM
>
> But lastUpdatetime of iceberg table is
> {code:java}
> "createTime": 1707398866970 => Thursday, February 8, 2024 6:57:46.970 PM
> "updateTime": 1707398867375. => Thursday, February 8, 2024 6:57:47.375 PM
> {code}
> Value is set to 2 days back. This is not then an incremental export , as it
> will fetch all updates from Tuesday, February 6, 2024 6:43:03.928 PM in next
> export
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (ATLAS-4838) Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
[
https://issues.apache.org/jira/browse/ATLAS-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah reassigned ATLAS-4838:
-
Assignee: Priyanshi Shah (was: Jayendra Parab)
> Export/Import : changeMarker is not set to entity's lastupdatetime or its
> closer timestamp value
>
>
> Key: ATLAS-4838
> URL: https://issues.apache.org/jira/browse/ATLAS-4838
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Sharmadha S
>Assignee: Priyanshi Shah
>Priority: Major
> Attachments: ATLAS-4838.patch
>
>
> When an hive table entity is exported using fetchType incremental with
> changeMarker 0 , after exporting , the changeMarker in the export response is
> not set to recent timestamp.
> For example , on exporting iceberg table with fetchType incremental and
> changeMarker as 0 :
> {code:java}
> 2024-02-08 13:28:43,838 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~
> export(item=AtlasObjectId{guid='null', typeName='iceberg_table',
> uniqueAttributes={qualifiedName:default.iceberg_table2@cm}}; matchType=null,
> fetchType=incremental): found 1 entities: options:
> {"itemsToExport":[{"typeName":"iceberg_table","uniqueAttributes":{"qualifiedName":"default.iceberg_table2@cm"}}],"options":{"fetchType":"incremental","changeMarker":0},"fetchTypeOptionValue":"incremental","skipLineageOptionValue":false,"changeTokenFromOptions":0}
> (StartEntityFetchByExportRequest:123)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ Assigned guid =
> f5e6342a-dfdf-4d54-a1b3-b24f987c4ecb to newly created entity (DataAccess:85)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ addAuditEntry: user: admin, source:
> cm, target: , operation: EXPORT (ExportImportAuditService:168)
> 2024-02-08 13:28:48,699 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ <== export(user=admin,
> from=10.19.28.124): status SUCCESS: changeMarker: 1707225183928
> (ExportService:107) {code}
> changeMarker from response is 1707225183928 => Tuesday, February 6, 2024
> 6:43:03.928 PM
>
> But lastUpdatetime of iceberg table is
> {code:java}
> "createTime": 1707398866970 => Thursday, February 8, 2024 6:57:46.970 PM
> "updateTime": 1707398867375. => Thursday, February 8, 2024 6:57:47.375 PM
> {code}
> Value is set to 2 days back. This is not then an incremental export , as it
> will fetch all updates from Tuesday, February 6, 2024 6:43:03.928 PM in next
> export
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4838) Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
[
https://issues.apache.org/jira/browse/ATLAS-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4838:
--
Attachment: ATLAS-4838.patch
> Export/Import : changeMarker is not set to entity's lastupdatetime or its
> closer timestamp value
>
>
> Key: ATLAS-4838
> URL: https://issues.apache.org/jira/browse/ATLAS-4838
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Sharmadha S
>Assignee: Jayendra Parab
>Priority: Major
> Attachments: ATLAS-4838.patch
>
>
> When an hive table entity is exported using fetchType incremental with
> changeMarker 0 , after exporting , the changeMarker in the export response is
> not set to recent timestamp.
> For example , on exporting iceberg table with fetchType incremental and
> changeMarker as 0 :
> {code:java}
> 2024-02-08 13:28:43,838 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~
> export(item=AtlasObjectId{guid='null', typeName='iceberg_table',
> uniqueAttributes={qualifiedName:default.iceberg_table2@cm}}; matchType=null,
> fetchType=incremental): found 1 entities: options:
> {"itemsToExport":[{"typeName":"iceberg_table","uniqueAttributes":{"qualifiedName":"default.iceberg_table2@cm"}}],"options":{"fetchType":"incremental","changeMarker":0},"fetchTypeOptionValue":"incremental","skipLineageOptionValue":false,"changeTokenFromOptions":0}
> (StartEntityFetchByExportRequest:123)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ Assigned guid =
> f5e6342a-dfdf-4d54-a1b3-b24f987c4ecb to newly created entity (DataAccess:85)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ addAuditEntry: user: admin, source:
> cm, target: , operation: EXPORT (ExportImportAuditService:168)
> 2024-02-08 13:28:48,699 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ <== export(user=admin,
> from=10.19.28.124): status SUCCESS: changeMarker: 1707225183928
> (ExportService:107) {code}
> changeMarker from response is 1707225183928 => Tuesday, February 6, 2024
> 6:43:03.928 PM
>
> But lastUpdatetime of iceberg table is
> {code:java}
> "createTime": 1707398866970 => Thursday, February 8, 2024 6:57:46.970 PM
> "updateTime": 1707398867375. => Thursday, February 8, 2024 6:57:47.375 PM
> {code}
> Value is set to 2 days back. This is not then an incremental export , as it
> will fetch all updates from Tuesday, February 6, 2024 6:43:03.928 PM in next
> export
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4842) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4842: -- Attachment: 0001-Hive-Incremental.patch > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4842 > URL: https://issues.apache.org/jira/browse/ATLAS-4842 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sheetal Shah >Assignee: Priyanshi Shah >Priority: Major > Attachments: 0001-Hive-Incremental.patch > > > Create two hive_tables in same database: > # create table hive1(id int); > # create table hive2(name string); > Perform incremental export of hive1 with changeMarker provided by previous > export. > Import the zip. > Expectation is only hive1 to be exported and imported, but it does full > export and hive2 is also imported. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ATLAS-4839) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah reassigned ATLAS-4839: - Assignee: Priyanshi Shah > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4839 > URL: https://issues.apache.org/jira/browse/ATLAS-4839 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sharmadha S >Assignee: Priyanshi Shah >Priority: Major > > Create 2 tables > # create table iceberg_table_1(id int) stored as iceberg; > # create table hive_table_1(id int); > Export iceberg table using fetchType incremental and changeMarker 0 > > Import the zip. > > Expectation is only iceberg_table to be exported and imported, but it does > full export and hive_table_1 is also imported. -- This message was sent by Atlassian Jira (v8.20.10#820010)
