[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive_table's metadata repeatly
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Summary: Partitionkeys was deleted when import hive_table's metadata repeatly (was: Partitionkeys was deleted when import hive_table's metadata) > Partitionkeys was deleted when import hive_table's metadata repeatly > > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-40-43-925.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-40-20-834.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive_table's metadata repeatly
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Description: I find two bugs when import hive_table's metadata into Atlas： The first time importing hive_table's metadata into Atlas, the partitonKeys was wrongly included in hive_table's columns: !image-2019-10-29-20-40-43-925.png! When update the metadata of hive_table, the partitonKeys was deleted wrongly: !image-2019-10-29-20-40-20-834.png! IIt seems to be a blocker of Atlas, looking forward to your early reply. was: The first time importing hive_table's metadata into Atlas, the partitonKeys was wrongly included in hive_table's columns: !image-2019-10-29-20-40-43-925.png! When update the metadata of hive_table, the partitonKeys was deleted wrongly: !image-2019-10-29-20-40-20-834.png! IIt seems to be a blocker of Atlas, looking forward to your early reply. > Partitionkeys was deleted when import hive_table's metadata repeatly > > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > I find two bugs when import hive_table's metadata into Atlas： > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-40-43-925.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-40-20-834.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive_table's metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Summary: Partitionkeys was deleted when import hive_table's metadata (was: Partitionkeys was deleted when import hive metadata) > Partitionkeys was deleted when import hive_table's metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-40-43-925.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-40-20-834.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961975#comment-16961975 ] Jiaqi Shan commented on ATLAS-3499: --- This issue is related to https://issues.apache.org/jira/browse/ATLAS-3392 > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-40-43-925.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-40-20-834.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Attachment: (was: image-2019-10-29-20-39-41-773.png) > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-40-43-925.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-40-20-834.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Description: The first time importing hive_table's metadata into Atlas, the partitonKeys was wrongly included in hive_table's columns: !image-2019-10-29-20-40-43-925.png! When update the metadata of hive_table, the partitonKeys was deleted wrongly: !image-2019-10-29-20-40-20-834.png! IIt seems to be a blocker of Atlas, looking forward to your early reply. was: The first time importing hive_table's metadata into Atlas, the partitonKeys was wrongly included in hive_table's columns: !image-2019-10-29-20-39-41-773.png! When update the metadata of hive_table, the partitonKeys was deleted wrongly: IIt seems to be a blocker of Atlas, looking forward to your early reply. > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-40-43-925.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-40-20-834.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Attachment: image-2019-10-29-20-40-20-834.png > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-39-41-773.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Attachment: (was: image-2019-10-29-20-32-26-069.png) > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-39-41-773.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Attachment: image-2019-10-29-20-40-43-925.png > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-40-20-834.png, > image-2019-10-29-20-40-43-925.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-39-41-773.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Attachment: image-2019-10-29-20-39-41-773.png > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-32-26-069.png, > image-2019-10-29-20-39-41-773.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-32-18-341.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-32-26-069.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Attachment: (was: image-2019-10-29-20-32-18-341.png) > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-32-26-069.png, > image-2019-10-29-20-39-41-773.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-32-18-341.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-32-26-069.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Description: The first time importing hive_table's metadata into Atlas, the partitonKeys was wrongly included in hive_table's columns: !image-2019-10-29-20-39-41-773.png! When update the metadata of hive_table, the partitonKeys was deleted wrongly: IIt seems to be a blocker of Atlas, looking forward to your early reply. was: The first time importing hive_table's metadata into Atlas, the partitonKeys was wrongly included in hive_table's columns: !image-2019-10-29-20-32-18-341.png! When update the metadata of hive_table, the partitonKeys was deleted wrongly: !image-2019-10-29-20-32-26-069.png! IIt seems to be a blocker of Atlas, looking forward to your early reply. > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-32-26-069.png, > image-2019-10-29-20-39-41-773.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-39-41-773.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Attachment: (was: image-2019-10-29-20-32-10-960.png) > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-32-26-069.png, > image-2019-10-29-20-39-41-773.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-32-18-341.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-32-26-069.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
[ https://issues.apache.org/jira/browse/ATLAS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3499: -- Attachment: (was: first.png) > Partitionkeys was deleted when import hive metadata > --- > > Key: ATLAS-3499 > URL: https://issues.apache.org/jira/browse/ATLAS-3499 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Jiaqi Shan >Priority: Blocker > Attachments: image-2019-10-29-20-32-10-960.png, > image-2019-10-29-20-32-18-341.png, image-2019-10-29-20-32-26-069.png > > > The first time importing hive_table's metadata into Atlas, the partitonKeys > was wrongly included in hive_table's columns: > !image-2019-10-29-20-32-18-341.png! > > When update the metadata of hive_table, the partitonKeys was deleted wrongly: > !image-2019-10-29-20-32-26-069.png! > IIt seems to be a blocker of Atlas, looking forward to your early reply. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3499) Partitionkeys was deleted when import hive metadata
Jiaqi Shan created ATLAS-3499: - Summary: Partitionkeys was deleted when import hive metadata Key: ATLAS-3499 URL: https://issues.apache.org/jira/browse/ATLAS-3499 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.0.0 Reporter: Jiaqi Shan Attachments: first.png, image-2019-10-29-20-32-10-960.png, image-2019-10-29-20-32-18-341.png, image-2019-10-29-20-32-26-069.png The first time importing hive_table's metadata into Atlas, the partitonKeys was wrongly included in hive_table's columns: !image-2019-10-29-20-32-18-341.png! When update the metadata of hive_table, the partitonKeys was deleted wrongly: !image-2019-10-29-20-32-26-069.png! IIt seems to be a blocker of Atlas, looking forward to your early reply. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading
[
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3474:
--
Attachment: (was: ATLAS-3474.patch)
> Atlas throws IOException when call api in multithreading
>
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3474.patch
>
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the
> requests timeout frequently. I found the following exception in
> atlas.20191017-104155.out:
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632)
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout
> expired: 3/3 ms at
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
> at
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
> at
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
> at
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
> at
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
> at
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913)
> at
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625)
> at
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
>
[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading
[
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3474:
--
Attachment: ATLAS-3474.patch
> Atlas throws IOException when call api in multithreading
>
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3474.patch
>
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the
> requests timeout frequently. I found the following exception in
> atlas.20191017-104155.out:
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632)
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout
> expired: 3/3 ms at
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
> at
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
> at
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
> at
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
> at
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
> at
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913)
> at
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625)
> at
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
>
[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading
[
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3474:
--
Attachment: ATLAS-3474.patch
> Atlas throws IOException when call api in multithreading
>
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3474.patch
>
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the
> requests timeout frequently. I found the following exception in
> atlas.20191017-104155.out:
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632)
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout
> expired: 3/3 ms at
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
> at
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
> at
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
> at
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
> at
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
> at
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913)
> at
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625)
> at
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
>
[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading
[
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3474:
--
Attachment: (was: ATLAS-3474.patch)
> Atlas throws IOException when call api in multithreading
>
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the
> requests timeout frequently. I found the following exception in
> atlas.20191017-104155.out:
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632)
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout
> expired: 3/3 ms at
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
> at
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
> at
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
> at
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
> at
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
> at
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913)
> at
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625)
> at
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
>
[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading
[
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3474:
--
Attachment: ATLAS-3474.patch
> Atlas throws IOException when call api in multithreading
>
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3474.patch
>
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the
> requests timeout frequently. I found the following exception in
> atlas.20191017-104155.out:
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632)
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout
> expired: 3/3 ms at
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
> at
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
> at
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
> at
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
> at
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
> at
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
> at
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
> at
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
> at
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
> at
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913)
> at
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625)
> at
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
>
[jira] [Created] (ATLAS-3474) Atlas throws IOException when call api in multithreading
Jiaqi Shan created ATLAS-3474:
-
Summary: Atlas throws IOException when call api in multithreading
Key: ATLAS-3474
URL: https://issues.apache.org/jira/browse/ATLAS-3474
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 1.1.0
Reporter: Jiaqi Shan
When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the
requests timeout frequently. I found the following exception in
atlas.20191017-104155.out:
{code:java}
2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~
/api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632)
java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout
expired: 3/3 ms at
org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at
org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at
org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
at
com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
at
com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
at
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
at
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at
com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
at
com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
at
com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at
com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
at
com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913)
at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625)
at
com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845) at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1712)
at org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:80) at
[jira] [Updated] (ATLAS-3172) Atlas Hive Hook lack of Jackson dependency
[
https://issues.apache.org/jira/browse/ATLAS-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3172:
--
Attachment: ATLAS-3172.patch
> Atlas Hive Hook lack of Jackson dependency
> --
>
> Key: ATLAS-3172
> URL: https://issues.apache.org/jira/browse/ATLAS-3172
> Project: Atlas
> Issue Type: Bug
>Reporter: jianxiong zhao
>Priority: Major
> Attachments: ATLAS-3172.patch
>
>
> After copying Hive Hook to ${ATLAS_HOME}, executing `hook-bin/import-hive.sh`
> always fails with lack of Jackson.XXX dependencies
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-3172) Atlas Hive Hook lack of Jackson dependency
[
https://issues.apache.org/jira/browse/ATLAS-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937429#comment-16937429
]
Jiaqi Shan commented on ATLAS-3172:
---
Our Hive version is 1.1.0. When updating HiveBridge from 1.1.0 to 2.0.0, I
{color:#66}encountered the same errors as [~theHarpers]. HiveBridge can't
find the following jars:*jackson-jaxrs-base-2.9.8.jar*,
*jackson-jaxrs-json-provider-2.9.8.jar* and
*jackson-module-jaxb-annotations-2.9.8.jar.*{color}
*Caused by: java.lang.ClassNotFoundException:
com.fasterxml.jackson.module.jaxb.JaxbAnnotationIntrospector*
Exception in thread "main" java.lang.NoClassDefFoundError:
com/fasterxml/jackson/module/jaxb/JaxbAnnotationIntrospector
at
com.fasterxml.jackson.jaxrs.json.JsonMapperConfigurator._resolveIntrospector(JsonMapperConfigurator.java:109)
at
com.fasterxml.jackson.jaxrs.json.JsonMapperConfigurator._resolveIntrospectors(JsonMapperConfigurator.java:84)
at
com.fasterxml.jackson.jaxrs.cfg.MapperConfiguratorBase._setAnnotations(MapperConfiguratorBase.java:120)
at
com.fasterxml.jackson.jaxrs.json.JsonMapperConfigurator.getDefaultMapper(JsonMapperConfigurator.java:45)
at
com.fasterxml.jackson.jaxrs.base.ProviderBase.locateMapper(ProviderBase.java:932)
at
com.fasterxml.jackson.jaxrs.base.ProviderBase._endpointForReading(ProviderBase.java:858)
at
com.fasterxml.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:766)
at
com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
at
com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:523)
at
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:409)
at
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:246)
at
org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:285)
at
org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:276)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findEntity(HiveMetaStoreBridge.java:799)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findDatabase(HiveMetaStoreBridge.java:764)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:405)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:284)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:254)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:171)
Caused by: java.lang.ClassNotFoundException:
com.fasterxml.jackson.module.jaxb.JaxbAnnotationIntrospector
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 20 more
*Caused by: java.lang.ClassNotFoundException:
com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider*
{code:java}
Exception in thread "main" java.lang.NoClassDefFoundError:
com/fasterxml/jackson/jaxrs/json/JacksonJaxbJsonProvider
at org.apache.atlas.AtlasBaseClient.getClient(AtlasBaseClient.java:270)
at
org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:453)
at
org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:448)
at org.apache.atlas.AtlasBaseClient.(AtlasBaseClient.java:132)
at org.apache.atlas.AtlasClientV2.(AtlasClientV2.java:82)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:134)
Caused by: java.lang.ClassNotFoundException:
com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 6 more
{code}
*Caused by: java.lang.ClassNotFoundException:
com.fasterxml.jackson.jaxrs.base.ProviderBase*
{code:java}
Exception in thread "main" java.lang.NoClassDefFoundError:
com/fasterxml/jackson/jaxrs/base/ProviderBase
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
[jira] [Updated] (ATLAS-3421) HiveBridge supports to skip tables without any update
[ https://issues.apache.org/jira/browse/ATLAS-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan updated ATLAS-3421: -- Attachment: ATLAS-3421.patch > HiveBridge supports to skip tables without any update > - > > Key: ATLAS-3421 > URL: https://issues.apache.org/jira/browse/ATLAS-3421 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Jiaqi Shan >Priority: Major > Attachments: ATLAS-3421.patch > > > h1. *Background* > In my company, There are more than 8+ hive tables in production. We will > import hive metadata to Atlas everyday to avoid any change is missed by Hive > Hook. This timed task causes two problem: > 1) It costs a large amount of time(more than 20 hours at a time) > 2) Can't use incremental export(every hive_table's updateTime was updated by > HiveBridge) > h1. *Solution* > When creating or updating a hive table, the transient_lastDdlTime would be > updated. So if the transient_lastDdlTime is the same in Atlas and metastore, > HiveBridge should skip updating the table's metadata. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-3421) HiveBridge supports to skip tables without any update
[ https://issues.apache.org/jira/browse/ATLAS-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Shan reassigned ATLAS-3421: - Assignee: Jiaqi Shan > HiveBridge supports to skip tables without any update > - > > Key: ATLAS-3421 > URL: https://issues.apache.org/jira/browse/ATLAS-3421 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Jiaqi Shan >Assignee: Jiaqi Shan >Priority: Major > Attachments: ATLAS-3421.patch > > > h1. *Background* > In my company, There are more than 8+ hive tables in production. We will > import hive metadata to Atlas everyday to avoid any change is missed by Hive > Hook. This timed task causes two problem: > 1) It costs a large amount of time(more than 20 hours at a time) > 2) Can't use incremental export(every hive_table's updateTime was updated by > HiveBridge) > h1. *Solution* > When creating or updating a hive table, the transient_lastDdlTime would be > updated. So if the transient_lastDdlTime is the same in Atlas and metastore, > HiveBridge should skip updating the table's metadata. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3421) HiveBridge supports to skip tables without any update
Jiaqi Shan created ATLAS-3421: - Summary: HiveBridge supports to skip tables without any update Key: ATLAS-3421 URL: https://issues.apache.org/jira/browse/ATLAS-3421 Project: Atlas Issue Type: Improvement Components: atlas-core Reporter: Jiaqi Shan h1. *Background* In my company, There are more than 8+ hive tables in production. We will import hive metadata to Atlas everyday to avoid any change is missed by Hive Hook. This timed task causes two problem: 1) It costs a large amount of time(more than 20 hours at a time) 2) Can't use incremental export(every hive_table's updateTime was updated by HiveBridge) h1. *Solution* When creating or updating a hive table, the transient_lastDdlTime would be updated. So if the transient_lastDdlTime is the same in Atlas and metastore, HiveBridge should skip updating the table's metadata. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3372) Failed to build atlas-dashboardv2 caused by too low node version
[
https://issues.apache.org/jira/browse/ATLAS-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3372:
--
Description:
When building atlas-dashboardv2, it's giving me the following error:
{quote}[INFO] Running 'grunt build-minify' in
C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
[INFO] Loading "clean.js" tasks...ERROR
[INFO] >> SyntaxError: Block-scoped declarations (let, const, function, class)
not yet supported outside strict mode
[INFO] Warning: Task "clean" not found. Use --force to continue.
[INFO]
[INFO] Aborted due to warnings.
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 02:21 min
[INFO] Finished at: 2019-08-14T20:14:30+08:00
[INFO]
[ERROR] Failed to execute goal
com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 3 (Exit
value: 3) -> [Help 1]
{quote}
I found this issue might be relevant:
[https://github.com/JeffreyWay/laravel-mix/issues/264]
The problem seems to be caused by too low node version.
I update node and npm version to be the same with Atlas-2.0.0 and it works.
In addition, this error didn't occur before 2019.8.13. Is there any change for
node or atlas?
was:
When building atlas-dashboardv2, it's giving me the following error:
{quote}[INFO] Running 'grunt build-minify' in
C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
[INFO] Loading "clean.js" tasks...ERROR
[INFO] >> SyntaxError: Block-scoped declarations (let, const, function, class)
not yet supported outside strict mode
[INFO] Warning: Task "clean" not found. Use --force to continue.
[INFO]
[INFO] Aborted due to warnings.
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 02:21 min
[INFO] Finished at: 2019-08-14T20:14:30+08:00
[INFO]
[ERROR] Failed to execute goal
com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 3 (Exit
value: 3) -> [Help 1]
{quote}
I found this issue might be relevant:
[https://github.com/JeffreyWay/laravel-mix/issues/264]
The problem seems to be caused by too low node version.
I update node version and npm version to be same with Atlas-2.0.0 and it works.
In addition, this error didn't occur before 2019.8.13. Is any change for node
or atlas?
> Failed to build atlas-dashboardv2 caused by too low node version
>
>
> Key: ATLAS-3372
> URL: https://issues.apache.org/jira/browse/ATLAS-3372
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Blocker
> Attachments: ATLAS-3372.patch
>
>
> When building atlas-dashboardv2, it's giving me the following error:
> {quote}[INFO] Running 'grunt build-minify' in
> C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
> [INFO] Loading "clean.js" tasks...ERROR
> [INFO] >> SyntaxError: Block-scoped declarations (let, const, function,
> class) not yet supported outside strict mode
> [INFO] Warning: Task "clean" not found. Use --force to continue.
> [INFO]
> [INFO] Aborted due to warnings.
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 02:21 min
> [INFO] Finished at: 2019-08-14T20:14:30+08:00
> [INFO]
>
> [ERROR] Failed to execute goal
> com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
> atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
> org.apache.commons.exec.ExecuteException: Process exited with an error: 3
> (Exit value: 3) -> [Help 1]
> {quote}
>
> I found this issue might be relevant:
> [https://github.com/JeffreyWay/laravel-mix/issues/264]
> The problem seems to be caused by too low node version.
> I update node and npm version to be the same with Atlas-2.0.0 and it works.
> In addition, this error didn't occur before 2019.8.13. Is there any change
> for node or atlas?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (ATLAS-3372) Failed to build atlas-dashboardv2 caused by too low node version
[
https://issues.apache.org/jira/browse/ATLAS-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3372:
--
Attachment: ATLAS-3372.patch
> Failed to build atlas-dashboardv2 caused by too low node version
>
>
> Key: ATLAS-3372
> URL: https://issues.apache.org/jira/browse/ATLAS-3372
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Blocker
> Attachments: ATLAS-3372.patch
>
>
> When building atlas-dashboardv2, it's giving me the following error:
> {quote}[INFO] Running 'grunt build-minify' in
> C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
> [INFO] Loading "clean.js" tasks...ERROR
> [INFO] >> SyntaxError: Block-scoped declarations (let, const, function,
> class) not yet supported outside strict mode
> [INFO] Warning: Task "clean" not found. Use --force to continue.
> [INFO]
> [INFO] Aborted due to warnings.
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 02:21 min
> [INFO] Finished at: 2019-08-14T20:14:30+08:00
> [INFO]
>
> [ERROR] Failed to execute goal
> com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
> atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
> org.apache.commons.exec.ExecuteException: Process exited with an error: 3
> (Exit value: 3) -> [Help 1]
> {quote}
>
> I found this issue might be relevant:
> [https://github.com/JeffreyWay/laravel-mix/issues/264]
> The problem seems to be caused by too low node version.
> I update node version and npm version to be same with Atlas-2.0.0 and it
> works.
> in addition, this error didn't occur before 2019.8.13. Is any change for
> node or atlas?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (ATLAS-3372) Failed to build atlas-dashboardv2 caused by too low node version
[
https://issues.apache.org/jira/browse/ATLAS-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3372:
--
Description:
When building atlas-dashboardv2, it's giving me the following error:
{quote}[INFO] Running 'grunt build-minify' in
C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
[INFO] Loading "clean.js" tasks...ERROR
[INFO] >> SyntaxError: Block-scoped declarations (let, const, function, class)
not yet supported outside strict mode
[INFO] Warning: Task "clean" not found. Use --force to continue.
[INFO]
[INFO] Aborted due to warnings.
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 02:21 min
[INFO] Finished at: 2019-08-14T20:14:30+08:00
[INFO]
[ERROR] Failed to execute goal
com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 3 (Exit
value: 3) -> [Help 1]
{quote}
I found this issue might be relevant:
[https://github.com/JeffreyWay/laravel-mix/issues/264]
The problem seems to be caused by too low node version.
I update node version and npm version to be same with Atlas-2.0.0 and it works.
In addition, this error didn't occur before 2019.8.13. Is any change for node
or atlas?
was:
When building atlas-dashboardv2, it's giving me the following error:
{quote}[INFO] Running 'grunt build-minify' in
C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
[INFO] Loading "clean.js" tasks...ERROR
[INFO] >> SyntaxError: Block-scoped declarations (let, const, function, class)
not yet supported outside strict mode
[INFO] Warning: Task "clean" not found. Use --force to continue.
[INFO]
[INFO] Aborted due to warnings.
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 02:21 min
[INFO] Finished at: 2019-08-14T20:14:30+08:00
[INFO]
[ERROR] Failed to execute goal
com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 3 (Exit
value: 3) -> [Help 1]
{quote}
I found this issue might be relevant:
[https://github.com/JeffreyWay/laravel-mix/issues/264]
The problem seems to be caused by too low node version.
I update node version and npm version to be same with Atlas-2.0.0 and it works.
in addition, this error didn't occur before 2019.8.13. Is any change for node
or atlas?
> Failed to build atlas-dashboardv2 caused by too low node version
>
>
> Key: ATLAS-3372
> URL: https://issues.apache.org/jira/browse/ATLAS-3372
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Blocker
> Attachments: ATLAS-3372.patch
>
>
> When building atlas-dashboardv2, it's giving me the following error:
> {quote}[INFO] Running 'grunt build-minify' in
> C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
> [INFO] Loading "clean.js" tasks...ERROR
> [INFO] >> SyntaxError: Block-scoped declarations (let, const, function,
> class) not yet supported outside strict mode
> [INFO] Warning: Task "clean" not found. Use --force to continue.
> [INFO]
> [INFO] Aborted due to warnings.
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 02:21 min
> [INFO] Finished at: 2019-08-14T20:14:30+08:00
> [INFO]
>
> [ERROR] Failed to execute goal
> com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
> atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
> org.apache.commons.exec.ExecuteException: Process exited with an error: 3
> (Exit value: 3) -> [Help 1]
> {quote}
>
> I found this issue might be relevant:
> [https://github.com/JeffreyWay/laravel-mix/issues/264]
> The problem seems to be caused by too low node version.
> I update node version and npm version to be same with Atlas-2.0.0 and it
> works.
> In addition, this error didn't occur before 2019.8.13. Is any change for
> node or atlas?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (ATLAS-3372) Failed to build atlas-dashboardv2 caused by too low node version
[
https://issues.apache.org/jira/browse/ATLAS-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3372:
--
Description:
When building atlas-dashboardv2, it's giving me the following error:
{quote}[INFO] Running 'grunt build-minify' in
C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
[INFO] Loading "clean.js" tasks...ERROR
[INFO] >> SyntaxError: Block-scoped declarations (let, const, function, class)
not yet supported outside strict mode
[INFO] Warning: Task "clean" not found. Use --force to continue.
[INFO]
[INFO] Aborted due to warnings.
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 02:21 min
[INFO] Finished at: 2019-08-14T20:14:30+08:00
[INFO]
[ERROR] Failed to execute goal
com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 3 (Exit
value: 3) -> [Help 1]
{quote}
I found this issue might be relevant:
[https://github.com/JeffreyWay/laravel-mix/issues/264]
The problem seems to be caused by too low node version.
I update node version and npm version to be same with Atlas-2.0.0 and it works.
in addition, this error didn't occur before 2019.8.13. Is any change for node
or atlas?
was:
When building atlas-dashboardv2, it's giving me the following error:
{quote}[INFO] Running 'grunt build-minify' in
C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
[INFO] Loading "clean.js" tasks...ERROR
[INFO] >> SyntaxError: Block-scoped declarations (let, const, function, class)
not yet supported outside strict mode
[INFO] Warning: Task "clean" not found. Use --force to continue.
[INFO]
[INFO] Aborted due to warnings.
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 02:21 min
[INFO] Finished at: 2019-08-14T20:14:30+08:00
[INFO]
[ERROR] Failed to execute goal
com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 3 (Exit
value: 3) -> [Help 1]
{quote}
I found this issue might be relevant:
[https://github.com/JeffreyWay/laravel-mix/issues/264] The problem seems to be
caused by too low node version.
I update node version and npm version to be same with Atlas-2.0.0 and it works.
in addition, this error didn't occur before 2019.8.13. Is any change for node
or atlas?
> Failed to build atlas-dashboardv2 caused by too low node version
>
>
> Key: ATLAS-3372
> URL: https://issues.apache.org/jira/browse/ATLAS-3372
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Blocker
>
> When building atlas-dashboardv2, it's giving me the following error:
> {quote}[INFO] Running 'grunt build-minify' in
> C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
> [INFO] Loading "clean.js" tasks...ERROR
> [INFO] >> SyntaxError: Block-scoped declarations (let, const, function,
> class) not yet supported outside strict mode
> [INFO] Warning: Task "clean" not found. Use --force to continue.
> [INFO]
> [INFO] Aborted due to warnings.
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 02:21 min
> [INFO] Finished at: 2019-08-14T20:14:30+08:00
> [INFO]
>
> [ERROR] Failed to execute goal
> com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
> atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
> org.apache.commons.exec.ExecuteException: Process exited with an error: 3
> (Exit value: 3) -> [Help 1]
> {quote}
>
> I found this issue might be relevant:
> [https://github.com/JeffreyWay/laravel-mix/issues/264]
> The problem seems to be caused by too low node version.
> I update node version and npm version to be same with Atlas-2.0.0 and it
> works.
> in addition, this error didn't occur before 2019.8.13. Is any change for
> node or atlas?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (ATLAS-3372) Failed to build atlas-dashboardv2 caused by too low node version
Jiaqi Shan created ATLAS-3372:
-
Summary: Failed to build atlas-dashboardv2 caused by too low node
version
Key: ATLAS-3372
URL: https://issues.apache.org/jira/browse/ATLAS-3372
Project: Atlas
Issue Type: Bug
Components: atlas-webui
Affects Versions: 1.1.0
Reporter: Jiaqi Shan
When building atlas-dashboardv2, it's giving me the following error:
{quote}[INFO] Running 'grunt build-minify' in
C:\ideaProject\atlas\anchor\atlas\dashboardv2\target
[INFO] Loading "clean.js" tasks...ERROR
[INFO] >> SyntaxError: Block-scoped declarations (let, const, function, class)
not yet supported outside strict mode
[INFO] Warning: Task "clean" not found. Use --force to continue.
[INFO]
[INFO] Aborted due to warnings.
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 02:21 min
[INFO] Finished at: 2019-08-14T20:14:30+08:00
[INFO]
[ERROR] Failed to execute goal
com.github.eirslett:frontend-maven-plugin:1.4:grunt (grunt dist) on project
atlas-dashboardv2: Failed to run task: 'grunt build-minify' failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 3 (Exit
value: 3) -> [Help 1]
{quote}
I found this issue might be relevant:
[https://github.com/JeffreyWay/laravel-mix/issues/264] The problem seems to be
caused by too low node version.
I update node version and npm version to be same with Atlas-2.0.0 and it works.
in addition, this error didn't occur before 2019.8.13. Is any change for node
or atlas?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (ATLAS-3319) Add api to get lineage by uniqueAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897118#comment-16897118
]
Jiaqi Shan commented on ATLAS-3319:
---
I found that i can't assign this Jira to myself. Does it means that i wasn't
added to Atlas contributor-ship? So i resend the email to
[dev@atlas.apache.org|mailto:dev@atlas.apache.org]，could you please check the
email？[~nixonrodrigues]
> Add api to get lineage by uniqueAttributes
> --
>
> Key: ATLAS-3319
> URL: https://issues.apache.org/jira/browse/ATLAS-3319
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
> Attachments: ATLAS-3319.patch
>
>
> To get lineage of a hive table, we need to go through the following two steps:
> * Call get /v2/entity/bulk/uniqueAttribute/type/\{typeName} to get guid of
> the hive table
> * Call /v2/lineage/\{guid} to get lineage of the hive table
> It seems that support getting lineage by uniqueAttributes will make api more
> easy to use. So we add the follow rest api to \{{LineageREST}}:
> /v2/lineage/uniqueAttribute/type/\{typeName}
> We also add the api to \{{AtlasClientV2}} accordingly
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (ATLAS-3319) Add api to get lineage by uniqueAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892596#comment-16892596
]
Jiaqi Shan commented on ATLAS-3319:
---
I've updated the java doc for api, can you please review it? [~nixonrodrigues]
> Add api to get lineage by uniqueAttributes
> --
>
> Key: ATLAS-3319
> URL: https://issues.apache.org/jira/browse/ATLAS-3319
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3319.patch
>
>
> To get lineage of a hive table, we need to go through the following two steps:
> * Call get /v2/entity/bulk/uniqueAttribute/type/\{typeName} to get guid of
> the hive table
> * Call /v2/lineage/\{guid} to get lineage of the hive table
> It seems that support getting lineage by uniqueAttributes will make api more
> easy to use. So we add the follow rest api to \{{LineageREST}}:
> /v2/lineage/uniqueAttribute/type/\{typeName}
> We also add the api to \{{AtlasClientV2}} accordingly
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (ATLAS-3319) Add api to get lineage by uniqueAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887633#comment-16887633
]
Jiaqi Shan commented on ATLAS-3319:
---
Thanks [~nixonrodrigues] for review! The review board url is:
[https://reviews.apache.org/r/71113/|https://reviews.apache.org/r/71113/]. It's
an honor to be contributor of Atlas!
> Add api to get lineage by uniqueAttributes
> --
>
> Key: ATLAS-3319
> URL: https://issues.apache.org/jira/browse/ATLAS-3319
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3319.patch
>
>
> To get lineage of a hive table, we need to go through the following two steps:
> * Call get /v2/entity/bulk/uniqueAttribute/type/\{typeName} to get guid of
> the hive table
> * Call /v2/lineage/\{guid} to get lineage of the hive table
> It seems that support getting lineage by uniqueAttributes will make api more
> easy to use. So we add the follow rest api to \{{LineageREST}}:
> /v2/lineage/uniqueAttribute/type/\{typeName}
> We also add the api to \{{AtlasClientV2}} accordingly
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (ATLAS-3319) Add api to get lineage by uniqueAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3319:
--
Flags: (was: Patch)
> Add api to get lineage by uniqueAttributes
> --
>
> Key: ATLAS-3319
> URL: https://issues.apache.org/jira/browse/ATLAS-3319
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3319.patch
>
>
> To get lineage of a hive table, we need to go through the following two steps:
> * Call get /v2/entity/bulk/uniqueAttribute/type/\{typeName} to get guid of
> the hive table
> * Call /v2/lineage/\{guid} to get lineage of the hive table
> It seems that support getting lineage by uniqueAttributes will make api more
> easy to use. So we add the follow rest api to \{{LineageREST}}:
> /v2/lineage/uniqueAttribute/type/\{typeName}
> We also add the api to \{{AtlasClientV2}} accordingly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3319) Add api to get lineage by uniqueAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3319:
--
Flags: Patch
> Add api to get lineage by uniqueAttributes
> --
>
> Key: ATLAS-3319
> URL: https://issues.apache.org/jira/browse/ATLAS-3319
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3319.patch
>
>
> To get lineage of a hive table, we need to go through the following two steps:
> * Call get /v2/entity/bulk/uniqueAttribute/type/\{typeName} to get guid of
> the hive table
> * Call /v2/lineage/\{guid} to get lineage of the hive table
> It seems that support getting lineage by uniqueAttributes will make api more
> easy to use. So we add the follow rest api to \{{LineageREST}}:
> /v2/lineage/uniqueAttribute/type/\{typeName}
> We also add the api to \{{AtlasClientV2}} accordingly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3319) Add api to get lineage by uniqueAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3319:
--
Fix Version/s: (was: 1.1.0)
(was: 2.0.0)
> Add api to get lineage by uniqueAttributes
> --
>
> Key: ATLAS-3319
> URL: https://issues.apache.org/jira/browse/ATLAS-3319
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3319.patch
>
>
> To get lineage of a hive table, we need to go through the following two steps:
> * Call get /v2/entity/bulk/uniqueAttribute/type/\{typeName} to get guid of
> the hive table
> * Call /v2/lineage/\{guid} to get lineage of the hive table
> It seems that support getting lineage by uniqueAttributes will make api more
> easy to use. So we add the follow rest api to \{{LineageREST}}:
> /v2/lineage/uniqueAttribute/type/\{typeName}
> We also add the api to \{{AtlasClientV2}} accordingly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3319) Add api to get lineage by uniqueAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3319:
--
Attachment: ATLAS-3319.patch
> Add api to get lineage by uniqueAttributes
> --
>
> Key: ATLAS-3319
> URL: https://issues.apache.org/jira/browse/ATLAS-3319
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.1.0, 2.0.0
>
> Attachments: ATLAS-3319.patch
>
>
> To get lineage of a hive table, we need to go through the following two steps:
> * Call get /v2/entity/bulk/uniqueAttribute/type/\{typeName} to get guid of
> the hive table
> * Call /v2/lineage/\{guid} to get lineage of the hive table
> It seems that support getting lineage by uniqueAttributes will make api more
> easy to use. So we add the follow rest api to \{{LineageREST}}:
> /v2/lineage/uniqueAttribute/type/\{typeName}
> We also add the api to \{{AtlasClientV2}} accordingly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-3319) Add api to get lineage by uniqueAttributes
Jiaqi Shan created ATLAS-3319:
-
Summary: Add api to get lineage by uniqueAttributes
Key: ATLAS-3319
URL: https://issues.apache.org/jira/browse/ATLAS-3319
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: 2.0.0, 1.1.0
Reporter: Jiaqi Shan
Fix For: 2.0.0, 1.1.0
To get lineage of a hive table, we need to go through the following two steps:
* Call get /v2/entity/bulk/uniqueAttribute/type/\{typeName} to get guid of the
hive table
* Call /v2/lineage/\{guid} to get lineage of the hive table
It seems that support getting lineage by uniqueAttributes will make api more
easy to use. So we add the follow rest api to \{{LineageREST}}:
/v2/lineage/uniqueAttribute/type/\{typeName}
We also add the api to \{{AtlasClientV2}} accordingly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3281) Allow excluding lineage related to deleted entities
[
https://issues.apache.org/jira/browse/ATLAS-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3281:
--
Description:
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is displayed as
follow:
!screenshot-1.png!
When {{excludeDeletedEndpoints}} is set to true, the lineage is displayed as
follow:
!screenshot-2.png!
was:
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
follow:
!screenshot-1.png!
When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
follow:
!screenshot-2.png!
> Allow excluding lineage related to deleted entities
> ---
>
> Key: ATLAS-3281
> URL: https://issues.apache.org/jira/browse/ATLAS-3281
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: ATLAS-3281.patch, screenshot-1.png, screenshot-2.png
>
>
> If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}}
> will still return hive_process related to the table. But in some scenarios,
> we don't want to display hive_process related to deleted entities in web UI.
> So we propose to add a parameter {{excludeDeletedEndpoints}} to
> {{getLineageQuery.getLineageQuery}}. When set to true, the function will
> exclude lineages related to deleted entities. We also add the parameter to
> {{AtlasLineageService}},
> {{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
> {{LineageREST}} accordingly.
> When {{excludeDeletedEndpoints}} is set to false, the lineage is displayed as
> follow:
> !screenshot-1.png!
> When {{excludeDeletedEndpoints}} is set to true, the lineage is displayed as
> follow:
> !screenshot-2.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: alter_lineage.jpg
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.jpg, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
>
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!alter_lineage.jpg!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.jpg, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !alter_lineage.jpg!
>
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
>
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: alter_lineage.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: 2.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!alter_lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: lineage.png
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !alter_lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: test.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !alter_lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!alter_lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!test.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.png, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !alter_lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: alter_lineage.png
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.png, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !test.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: 5.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !test.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!test.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!5.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !test.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: test.png
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !5.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: screenshot-1.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, 5.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !5.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!5.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
(ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
!5.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, 5.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !5.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
(ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
!5.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
(ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, 5.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> (ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
> !5.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: 5.png
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, 5.png, screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> (ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
(ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
!screenshot-1.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!screenshot-1.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> (ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
> !screenshot-1.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3281) Allow excluding lineage related to deleted entities
[
https://issues.apache.org/jira/browse/ATLAS-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3281:
--
Description:
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
follow:
!screenshot-1.png!
When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
follow:
!screenshot-2.png!
was:
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
follow:
When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
follow:
> Allow excluding lineage related to deleted entities
> ---
>
> Key: ATLAS-3281
> URL: https://issues.apache.org/jira/browse/ATLAS-3281
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}}
> will still return hive_process related to the table. But in some scenarios,
> we don't want to display hive_process related to deleted entities in web UI.
> So we propose to add a parameter {{excludeDeletedEndpoints}} to
> {{getLineageQuery.getLineageQuery}}. When set to true, the function will
> exclude lineages related to deleted entities. We also add the parameter to
> {{AtlasLineageService}},
> {{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
> {{LineageREST}} accordingly.
> When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
> follow:
> !screenshot-1.png!
> When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
> follow:
> !screenshot-2.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3281) Allow excluding lineage related to deleted entities
[
https://issues.apache.org/jira/browse/ATLAS-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3281:
--
Attachment: screenshot-1.png
> Allow excluding lineage related to deleted entities
> ---
>
> Key: ATLAS-3281
> URL: https://issues.apache.org/jira/browse/ATLAS-3281
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}}
> will still return hive_process related to the table. But in some scenarios,
> we don't want to display hive_process related to deleted entities in web UI.
> So we propose to add a parameter {{excludeDeletedEndpoints}} to
> {{getLineageQuery.getLineageQuery}}. When set to true, the function will
> exclude lineages related to deleted entities. We also add the parameter to
> {{AtlasLineageService}},
> {{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
> {{LineageREST}} accordingly.
> When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
> follow:
> When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
> follow:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-3281) Allow excluding lineage related to deleted entities
Jiaqi Shan created ATLAS-3281:
-
Summary: Allow excluding lineage related to deleted entities
Key: ATLAS-3281
URL: https://issues.apache.org/jira/browse/ATLAS-3281
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: 1.1.0
Reporter: Jiaqi Shan
Fix For: 1.1.0
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
follow:
When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
follow:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: 0001-ATLAS-3280.patch
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !screenshot-1.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!screenshot-1.png! !image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !screenshot-1.png! !image-2019-06-17-16-50-07-833.png|thumbnail!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: image-2019-06-17-16-50-07-833.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !screenshot-1.png! !image-2019-06-17-16-50-07-833.png|thumbnail!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!screenshot-1.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!screenshot-1.png! !image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !screenshot-1.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image.jpg|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: image-2019-06-17-16-50-07-833.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !image-2019-06-17-16-50-07-833.png|thumbnail!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image.jpg|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
[^image-2019-06-17-16-50-07-833.png]
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: image-2019-06-17-16-50-07-833.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !image.jpg|thumbnail!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
[^image-2019-06-17-16-50-07-833.png]
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: image-2019-06-17-16-50-07-833.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> [^image-2019-06-17-16-50-07-833.png]
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
Jiaqi Shan created ATLAS-3280:
-
Summary: Hive Hook creates lineage incorrectly when alter external
table or view
Key: ATLAS-3280
URL: https://issues.apache.org/jira/browse/ATLAS-3280
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 2.0.0, 1.1.0, 1.0.0
Reporter: Jiaqi Shan
Fix For: 2.0.0, 1.1.0, 1.0.0
Attachments: image-2019-06-17-16-50-07-833.png
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3140) Hive Hook throws InvalidTableException when getting metadata of _dummy_database._dummy_table
[
https://issues.apache.org/jira/browse/ATLAS-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3140:
--
Description:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dummy table. So we use
{{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
calling {{BaseHiveEvent.toAtlasEntity()}}
was:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
{{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
calling {{BaseHiveEvent.toAtlasEntity()}}
> Hive Hook throws InvalidTableException when getting metadata of
>
[jira] [Updated] (ATLAS-3140) Hive Hook throws InvalidTableException when getting metadata of _dummy_database._dummy_table
[
https://issues.apache.org/jira/browse/ATLAS-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3140:
--
Attachment: ATLAS-3140.patch
> Hive Hook throws InvalidTableException when getting metadata of
> _dummy_database._dummy_table
>
>
> Key: ATLAS-3140
> URL: https://issues.apache.org/jira/browse/ATLAS-3140
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Critical
> Fix For: 1.0.0, 1.1.0
>
> Attachments: ATLAS-3140.patch
>
>
> When we run sql like 'select 1+1', following error is logged in hive.log:
> {code:java}
> 2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167))
> - HiveHook.run(): failed to process operation QUERY
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> _dummy_table
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
> The exception is caused by getting metadata from a dual table. So we use
> {{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
> calling {{BaseHiveEvent.toAtlasEntity()}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3140) Hive Hook throws InvalidTableException when getting metadata of _dummy_database._dummy_table
[
https://issues.apache.org/jira/browse/ATLAS-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3140:
--
Summary: Hive Hook throws InvalidTableException when getting metadata of
_dummy_database._dummy_table (was: Hive Hook throws InvalidTableException when
select from _dummy_database._dummy_table)
> Hive Hook throws InvalidTableException when getting metadata of
> _dummy_database._dummy_table
>
>
> Key: ATLAS-3140
> URL: https://issues.apache.org/jira/browse/ATLAS-3140
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Critical
> Fix For: 1.0.0, 1.1.0
>
>
> When we run sql like 'select 1+1', following error is logged in hive.log:
> {code:java}
> 2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167))
> - HiveHook.run(): failed to process operation QUERY
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> _dummy_table
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
> The exception is caused by getting metadata from a dual table. So we use
> {{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
> calling {{BaseHiveEvent.toAtlasEntity()}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3140) Hive Hook throws InvalidTableException when select from _dummy_database._dummy_table
[
https://issues.apache.org/jira/browse/ATLAS-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3140:
--
Description:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
{{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
calling {{BaseHiveEvent.toAtlasEntity()}}
was:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
{{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
calling {{BaseHiveEvent.toAtlasEntit}}
> Hive Hook throws InvalidTableException when select from
> _dummy_database._dummy_table
>
[jira] [Updated] (ATLAS-3140) Hive Hook throws InvalidTableException when select from _dummy_database._dummy_table
[
https://issues.apache.org/jira/browse/ATLAS-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3140:
--
Description:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
{{Entity.isDummy()}} to judge whether the entity is dummy when Hive Hook
calling {{BaseHiveEvent.toAtlasEntit}}
was:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
Entity.isDummy() to judge whether the entity is dummy when Hive Hook calling
BaseHiveEvent.toAtlasEntity()
> Hive Hook throws InvalidTableException when select from
> _dummy_database._dummy_table
>
[jira] [Updated] (ATLAS-3140) Hive Hook throws InvalidTableException when select from _dummy_database._dummy_table
[
https://issues.apache.org/jira/browse/ATLAS-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3140:
--
Description:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
Entity.isDummy() to judge whether the entity is dummy when Hive Hook calling
BaseHiveEvent.toAtlasEntity()
was:
Hive Hook throws InvalidTableException when select from
_dummy_database._dummy_table
{panel:title=我的标题}
Hive Hook throws InvalidTableException when select from
_dummy_database._dummy_table
{panel}
Hive Hook throws InvalidTableException when select from
_dummy_database._dummy_table
> Hive Hook throws InvalidTableException when select from
> _dummy_database._dummy_table
>
>
> Key: ATLAS-3140
> URL: https://issues.apache.org/jira/browse/ATLAS-3140
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Critical
> Fix For: 1.0.0, 1.1.0
>
>
> When we run sql like 'select 1+1', following error is logged in hive.log:
> {code:java}
> 2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167))
> - HiveHook.run(): failed to process operation QUERY
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> _dummy_table
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
> at
> org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
> at
> org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
> at
[jira] [Updated] (ATLAS-3140) Hive Hook throws InvalidTableException when select from _dummy_database._dummy_table
[
https://issues.apache.org/jira/browse/ATLAS-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3140:
--
Description:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
Entity.isDummy() to judge whether the entity is dummy when Hive Hook calling
BaseHiveEvent.toAtlasEntity()
was:
When we run sql like 'select 1+1', following error is logged in hive.log:
{code:java}
2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) -
HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
_dummy_table
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
at
org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
at
org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
The exception is caused by getting metadata from a dual table. So we use
Entity.isDummy() to judge whether the entity is dummy when Hive Hook calling
BaseHiveEvent.toAtlasEntity()
> Hive Hook throws InvalidTableException when select from
> _dummy_database._dummy_table
>
[jira] [Created] (ATLAS-3140) Hive Hook throws InvalidTableException when select from _dummy_database._dummy_table
Jiaqi Shan created ATLAS-3140:
-
Summary: Hive Hook throws InvalidTableException when select from
_dummy_database._dummy_table
Key: ATLAS-3140
URL: https://issues.apache.org/jira/browse/ATLAS-3140
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: 1.1.0, 1.0.0
Reporter: Jiaqi Shan
Fix For: 1.1.0, 1.0.0
Hive Hook throws InvalidTableException when select from
_dummy_database._dummy_table
{panel:title=我的标题}
Hive Hook throws InvalidTableException when select from
_dummy_database._dummy_table
{panel}
Hive Hook throws InvalidTableException when select from
_dummy_database._dummy_table
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-3109) Allow ignoring relationshipAttributes for AtlasEntityStore.getByUniqueAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806338#comment-16806338
]
Jiaqi Shan commented on ATLAS-3109:
---
Thanks [~madhan.neethiraj] for review!
> Allow ignoring relationshipAttributes for
> AtlasEntityStore.getByUniqueAttributes
>
>
> Key: ATLAS-3109
> URL: https://issues.apache.org/jira/browse/ATLAS-3109
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
> Attachments: ATLAS-3109-2.patch,
> ATLAS-3109-Allow-ignoring-relationshipAttributes-for.patch
>
>
> We encountered a problem when using
> {{AtlasEntityStore.getByUniqueAttributes}}. One Hive database has 3000+
> tables, getting metadata of the database by
> {{AtlasEntityStore.getByUniqueAttributes}} costs more than 1 seconds.
> After investigation, we realized that
> {{AtlasEntityStore.getByUniqueAttributes}} wil return all attributes and
> relationshipAttributes of the entity. In actuality we need'n to get
> relationshiAttributes like tables.
> We propose to add a parameter {{ignoreRelationship}} to
> {{AtlasEntityStore.getByUniqueAttributes}}. When set to true, the functions
> only return attributes of the entity and ignore relationshipAttributes. We
> also add the parameter to {{AtlasClientV2}} and {{EntityREST}} accordingly.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-3109) Allow ignoring relationshipAttributes for AtlasEntityStore.getByUniqueAttributes
Jiaqi Shan created ATLAS-3109:
-
Summary: Allow ignoring relationshipAttributes for
AtlasEntityStore.getByUniqueAttributes
Key: ATLAS-3109
URL: https://issues.apache.org/jira/browse/ATLAS-3109
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: trunk
Reporter: Jiaqi Shan
We encountered a problem when using {{AtlasEntityStore.getByUniqueAttributes}}.
One Hive database has 3000+ tables, getting metadata of the database by
{{AtlasEntityStore.getByUniqueAttributes}} costs more than 1 seconds.
After investigation, we realized that
{{AtlasEntityStore.getByUniqueAttributes}} wil return all attributes and
relationshipAttributes of the entity. In actuality we need'n to get
relationshiAttributes like tables.
We propose to add a parameter {{ignoreRelationship}} to
{{AtlasEntityStore.getByUniqueAttributes}}. When set to true, the functions
only return attributes of the entity and ignore relationshipAttributes. We also
add the parameter to {{AtlasClientV2}} and {{EntityREST}} accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2955) Support multithreading NotificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2955:
--
Issue Type: Wish (was: Improvement)
> Support multithreading NotificationHookConsumer
> ---
>
> Key: ATLAS-2955
> URL: https://issues.apache.org/jira/browse/ATLAS-2955
> Project: Atlas
> Issue Type: Wish
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> Our Hive clients will send more than 16000 Kafka messages to Atlas
> every hour, which will cause Kafka consumer lag. We find that Atlas only
> allows to create single instance for {{NotificationConsumer}} in version
> 1.0.0.
> Will Atlas support multithreadind {{NotificationHookConsumer}} to
> increase write performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2955) Support multithreading NotificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2955:
--
Summary: Support multithreading NotificationHookConsumer (was: Support
multithreading notificationHookConsumer)
> Support multithreading NotificationHookConsumer
> ---
>
> Key: ATLAS-2955
> URL: https://issues.apache.org/jira/browse/ATLAS-2955
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> Our Hive clients will send more than 16000 Kafka messages to Atlas
> every hour, which will cause Kafka consumer lag. We find that Atlas only
> allows to create single instance for {{NotificationConsumer}} in version
> 1.0.0.
> Will Atlas support multithreadind {{NotificationHookConsumer}} to
> increase write performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2955) Support multithreading notificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2955:
--
Description: Our Hive clients will send more than 16000 Kafka messages to
Atlas every hour, which will cause Kafka consumer lag. We find that Atlas only
allows to create single instance for {{NotificationConsumer}} in version 1.0.0.
Will Atlas support multithreadind {{NotificationHookConsumer}} to increase
write performance later? (was: Our Hive clients will send more than 16000
Kafka messages to Atlas every hour, which will cause Kafka consumer lag. We
find that Atlas only allow to create single instance for
{{NotificationConsumer. }}in version 1.0.0. Will Atlas support multithreading
{{notificationHookConsumer}} to increase write performance later?)
> Support multithreading notificationHookConsumer
> ---
>
> Key: ATLAS-2955
> URL: https://issues.apache.org/jira/browse/ATLAS-2955
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> Our Hive clients will send more than 16000 Kafka messages to Atlas every
> hour, which will cause Kafka consumer lag. We find that Atlas only allows to
> create single instance for {{NotificationConsumer}} in version 1.0.0. Will
> Atlas support multithreadind {{NotificationHookConsumer}} to increase write
> performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2955) Support multithreading notificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2955:
--
Description:
Our Hive clients will send more than 16000 Kafka messages to Atlas every
hour, which will cause Kafka consumer lag. We find that Atlas only allows to
create single instance for {{NotificationConsumer}} in version 1.0.0.
Will Atlas support multithreadind {{NotificationHookConsumer}} to
increase write performance later?
was:Our Hive clients will send more than 16000 Kafka messages to Atlas every
hour, which will cause Kafka consumer lag. We find that Atlas only allows to
create single instance for {{NotificationConsumer}} in version 1.0.0. Will
Atlas support multithreadind {{NotificationHookConsumer}} to increase write
performance later?
> Support multithreading notificationHookConsumer
> ---
>
> Key: ATLAS-2955
> URL: https://issues.apache.org/jira/browse/ATLAS-2955
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> Our Hive clients will send more than 16000 Kafka messages to Atlas
> every hour, which will cause Kafka consumer lag. We find that Atlas only
> allows to create single instance for {{NotificationConsumer}} in version
> 1.0.0.
> Will Atlas support multithreadind {{NotificationHookConsumer}} to
> increase write performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2955) Support multithreading notificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2955:
--
Description:
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for
{\{NotificationConsumer}}
in version 1.0.0. Will Atlas support multithreading
\{{notificationHookConsumer}} to increase write performance later?
was:
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for
{NotificationConsumer}
in version 1.0.0. Will Atlas support multithreading \{notificationHookConsumer}
to increase write performance later?
> Support multithreading notificationHookConsumer
> ---
>
> Key: ATLAS-2955
> URL: https://issues.apache.org/jira/browse/ATLAS-2955
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> Our Hive clients will send more than 16000 Kafka messages to Atlas every
> hour, which will cause Kafka consumer lag. We find that Atlas only allow to
> create single instance for
> {\{NotificationConsumer}}
> in version 1.0.0. Will Atlas support multithreading
> \{{notificationHookConsumer}} to increase write performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2955) Support multithreading notificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2955:
--
Description:
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for
{{NotificationConsumer. }}in version 1.0.0. Will Atlas support multithreading
{{notificationHookConsumer}} to increase write performance later?
was:
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for
{\{NotificationConsumer}}
in version 1.0.0. Will Atlas support multithreading
\{{notificationHookConsumer}} to increase write performance later?
> Support multithreading notificationHookConsumer
> ---
>
> Key: ATLAS-2955
> URL: https://issues.apache.org/jira/browse/ATLAS-2955
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> Our Hive clients will send more than 16000 Kafka messages to Atlas every
> hour, which will cause Kafka consumer lag. We find that Atlas only allow to
> create single instance for
> {{NotificationConsumer. }}in version 1.0.0. Will Atlas support multithreading
> {{notificationHookConsumer}} to increase write performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-2955) Support multithreading notificationHookConsumer
Jiaqi Shan created ATLAS-2955:
-
Summary: Support multithreading notificationHookConsumer
Key: ATLAS-2955
URL: https://issues.apache.org/jira/browse/ATLAS-2955
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: 1.0.0
Reporter: Jiaqi Shan
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for {{
NotificationConsumer }} in version 1.0.0. Will Atlas support multithreading \{{
notificationHookConsumer }} to increase write performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2955) Support multithreading notificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2955:
--
Description:
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for {
NotificationConsumer } in version 1.0.0. Will Atlas support multithreading \{
notificationHookConsumer } to increase write performance later?
was:
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for {{
NotificationConsumer }} in version 1.0.0. Will Atlas support multithreading \{{
notificationHookConsumer }} to increase write performance later?
> Support multithreading notificationHookConsumer
> ---
>
> Key: ATLAS-2955
> URL: https://issues.apache.org/jira/browse/ATLAS-2955
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> Our Hive clients will send more than 16000 Kafka messages to Atlas every
> hour, which will cause Kafka consumer lag. We find that Atlas only allow to
> create single instance for {
> NotificationConsumer } in version 1.0.0. Will Atlas support multithreading \{
> notificationHookConsumer } to increase write performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2955) Support multithreading notificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2955:
--
Description:
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for
{NotificationConsumer}
in version 1.0.0. Will Atlas support multithreading \{notificationHookConsumer}
to increase write performance later?
was:
Our Hive clients will send more than 16000 Kafka messages to Atlas every hour,
which will cause Kafka consumer lag. We find that Atlas only allow to create
single instance for {
NotificationConsumer } in version 1.0.0. Will Atlas support multithreading \{
notificationHookConsumer } to increase write performance later?
> Support multithreading notificationHookConsumer
> ---
>
> Key: ATLAS-2955
> URL: https://issues.apache.org/jira/browse/ATLAS-2955
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> Our Hive clients will send more than 16000 Kafka messages to Atlas every
> hour, which will cause Kafka consumer lag. We find that Atlas only allow to
> create single instance for
> {NotificationConsumer}
> in version 1.0.0. Will Atlas support multithreading
> \{notificationHookConsumer} to increase write performance later?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
When creating a Hive process, ColumnEntity will update with the value of
TableEntity's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metadata changed in ColumnEntity, the difference of
table's AtlasObjectId is caused by Hive Hook setting a new guid for
TableEntity. So maybe it's not necessary to update Hive column entity in this
instance.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification. But in situation deleting and re-creating the
entity with the same uniqueAttributes, this solution goes wrong.
Is there any other good solution to aviod this problem?
was:
When creating a Hive process, Hive column entity will update with the
value of table's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> When creating a Hive process, ColumnEntity will update with the value of
> TableEntity's AtlasObjectId. There is an example to show the differences
> bwtween CurrentEntity and EntityInStore.
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column',
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
When creating a Hive process, Hive column entity will update with the
value of table's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
was:
When creating a Hive process, ColumnEntity will update with the value of
TableEntity's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metadata changed in ColumnEntity, the difference of
table's AtlasObjectId is caused by Hive Hook setting a new guid for
TableEntity. So maybe it's not necessary to update Hive column entity in this
instance.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification. But in situation deleting and re-creating the
entity with the same uniqueAttributes, this solution goes wrong.
Is there any other good solution to aviod this problem?
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> When creating a Hive process, Hive column entity will update with the
> value of table's AtlasObjectId. There is an example to show the differences
> bwtween CurrentEntity and EntityInStore.
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column',
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
When creating a Hive process, ColumnEntity will update with the value of
TableEntity's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metadata changed in ColumnEntity, the difference of
table's AtlasObjectId is caused by Hive Hook setting a new guid for
TableEntity. So maybe it's not necessary to update Hive column entity in this
instance.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification. But in situation deleting and re-creating the
entity with the same uniqueAttributes, this solution goes wrong.
Is there any other good solution to aviod this problem?
was:
When creating a Hive process, ColumnEntity will update with the value of
TableEntity's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metadata changed in ColumnEntity, the difference of
table's AtlasObjectId is caused by Hive Hook setting a new guid for
TableEntity. So I think maybe it's not necessary to update Hive column entity
in this instance.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification. But in situation deleting and re-creating the
entity with the same uniqueAttributes, this solution goes wrong.
Is there any other good solution to aviod this problem?
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> When creating a Hive process, ColumnEntity will update with the value of
> TableEntity's AtlasObjectId. There is an example to show the differences
> bwtween CurrentEntity and EntityInStore.
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
>
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
When creating a Hive process, ColumnEntity will update with the value of
TableEntity's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metadata changed in ColumnEntity, the difference of
table's AtlasObjectId is caused by Hive Hook setting a new guid for
TableEntity. So I think maybe it's not necessary to update Hive column entity
in this instance.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification. But in situation deleting and re-creating the
entity with the same uniqueAttributes, this solution goes wrong.
Is there any other good solution to aviod this problem?
was:
When creating a Hive process, Hive column entity will update with the
value of table's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.Is there any other good solution
to aviod this problem?
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> When creating a Hive process, ColumnEntity will update with the value of
> TableEntity's AtlasObjectId. There is an example to show the differences
> bwtween CurrentEntity and EntityInStore.
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column',
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
When creating a Hive process, Hive column entity will update with the
value of table's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.Is there any other good solution
to aviod this problem?
was:
When creating a Hive process, Hive column entity will update with the
value of table's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metedata changed in columnEntity, so I think maybe it's
unnecessary to update columnEntity if only need to update the AtlasObJectId.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.Is there any other good solution
to aviod this problem?
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> When creating a Hive process, Hive column entity will update with the
> value of table's AtlasObjectId. There is an example to show the differences
> bwtween CurrentEntity and EntityInStore.
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
> position:1, type:int,
>
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
When creating a Hive process, Hive column entity will update with the
value of table's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metedata changed in columnEntity, so I think maybe it's
unnecessary to update columnEntity if only need to update the AtlasObJectId.
We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.Is there any other good solution
to aviod this problem?
was:
When creating a Hive process, Hive column entity will update with the
value of table's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> When creating a Hive process, Hive column entity will update with the
> value of table's AtlasObjectId. There is an example to show the differences
> bwtween CurrentEntity and EntityInStore.
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
> position:1, type:int,
>
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
When creating a Hive process, Hive column entity will update with the
value of table's AtlasObjectId. There is an example to show the differences
bwtween CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
was:
{{ When creating a Hive process, columnEntity wil update with the value of
table's AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.}}
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> When creating a Hive process, Hive column entity will update with the
> value of table's AtlasObjectId. There is an example to show the differences
> bwtween CurrentEntity and EntityInStore.
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
> position:1, type:int,
>
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
{{ When creating a Hive process, columnEntity wil update with the value of
table's AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.}}
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
was:
{{ When creating a Hive process, columnEntity wil update with the value of
table's AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.}}
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> {{ When creating a Hive process, columnEntity wil update with the value of
> table's AtlasObjectId. There is an example to show the differences bwtween
> CurrentEntity and EntityInStore.}}
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
> position:1, type:int,
>
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
{{ When creating a Hive process, columnEntity wil update with the value of
table's AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.}}
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
was:
{{ When creating a Hive process, columnEntity wil update with the value of
table's AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.}}
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> {{ When creating a Hive process, columnEntity wil update with the value of
> table's AtlasObjectId. There is an example to show the differences bwtween
> CurrentEntity and EntityInStore.}}
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
> position:1, type:int,
>
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
{{ When creating a Hive process, columnEntity wil update with the value of
table's AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.}}
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
was:
{{ When creating a Hive process, columnEntity wil update with the value of
table's AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.}}
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> {{ When creating a Hive process, columnEntity wil update with the value of
> table's AtlasObjectId. There is an example to show the differences bwtween
> CurrentEntity and EntityInStore.}}
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
> position:1, type:int,
>
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
{{ When creating a Hive process, columnEntity wil update with the value of
table's AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.}}
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
{{ Actually there is no metedata changed in columnEntity, so I think maybe
it's unnecessary to update columnEntity if only need to update the
AtlasObJectId.}}
{{ We propose to add a LRU cache to skip updating the same entitiy which was
sent in an earlier notification and it works.}}{{Is there any other good
solution to aviod this problem?}}
was:
When creating a Hive process, columnEntity wil update with the value of table's
AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metedata changed in columnEntity, so I think maybe it's
unnecessary to update columnEntity if only need to update the AtlasObJectId.
We propose to add a LRU cache to skip updating the same entitiy which was sent
in an earlier notification and it works.
Is there any other good solution to aviod this problem?
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> {{ When creating a Hive process, columnEntity wil update with the value of
> table's AtlasObjectId. There is an example to show the differences bwtween
> CurrentEntity and EntityInStore.}}
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
> position:1, type:int,
> {color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
>
[jira] [Updated] (ATLAS-2916) Unnecessary entity update causes by different AtlasObjectId
[
https://issues.apache.org/jira/browse/ATLAS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-2916:
--
Description:
When creating a Hive process, columnEntity wil update with the value of table's
AtlasObjectId. There is an example to show the differences bwtween
CurrentEntity and EntityInStore.
{panel:title=CurrentEntity}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
typeName='hive_table',
uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=null, createdBy='null', updatedBy='null', createTime=null,
updateTime=null, version=0, relationshipAttributes=[], classifications=[],
meanings=[]}
{panel}
{panel:title=EntityInStore}
AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
position:1, type:int,
{color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
typeName='hive_table',
uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
version=0, relationshipAttributes=[], classifications=[], meanings=[]}
{panel}
Actually there is no metedata changed in columnEntity, so I think maybe it's
unnecessary to update columnEntity if only need to update the AtlasObJectId.
We propose to add a LRU cache to skip updating the same entitiy which was sent
in an earlier notification and it works.
Is there any other good solution to aviod this problem?
was:
When creating a Hive process, columnEntity wil update with the value of table's
AtlasObjectId. There is an example to show the differences bwtween localEntity
and EntityInsore.
Actually there is no metedata changed in columnEntity, so I think it's
unnecessary to update columnEntity if only need to update the AtlasObJectId.
I wonder if there is any condition need to eqauls two AtlasObjectId which guid
is null?
> Unnecessary entity update causes by different AtlasObjectId
> ---
>
> Key: ATLAS-2916
> URL: https://issues.apache.org/jira/browse/ATLAS-2916
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Jiaqi Shan
>Priority: Minor
>
> When creating a Hive process, columnEntity wil update with the value of
> table's AtlasObjectId. There is an example to show the differences bwtween
> CurrentEntity and EntityInStore.
> {panel:title=CurrentEntity}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, comment:t, position:1, type:int,
> table:{color:#d04437}AtlasObjectId\{guid='-17020754238878791',
> typeName='hive_table',
> uniqueAttributes={qualifiedName:b.yingbang@bdyf}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=null, createdBy='null', updatedBy='null', createTime=null,
> updateTime=null, version=0, relationshipAttributes=[], classifications=[],
> meanings=[]}
> {panel}
> {panel:title=EntityInStore}
> AtlasEntity{AtlasStruct{typeName='hive_column', attributes=[owner:bi_sh,
> qualifiedName:bi.yingbang.t@bdyf, name:t, description:null, comment:t,
> position:1, type:int,
> {color:#d04437}table:AtlasObjectId\{guid='da35aff2-9851-499d-99cf-f1fbafb6e92b',
> typeName='hive_table',
> uniqueAttributes={}}{color}]}guid='431c8847-8fd2-454d-b77a-19aeef0d6b9b',
> status=ACTIVE, createdBy='bi_sh', updatedBy='bi_sh',
> createTime=2018-10-09T11:26:51.685Z, updateTime=2018-10-09T11:26:51.685Z,
> version=0, relationshipAttributes=[], classifications=[], meanings=[]}
> {panel}
> Actually there is no metedata changed in columnEntity, so I think maybe it's
> unnecessary to update columnEntity if only need to update the AtlasObJectId.
> We propose to add a LRU cache to skip updating the same entitiy which was
> sent in an earlier notification and it works.
> Is there any other good solution to aviod this problem?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
