[jira] [Commented] (ATLAS-541) Soft deletes
[
https://issues.apache.org/jira/browse/ATLAS-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223833#comment-15223833
]
Shwetha G S commented on ATLAS-541:
---
{quote}
Shouldn't Atlas consider giving an option for the users to either go for 'soft'
or 'hard' delete
{quote}
We can give an option if there is a usecase. From governance perspective, it
makes sense to have soft deletes on metadata. So, I want to understand your
usecases
{quote}
As you right pointed out, report queries (where a subset of fields from a type
source are queried) would probably return results from deleted entities (if the
DSL query translation does not inject an extra predicate to filter out deleted
entities). In case of 'soft' delete, there should be an option to 'purge'
deleted entities of a given type or all types.
{quote}
Yes, the plan is to push the status predicate to the gremlin query. But,
because of short time that we have now, we will do predicate push later. Yes,
we will also need purge option
If there was predicate push in search, would you still need an option for hard
delete? Note that soft delete will increase the data size.
> Soft deletes
>
>
> Key: ATLAS-541
> URL: https://issues.apache.org/jira/browse/ATLAS-541
> Project: Atlas
> Issue Type: New Feature
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>
> We don't have graph versioning currently and hard deletes are not acceptable
> for data governance. This jira tracks the proposal for soft deletes which can
> mark an entity as deleted and by default search should return only active
> entities. However, there should be an option to retrieve deleted entities
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-541) Soft deletes
[
https://issues.apache.org/jira/browse/ATLAS-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223868#comment-15223868
]
Shwetha G S commented on ATLAS-541:
---
{quote}
We use a metadata repository not just for business-metadata like
catalog/glossary, but as an operational metadata store as well. In cases of
operational metadata, we often tend to persist metadata more in transient form
(like staging), and finally publish something which is curated/approved as the
final copy for use with data lineage. So expect lot of such staged metadata
which needs to be discarded permanently from the system.
{quote}
Would it help if we enable/disable soft delete at type level(depending on the
typeName in type system), instead of at atlas system level?
{quote}
In my opinion, in the absence of versioning, a decent audit trail of an entity
state during its lifecycle adds to the needs of regulartory governance
requirements. But I am not sure how much a 'soft' delete can satisfy such
governance requirements.
{quote}
You mean the audit trail before the entity is deleted? Haven't you had any asks
of ability to search on deleted entities or the important entity types are
never deleted?
> Soft deletes
>
>
> Key: ATLAS-541
> URL: https://issues.apache.org/jira/browse/ATLAS-541
> Project: Atlas
> Issue Type: New Feature
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>
> We don't have graph versioning currently and hard deletes are not acceptable
> for data governance. This jira tracks the proposal for soft deletes which can
> mark an entity as deleted and by default search should return only active
> entities. However, there should be an option to retrieve deleted entities
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-541) Soft deletes
[
https://issues.apache.org/jira/browse/ATLAS-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223868#comment-15223868
]
Shwetha G S edited comment on ATLAS-541 at 4/4/16 12:02 PM:
{quote}
We use a metadata repository not just for business-metadata like
catalog/glossary, but as an operational metadata store as well. In cases of
operational metadata, we often tend to persist metadata more in transient form
(like staging), and finally publish something which is curated/approved as the
final copy for use with data lineage. So expect lot of such staged metadata
which needs to be discarded permanently from the system.
{quote}
Would it help if we enable/disable soft delete at type level(depending on the
typeName in type system), instead of at atlas system level?
{quote}
In my opinion, in the absence of versioning, a decent audit trail of an entity
state during its lifecycle adds to the needs of regulartory governance
requirements. But I am not sure how much a 'soft' delete can satisfy such
governance requirements.
{quote}
You mean the audit trail before the entity is deleted? Haven't you come across
any asks where the users want to search on deleted entities? Or is it that the
users manage it themselves using deleted attribute in client space?
was (Author: shwethags):
{quote}
We use a metadata repository not just for business-metadata like
catalog/glossary, but as an operational metadata store as well. In cases of
operational metadata, we often tend to persist metadata more in transient form
(like staging), and finally publish something which is curated/approved as the
final copy for use with data lineage. So expect lot of such staged metadata
which needs to be discarded permanently from the system.
{quote}
Would it help if we enable/disable soft delete at type level(depending on the
typeName in type system), instead of at atlas system level?
{quote}
In my opinion, in the absence of versioning, a decent audit trail of an entity
state during its lifecycle adds to the needs of regulartory governance
requirements. But I am not sure how much a 'soft' delete can satisfy such
governance requirements.
{quote}
You mean the audit trail before the entity is deleted? Haven't you had any asks
of ability to search on deleted entities or the important entity types are
never deleted?
> Soft deletes
>
>
> Key: ATLAS-541
> URL: https://issues.apache.org/jira/browse/ATLAS-541
> Project: Atlas
> Issue Type: New Feature
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>
> We don't have graph versioning currently and hard deletes are not acceptable
> for data governance. This jira tracks the proposal for soft deletes which can
> mark an entity as deleted and by default search should return only active
> entities. However, there should be an option to retrieve deleted entities
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-540) API to retrieve entity version events
[ https://issues.apache.org/jira/browse/ATLAS-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-540: -- Attachment: ATLAS-540-v2.patch Re-based patch > API to retrieve entity version events > - > > Key: ATLAS-540 > URL: https://issues.apache.org/jira/browse/ATLAS-540 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-540-v2.patch, ATLAS-540.patch > > > We will need an API that will take entity id as input and returns the events > in the decreasing order of timestamp. The API should return n events and > return the timestamp of the next event -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-632) Maven fails to build sqoop-bridge: unable to download dependency jar
[
https://issues.apache.org/jira/browse/ATLAS-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15225695#comment-15225695
]
Shwetha G S commented on ATLAS-632:
---
When I deleted from my local mvn repo and tried building, mvn downloaded the
dependencies correctly:
{noformat}
sqoop-bridge$ rm -rf ~/.m2/repository/org/apache/sqoop/
sqoop-bridge$ mcis
[INFO] Scanning for projects...
[INFO]
[INFO]
[INFO] Building Apache Atlas Sqoop Bridge 0.7-incubating-SNAPSHOT
[INFO]
Downloading:
http://repo1.maven.org/maven2/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195/sqoop-1.4.6.2.3.99.0-195.pom
Downloading:
http://repo.hortonworks.com/content/repositories/releases/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195/sqoop-1.4.6.2.3.99.0-195.pom
Downloaded:
http://repo.hortonworks.com/content/repositories/releases/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195/sqoop-1.4.6.2.3.99.0-195.pom
(2 KB at 1.3 KB/sec)
Downloading:
http://repo1.maven.org/maven2/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195/sqoop-1.4.6.2.3.99.0-195.jar
Downloading:
http://repo.hortonworks.com/content/repositories/releases/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195/sqoop-1.4.6.2.3.99.0-195.jar
Downloaded:
http://repo.hortonworks.com/content/repositories/releases/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195/sqoop-1.4.6.2.3.99.0-195.jar
(997 KB at 183.6 KB/sec)
{noformat}
> Maven fails to build sqoop-bridge: unable to download dependency jar
>
>
> Key: ATLAS-632
> URL: https://issues.apache.org/jira/browse/ATLAS-632
> Project: Atlas
> Issue Type: Bug
>Reporter: David Kantor
>Priority: Critical
>
> When building the sqoop-bridge project on a system that has not previously
> built this project and requires dependencies to be downloaded, the build
> fails when trying to download the pom for
> org.apache.sqoop:sqoop:pom:1.4.6.2.3.99.0-195:
> {{[ERROR] Failed to execute goal on project sqoop-bridge: Could not resolve
> dependencies for project
> org.apache.atlas:sqoop-bridge:jar:0.7-incubating-SNAPSHOT: Failed to collect
> dependencies at org.apache.sqoop:sqoop:jar:1.4.6.2.3.99.0-195: Failed to read
> artifact descriptor for org.apache.sqoop:sqoop:jar:1.4.6.2.3.99.0-195: Could
> not transfer artifact org.apache.sqoop:sqoop:pom:1.4.6.2.3.99.0-195 from/to
> hortonworks.repo (http://repo.hortonworks.com/content/repositories/releases):
> Failed to transfer file:
> http://repo.hortonworks.com/content/repositories/releases/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195/sqoop-1.4.6.2.3.99.0-195.pom.
> Return code is: 500 , ReasonPhrase:Server Error}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-592) Queue for failed hook messages
[ https://issues.apache.org/jira/browse/ATLAS-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15217630#comment-15217630 ] Shwetha G S edited comment on ATLAS-592 at 3/30/16 8:13 AM: AtlasHook is client side, this bug was for failures in atlas server side. Will create another bug for it. Thanks. Created ATLAS-602 was (Author: shwethags): AtlasHook is client side, this bug was for failures in atlas server side. Will create another bug for it. Thanks > Queue for failed hook messages > -- > > Key: ATLAS-592 > URL: https://issues.apache.org/jira/browse/ATLAS-592 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S > Fix For: 0.7-incubating > > > If NotificationHookConsumer fails to process a message, its just logged and > the message is lost. Instead, it should be added to failed queue, possibly > for manual intervention -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-592) Queue for failed hook messages
[ https://issues.apache.org/jira/browse/ATLAS-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15217630#comment-15217630 ] Shwetha G S commented on ATLAS-592: --- AtlasHook is client side, this bug was for failures in atlas server side. Will create another bug for it. Thanks > Queue for failed hook messages > -- > > Key: ATLAS-592 > URL: https://issues.apache.org/jira/browse/ATLAS-592 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S > Fix For: 0.7-incubating > > > If NotificationHookConsumer fails to process a message, its just logged and > the message is lost. Instead, it should be added to failed queue, possibly > for manual intervention -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-540) API to retrieve entity version events
[ https://issues.apache.org/jira/browse/ATLAS-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-540: -- Attachment: ATLAS-540.patch > API to retrieve entity version events > - > > Key: ATLAS-540 > URL: https://issues.apache.org/jira/browse/ATLAS-540 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-540.patch > > > We will need an API that will take entity id as input and returns the events > in the decreasing order of timestamp. The API should return n events and > return the timestamp of the next event -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-620) Disable hbase based entity audit
[ https://issues.apache.org/jira/browse/ATLAS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-620: -- Attachment: (was: ATLAS-620.patch) > Disable hbase based entity audit > > > Key: ATLAS-620 > URL: https://issues.apache.org/jira/browse/ATLAS-620 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > > Currently, the default atlas_start requires external hbase setup(for entity > audit) and doesn't work out of the box. As part of ATLAS-498, there will be > standalone hbase that entity audit can use. Till then, switch to in-memory > based solution so that atlas_start works out of the box -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-620) Disable hbase based entity audit
[ https://issues.apache.org/jira/browse/ATLAS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-620: -- Attachment: ATLAS-620-v2.patch > Disable hbase based entity audit > > > Key: ATLAS-620 > URL: https://issues.apache.org/jira/browse/ATLAS-620 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-620-v2.patch, ATLAS-620.patch > > > Currently, the default atlas_start requires external hbase setup(for entity > audit) and doesn't work out of the box. As part of ATLAS-498, there will be > standalone hbase that entity audit can use. Till then, switch to in-memory > based solution so that atlas_start works out of the box -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-620) Disable hbase based entity audit
[ https://issues.apache.org/jira/browse/ATLAS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-620: -- Attachment: (was: ATLAS-620-v2.patch) > Disable hbase based entity audit > > > Key: ATLAS-620 > URL: https://issues.apache.org/jira/browse/ATLAS-620 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > > Currently, the default atlas_start requires external hbase setup(for entity > audit) and doesn't work out of the box. As part of ATLAS-498, there will be > standalone hbase that entity audit can use. Till then, switch to in-memory > based solution so that atlas_start works out of the box -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-620) Disable hbase based entity audit
[ https://issues.apache.org/jira/browse/ATLAS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-620: -- Attachment: ATLAS-620-v3.patch > Disable hbase based entity audit > > > Key: ATLAS-620 > URL: https://issues.apache.org/jira/browse/ATLAS-620 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-620-v3.patch > > > Currently, the default atlas_start requires external hbase setup(for entity > audit) and doesn't work out of the box. As part of ATLAS-498, there will be > standalone hbase that entity audit can use. Till then, switch to in-memory > based solution so that atlas_start works out of the box -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-620) Disable hbase based entity audit
[ https://issues.apache.org/jira/browse/ATLAS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-620: -- Attachment: ATLAS-620-v4.patch Removed HBaseBasedAuditRepository as ActiveStateChangeHandler. Thanks Hemanth for pointing it out > Disable hbase based entity audit > > > Key: ATLAS-620 > URL: https://issues.apache.org/jira/browse/ATLAS-620 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-620-v3.patch, ATLAS-620-v4.patch > > > Currently, the default atlas_start requires external hbase setup(for entity > audit) and doesn't work out of the box. As part of ATLAS-498, there will be > standalone hbase that entity audit can use. Till then, switch to in-memory > based solution so that atlas_start works out of the box -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-620) Disable hbase based entity audit
Shwetha G S created ATLAS-620: - Summary: Disable hbase based entity audit Key: ATLAS-620 URL: https://issues.apache.org/jira/browse/ATLAS-620 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Assignee: Shwetha G S Priority: Critical Fix For: 0.7-incubating Currently, the default atlas_start requires external hbase setup(for entity audit) and doesn't work out of the box. As part of ATLAS-498, there will be standalone hbase that entity audit can use. Till then, switch to in-memory based solution so that atlas_start works out of the box -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-620) Disable hbase based entity audit
[ https://issues.apache.org/jira/browse/ATLAS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-620: -- Attachment: ATLAS-620.patch > Disable hbase based entity audit > > > Key: ATLAS-620 > URL: https://issues.apache.org/jira/browse/ATLAS-620 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-620.patch > > > Currently, the default atlas_start requires external hbase setup(for entity > audit) and doesn't work out of the box. As part of ATLAS-498, there will be > standalone hbase that entity audit can use. Till then, switch to in-memory > based solution so that atlas_start works out of the box -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-610) Add support for Titan 1 / TinkerPop 3
[ https://issues.apache.org/jira/browse/ATLAS-610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-610: -- Assignee: Jeffrey Hagelberg > Add support for Titan 1 / TinkerPop 3 > - > > Key: ATLAS-610 > URL: https://issues.apache.org/jira/browse/ATLAS-610 > Project: Atlas > Issue Type: Task >Affects Versions: trunk >Reporter: Jeffrey Hagelberg >Assignee: Jeffrey Hagelberg > > We need to be able to run Atlas on top of Titan 1.0.0-incubating. This needs > to be done in a way that allows Atlas to continue to support running with > Titan 0.5.4. This is being implemented by adding an abstraction layer > between Atlas and Titan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-610) Add support for Titan 1 / TinkerPop 3
[ https://issues.apache.org/jira/browse/ATLAS-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15224488#comment-15224488 ] Shwetha G S commented on ATLAS-610: --- Done. Thanks > Add support for Titan 1 / TinkerPop 3 > - > > Key: ATLAS-610 > URL: https://issues.apache.org/jira/browse/ATLAS-610 > Project: Atlas > Issue Type: Task >Affects Versions: trunk >Reporter: Jeffrey Hagelberg >Assignee: Jeffrey Hagelberg > > We need to be able to run Atlas on top of Titan 1.0.0-incubating. This needs > to be done in a way that allows Atlas to continue to support running with > Titan 0.5.4. This is being implemented by adding an abstraction layer > between Atlas and Titan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-621) Introduce entity state in Id object
[ https://issues.apache.org/jira/browse/ATLAS-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-621: -- Description: Add entity state with ACTIVE and DELETED. The state should be returned in get entity definition and in search results where entity is returned > Introduce entity state in Id object > --- > > Key: ATLAS-621 > URL: https://issues.apache.org/jira/browse/ATLAS-621 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > > Add entity state with ACTIVE and DELETED. The state should be returned in get > entity definition and in search results where entity is returned -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-622) Modify delete to soft delete
Shwetha G S created ATLAS-622: - Summary: Modify delete to soft delete Key: ATLAS-622 URL: https://issues.apache.org/jira/browse/ATLAS-622 Project: Atlas Issue Type: Sub-task Reporter: Shwetha G S Assignee: Shwetha G S Currently, in entity delete API, the entity and its related entities(composite entities) are deleted and there is no trace of it in the system. Instead, change delete to mark the entities to be deleted with state=DELETED -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-621) Introduce entity state in Id object
[ https://issues.apache.org/jira/browse/ATLAS-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-621: -- Description: Add entity state with ACTIVE and DELETED. The state should be returned in get entity definition and in search results where entity is returned. In entity create, mark the state as ACTIVE (was: Add entity state with ACTIVE and DELETED. The state should be returned in get entity definition and in search results where entity is returned) > Introduce entity state in Id object > --- > > Key: ATLAS-621 > URL: https://issues.apache.org/jira/browse/ATLAS-621 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > > Add entity state with ACTIVE and DELETED. The state should be returned in get > entity definition and in search results where entity is returned. In entity > create, mark the state as ACTIVE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-621) Introduce entity state in Id object
[ https://issues.apache.org/jira/browse/ATLAS-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-621: -- Attachment: ATLAS-621.patch > Introduce entity state in Id object > --- > > Key: ATLAS-621 > URL: https://issues.apache.org/jira/browse/ATLAS-621 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-621.patch > > > Add entity state with ACTIVE and DELETED. The state should be returned in get > entity definition and in search results where entity is returned. In entity > create, mark the state as ACTIVE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-622) Modify delete to soft delete
[ https://issues.apache.org/jira/browse/ATLAS-622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223706#comment-15223706 ] Shwetha G S commented on ATLAS-622: --- [~dkantor], do you have a requirement to have hard delete? Will soft deletes work for you? > Modify delete to soft delete > > > Key: ATLAS-622 > URL: https://issues.apache.org/jira/browse/ATLAS-622 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > > Currently, in entity delete API, the entity and its related > entities(composite entities) are deleted and there is no trace of it in the > system. Instead, change delete to mark the entities to be deleted with > state=DELETED -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-585) NotificationHookConsumer creates new AtlasClient for every message
Shwetha G S created ATLAS-585: - Summary: NotificationHookConsumer creates new AtlasClient for every message Key: ATLAS-585 URL: https://issues.apache.org/jira/browse/ATLAS-585 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Assignee: Shwetha G S Fix For: 0.7-incubating Every AtlasClient instance creates com.sun.jersey.api.client.Client which is expensive. As part of ATLAS-577, NotificationHookConsumer creates new atlas client for every message. This is done to pass the user information(the user who created the message/entities) through the APIs. This wouldn't work in kerberos environment where the credentials are not available on the atlas server. We should probably make the API call as atlas user and pass in the actual user information in another query param(like doas user) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-577) Integrate entity audit with DefaultMetadataService
[ https://issues.apache.org/jira/browse/ATLAS-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208088#comment-15208088 ] Shwetha G S commented on ATLAS-577: --- Created ATLAS-585 to track the user related testing > Integrate entity audit with DefaultMetadataService > -- > > Key: ATLAS-577 > URL: https://issues.apache.org/jira/browse/ATLAS-577 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-577.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-598) runing "mvn install" failed, it shows that some plugins can't be executed
[ https://issues.apache.org/jira/browse/ATLAS-598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15221128#comment-15221128 ] Shwetha G S commented on ATLAS-598: --- This plugin is not part of tests. It probably some issue in your environment. I will resolve this jira for now, re-open if you see the issue again > runing "mvn install" failed, it shows that some plugins can't be executed > - > > Key: ATLAS-598 > URL: https://issues.apache.org/jira/browse/ATLAS-598 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Environment: OS: Ubuntu 14.04 LTS, amd 64 > Apache Maven 3.3.9 > Java version: 1.7.0_95 >Reporter: Ethan Hsieh > Labels: atlas, build, install, maven > Original Estimate: 24h > Remaining Estimate: 24h > > I tried to install Apache Atlas according the Official guide, and typed > "export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m" && mvn clean install". > But the terminal showed that > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) > on project apache-atlas: Error finding remote resources manifests: > /usr/local/atlas/target/maven-shared-archive-resources/META-INF/NOTICE (No > such file or directory) -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > I also typed "mvn clean" first, it would be OK, but then I typed "mvn > install" or "mvn install -U", it would still show Error. > Please help me. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-598) runing "mvn install" failed, it shows that some plugins can't be executed
[ https://issues.apache.org/jira/browse/ATLAS-598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-598. --- Resolution: Cannot Reproduce > runing "mvn install" failed, it shows that some plugins can't be executed > - > > Key: ATLAS-598 > URL: https://issues.apache.org/jira/browse/ATLAS-598 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Environment: OS: Ubuntu 14.04 LTS, amd 64 > Apache Maven 3.3.9 > Java version: 1.7.0_95 >Reporter: Ethan Hsieh > Labels: atlas, build, install, maven > Original Estimate: 24h > Remaining Estimate: 24h > > I tried to install Apache Atlas according the Official guide, and typed > "export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m" && mvn clean install". > But the terminal showed that > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) > on project apache-atlas: Error finding remote resources manifests: > /usr/local/atlas/target/maven-shared-archive-resources/META-INF/NOTICE (No > such file or directory) -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > I also typed "mvn clean" first, it would be OK, but then I typed "mvn > install" or "mvn install -U", it would still show Error. > Please help me. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-609) Secure calls to HBase
Shwetha G S created ATLAS-609: - Summary: Secure calls to HBase Key: ATLAS-609 URL: https://issues.apache.org/jira/browse/ATLAS-609 Project: Atlas Issue Type: Sub-task Reporter: Shwetha G S -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-669) GraphHelper.getEdgeForLabel() can use edge index
Shwetha G S created ATLAS-669: - Summary: GraphHelper.getEdgeForLabel() can use edge index Key: ATLAS-669 URL: https://issues.apache.org/jira/browse/ATLAS-669 Project: Atlas Issue Type: Bug Reporter: Shwetha G S GraphHelper.getEdgeForLabel() currently goes through all edges and gets the latest deleted edge. This can be optimised and use edge index and ordering -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-543) Entity Instance requests should not require ID element for new Entities
[
https://issues.apache.org/jira/browse/ATLAS-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240998#comment-15240998
]
Shwetha G S commented on ATLAS-543:
---
+1
> Entity Instance requests should not require ID element for new Entities
> ---
>
> Key: ATLAS-543
> URL: https://issues.apache.org/jira/browse/ATLAS-543
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
> Environment: Ubuntu 14, OpenJDK 64-Bit 1.7.0_95
>Reporter: Joseph Niemiec
>Assignee: Harish Jaiprakash
>Priority: Minor
> Attachments: ATLAS-543.patch
>
>
> When utilizing the REST API to create an 'new' Entity Instance of a given
> Type all ID elements for the class and structs are required, requests without
> these elements will fail despite the fact a random GUID will be assigned at
> instance time.
> #
> Example 1 (Good Entity Posts correctly)
> #
> {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "id": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "id": "-984848",
> "version": 0,
> "typeName": "HDFS_RESOURCE"
> },
> "typeName": "HDFS_RESOURCE",
> "values": {
> "name": "Cluser_A_DevFolder_A",
> "description": "Fully Public Dev Folder",
> "resource": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "id": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "id": "-2630837415522",
> "version": 0,
> "typeName": "HDFS_OBJECT"
> },
> "typeName": "HDFS_OBJECT",
> "values": {
> "uri": "/user/dev/a",
> "isDir" : true
> },
> "traitNames": [],
> "traits": {}
> }
> },
> "traitNames": ["Public"],
> "traits": {
> "Public": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "Public",
> "values": { }
> }
> }
> }
>
> Example #2 Bad Entity that fails.
>
> {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "typeName": "HDFS_RESOURCE",
> "values": {
> "name": "Cluser_A_DevFolder_A",
> "description": "Fully Public Dev Folder",
> "resource": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "typeName": "HDFS_OBJECT",
> "values": {
> "uri": "/user/dev/a",
> "isDir" : true
> },
> "traitNames": [],
> "traits": {}
> }
> },
> "traitNames": ["Public"],
> "traits": {
> "Public": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "Public",
> "values": { }
> }
> }
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-670) Entity update/delete operations shouldn't be supported on deleted entities
Shwetha G S created ATLAS-670: - Summary: Entity update/delete operations shouldn't be supported on deleted entities Key: ATLAS-670 URL: https://issues.apache.org/jira/browse/ATLAS-670 Project: Atlas Issue Type: Sub-task Reporter: Shwetha G S Assignee: Shwetha G S -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-594) alter table rename doesnt work across databases
[ https://issues.apache.org/jira/browse/ATLAS-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241011#comment-15241011 ] Shwetha G S commented on ATLAS-594: --- +1 > alter table rename doesnt work across databases > --- > > Key: ATLAS-594 > URL: https://issues.apache.org/jira/browse/ATLAS-594 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating, 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-594.patch > > > Alter table rename across dbs doesnt work – alter table db1.tb1 rename to > db2.tb2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-494) UI Authentication
[ https://issues.apache.org/jira/browse/ATLAS-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-494: -- Summary: UI Authentication (was: Authentication ) > UI Authentication > -- > > Key: ATLAS-494 > URL: https://issues.apache.org/jira/browse/ATLAS-494 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Nixon Rodrigues > Fix For: 0.7-incubating > > Attachments: ATLAS-494.1.patch, ATLAS-494.2.patch, ATLAS-494.3.patch, > ATLAS-494.4.patch, ATLAS-494.patch > > > Atlas needs an authentication mechanism besides kerberos. > Requirements > - Provide ability to authenticate against local file > - Provide Authentication that supports LDAP > - For consistency, configuration should use exactly the same configuration > (properties) as Ranger & Knox -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-708) Enable AtlasAuthenticationSimpleFilterTest and AtlasAuthenticationKerberosFilterTest
Shwetha G S created ATLAS-708: - Summary: Enable AtlasAuthenticationSimpleFilterTest and AtlasAuthenticationKerberosFilterTest Key: ATLAS-708 URL: https://issues.apache.org/jira/browse/ATLAS-708 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Tests in AtlasAuthenticationKerberosFilterTest and AtlasAuthenticationSimpleFilterTest are disabled currently -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-707) Atlas supports proxy user, but doesn't have proxy user controls
Shwetha G S created ATLAS-707: - Summary: Atlas supports proxy user, but doesn't have proxy user controls Key: ATLAS-707 URL: https://issues.apache.org/jira/browse/ATLAS-707 Project: Atlas Issue Type: Bug Reporter: Shwetha G S AtlasClient uses DelegationTokenAuthenticatedURL which adds doAs param and it supports proxy UGI. But on the atlas server, there are no checks on who are allowed to do impersonation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-728) Fix few typos in committer email IDs
[ https://issues.apache.org/jira/browse/ATLAS-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263889#comment-15263889 ] Shwetha G S commented on ATLAS-728: --- +1 > Fix few typos in committer email IDs > > > Key: ATLAS-728 > URL: https://issues.apache.org/jira/browse/ATLAS-728 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > Attachments: ATLAS-728.patch > > > pom.xml has some typos in committer email IDs. Filing this ticket to fix them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-712) Support getTrait() API
Shwetha G S created ATLAS-712: - Summary: Support getTrait() API Key: ATLAS-712 URL: https://issues.apache.org/jira/browse/ATLAS-712 Project: Atlas Issue Type: Improvement Reporter: Suma Shivaprasad Given entity id and trait name, support rest API that returns the trait instance for the entity. Currently, the other way of getting it is getEntity() which returns full entity with all trait instances -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-556) Hive hook fails for select without table
[
https://issues.apache.org/jira/browse/ATLAS-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-556:
--
Assignee: Vimal Sharma
> Hive hook fails for select without table
>
>
> Key: ATLAS-556
> URL: https://issues.apache.org/jira/browse/ATLAS-556
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Vimal Sharma
> Fix For: 0.7-incubating
>
>
> Reported from: From
> https://community.hortonworks.com/questions/21766/hive-queries-without-from-clause-seem-to-fail-when.html
> Command: select 42
> {noformat}
> 2016-03-09 11:30:05,669 WARN - [main:] ~ Failed to get database
> _dummy_database, returning NoSuchObjectException (ObjectStore:568)
> FAILED: Hive Internal Error: java.lang.NullPointerException(null)
> java.lang.NullPointerException
> at
> org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:325)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:387)
> at
> org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:224)
> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:182)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1520)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
> at
> org.apache.atlas.hive.hook.HiveHookIT.runCommand(HiveHookIT.java:75)
> at
> org.apache.atlas.hive.hook.HiveHookIT.testSelect2(HiveHookIT.java:265)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:673)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:842)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1166)
> at
> org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
> at org.testng.TestRunner.runWorkers(TestRunner.java:1178)
> at org.testng.TestRunner.privateRun(TestRunner.java:757)
> at org.testng.TestRunner.run(TestRunner.java:608)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
> at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
> at org.testng.SuiteRunner.run(SuiteRunner.java:240)
> at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
> at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
> at org.testng.TestNG.runSuitesSequentially(TestNG.java:1158)
> at org.testng.TestNG.runSuitesLocally(TestNG.java:1083)
> at org.testng.TestNG.run(TestNG.java:999)
> at
> org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
> at
> org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
> at
> org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:113)
> at
> org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-602) Hooks stuck in case of failure
[ https://issues.apache.org/jira/browse/ATLAS-602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-602: -- Assignee: Vimal Sharma > Hooks stuck in case of failure > -- > > Key: ATLAS-602 > URL: https://issues.apache.org/jira/browse/ATLAS-602 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > > in the case of whoever uses AtlasHook.notifyEntities(), the retry loop > doesn't ever exit, which is a problem as well. Thanks [~yhemanth] for > reporting it -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-712) Support getTrait() API
[ https://issues.apache.org/jira/browse/ATLAS-712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-712: -- Assignee: Vimal Sharma > Support getTrait() API > -- > > Key: ATLAS-712 > URL: https://issues.apache.org/jira/browse/ATLAS-712 > Project: Atlas > Issue Type: Improvement >Reporter: Suma Shivaprasad >Assignee: Vimal Sharma > > Given entity id and trait name, support rest API that returns the trait > instance for the entity. Currently, the other way of getting it is > getEntity() which returns full entity with all trait instances -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-710) Soft deletes: GET response on a deleted hive tables should have references to hive_db and hive_storagedesc
[
https://issues.apache.org/jira/browse/ATLAS-710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S reassigned ATLAS-710:
-
Assignee: Shwetha G S
> Soft deletes: GET response on a deleted hive tables should have references to
> hive_db and hive_storagedesc
> --
>
> Key: ATLAS-710
> URL: https://issues.apache.org/jira/browse/ATLAS-710
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>
> Soft deletes: GET response on a deleted hive tables does not have references
> to hive_db and hive_storagedesc.
> After creating the hive_table, get response shows references to db, sd and
> columns
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/1b11b234-7360-4679-93f9-d1c01f839f79'
> -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language:
> en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X
> 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112
> Safari/537.36' -H 'Accept: application/json, text/plain, */*' -H 'Referer:
> http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=3frbae3qwzi2xkdjyvz8nyko' -H 'Connection: keep-alive' --compressed
> | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 32710 32710 0 2088 0 --:--:-- 0:00:01 --:--:-- 2088
> {
> "GUID": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "definition": {
> "id": {
> "id": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "2c50426e-7db3-4912-9bbc-13150e962ed5",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "name",
> "qualifiedName":
> "default.after_changes_13.name@primary",
> "type": "string"
> }
> },
> {
> "id": {
> "id": "2af81d25-bbcd-44bd-95b8-4fee915c9dba",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "age",
> "qualifiedName":
> "default.after_changes_13.age@primary",
> "type": "int"
> }
> }
> ],
> "comment": null,
> "createTime": "2016-04-26T09:27:29.000Z",
> "db": {
> "id": "b23cc76a-32e4-4137-860e-8cac2689ba0e",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_db",
> "version": 0
> },
> "description": null,
> "lastAccessTime": "2016-04-26T09:27:29.000Z",
> "name": "default.after_changes_13@primary",
> "owner": "apathan",
> "parameters": {
> "transient_lastDdlTime": "1461662849"
> },
> "partitionKeys": [
> {
> "id": {
> "id": "84db2789-82b3-460e-a827-ee8f94c786cc",
>
[jira] [Resolved] (ATLAS-714) serial look up info of apple devices
[ https://issues.apache.org/jira/browse/ATLAS-714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-714. --- Resolution: Invalid > serial look up info of apple devices > > > Key: ATLAS-714 > URL: https://issues.apache.org/jira/browse/ATLAS-714 > Project: Atlas > Issue Type: Improvement >Reporter: John > > warranty status -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-715) SERIAL
[ https://issues.apache.org/jira/browse/ATLAS-715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-715. --- Resolution: Invalid > SERIAL > -- > > Key: ATLAS-715 > URL: https://issues.apache.org/jira/browse/ATLAS-715 > Project: Atlas > Issue Type: Bug >Reporter: John > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-716) Entity update/delete notifications
Shwetha G S created ATLAS-716: - Summary: Entity update/delete notifications Key: ATLAS-716 URL: https://issues.apache.org/jira/browse/ATLAS-716 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Assignee: Shwetha G S Fix For: 0.7-incubating Entity delete can result in some entity updates(for reverse attributes). The listeners should be called accordingly. Also all the entities created/updated/deleted should be returned in the API response -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-499) Automatically update inverse references to prevent repository corruption
[ https://issues.apache.org/jira/browse/ATLAS-499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-499: -- Fix Version/s: 0.7-incubating > Automatically update inverse references to prevent repository corruption > > > Key: ATLAS-499 > URL: https://issues.apache.org/jira/browse/ATLAS-499 > Project: Atlas > Issue Type: Improvement >Reporter: David Kantor >Assignee: David Kantor > Fix For: 0.7-incubating > > > For references that are defined in the type system as bi-directional, Atlas > does not enforce that clients update both sides of the reference, and relies > on well behaved client applications for repository consistency. This allows > the repository to be easily corrupted with unbalanced references. Atlas > should automatically update the inverse reference to avoid this corruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-716) Entity update/delete notifications
[ https://issues.apache.org/jira/browse/ATLAS-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-716: -- Attachment: ATLAS-716.patch > Entity update/delete notifications > -- > > Key: ATLAS-716 > URL: https://issues.apache.org/jira/browse/ATLAS-716 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-716.patch > > > Entity delete can result in some entity updates(for reverse attributes). The > listeners should be called accordingly. Also all the entities > created/updated/deleted should be returned in the API response -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-716) Entity update/delete notifications
[ https://issues.apache.org/jira/browse/ATLAS-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-716: -- Attachment: ATLAS-716-v2.patch re-based patch > Entity update/delete notifications > -- > > Key: ATLAS-716 > URL: https://issues.apache.org/jira/browse/ATLAS-716 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-716-v2.patch, ATLAS-716.patch > > > Entity delete can result in some entity updates(for reverse attributes). The > listeners should be called accordingly. Also all the entities > created/updated/deleted should be returned in the API response -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-681) update committer/ppmc members in the pom.xml
[ https://issues.apache.org/jira/browse/ATLAS-681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255846#comment-15255846 ] Shwetha G S commented on ATLAS-681: --- Updated the sites - http://atlas.incubator.apache.org/team-list.html and http://incubator.apache.org/projects/atlas.html > update committer/ppmc members in the pom.xml > > > Key: ATLAS-681 > URL: https://issues.apache.org/jira/browse/ATLAS-681 > Project: Atlas > Issue Type: Task >Affects Versions: 0.7-incubating >Reporter: Selvamohan Neethiraj >Assignee: Selvamohan Neethiraj > Fix For: 0.7-incubating > > Attachments: > 0001-ATLAS-681-update-to-ATLAS-team-members-in-the-pom.xm.patch > > > Update pom.xml with correct roles for current team members and add new > committers -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-701) atlas-admin-site.xml contains actual endpoints
Shwetha G S created ATLAS-701: - Summary: atlas-admin-site.xml contains actual endpoints Key: ATLAS-701 URL: https://issues.apache.org/jira/browse/ATLAS-701 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Fix For: 0.7-incubating https://github.com/apache/incubator-atlas/blob/master/webapp/src/main/resources/atlas-admin-site.xml is bundled in webapp and contains actual endpoints. Instead, distro/conf should have a template file -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-690) Read timed out exceptions when tables are imported into Atlas.
[
https://issues.apache.org/jira/browse/ATLAS-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256210#comment-15256210
]
Shwetha G S commented on ATLAS-690:
---
{quote}
If the attribute is unique, add a separate composite index with two keys - the
unique attribute name and the state. Mark uniqueness on this. This is required
now with soft deletes, because it is now possible to have vertices that have
the same unique attribute but with different states.
{quote}
We can't mark this as unique. There can be multiple entities for same attribute
value+state as there can be multiple deleted entities
> Read timed out exceptions when tables are imported into Atlas.
> --
>
> Key: ATLAS-690
> URL: https://issues.apache.org/jira/browse/ATLAS-690
> Project: Atlas
> Issue Type: Bug
> Environment: Atlas with External Kafka/ HBase / Solr
> atlas.notification.hook.numthreads=5
> ATLAS_HOOK created with 5 partitions
>Reporter: Sharmadha Sainath
>Assignee: Hemanth Yamijala
>Priority: Blocker
>
> When 1000 tables are imported into Atlas using Hive hook,Read time out
> exceptions occur. This happened with the latest Atlas build with commit id :
> 922a83c9a10e857d54855463225e9a5c375bc2b9.
>• Hive ingestion was completed in 1 minute 50 secs.
>• Atlas ingestion took more than an hour .
> With Last 1000 tables run that was done in Atlas with commit id :
> b9575f29df3cc014f1b076abf52d88249bf4d0ef,
> • Hive ingestion was completed in 3 minutes
> • Atlas ingestion by 5 minutes.
> The Exception stack trace :
> Error handling message
> org.apache.atlas.notification.hook.HookNotification$EntityUpdateRequest@7474dd2d
> (NotificationHookConsumer:224)
> com.sun.jersey.api.client.ClientHandlerException:
> java.net.SocketTimeoutException: Read timed out
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
> at com.sun.jersey.api.client.Client.handle(Client.java:652)
> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634)
> at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:911)
> at org.apache.atlas.AtlasClient.callAPIWithRetries(AtlasClient.java:565)
> at org.apache.atlas.AtlasClient.callAPI(AtlasClient.java:935)
> at org.apache.atlas.AtlasClient.updateEntities(AtlasClient.java:530)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:216)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:689)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1324)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-710) Soft deletes: GET response on a deleted hive tables should have references to hive_db and hive_storagedesc
[
https://issues.apache.org/jira/browse/ATLAS-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15265371#comment-15265371
]
Shwetha G S commented on ATLAS-710:
---
The issue is because schema tab in UI uses schema API which lists all columns
for a given table name. It doesn't filter the deleted entities. Instead, schema
API should give the schema for a given guid.
> Soft deletes: GET response on a deleted hive tables should have references to
> hive_db and hive_storagedesc
> --
>
> Key: ATLAS-710
> URL: https://issues.apache.org/jira/browse/ATLAS-710
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>
> Soft deletes: GET response on a deleted hive tables does not have references
> to hive_db and hive_storagedesc.
> After creating the hive_table, get response shows references to db, sd and
> columns
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/1b11b234-7360-4679-93f9-d1c01f839f79'
> -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language:
> en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X
> 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112
> Safari/537.36' -H 'Accept: application/json, text/plain, */*' -H 'Referer:
> http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=3frbae3qwzi2xkdjyvz8nyko' -H 'Connection: keep-alive' --compressed
> | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 32710 32710 0 2088 0 --:--:-- 0:00:01 --:--:-- 2088
> {
> "GUID": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "definition": {
> "id": {
> "id": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "2c50426e-7db3-4912-9bbc-13150e962ed5",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "name",
> "qualifiedName":
> "default.after_changes_13.name@primary",
> "type": "string"
> }
> },
> {
> "id": {
> "id": "2af81d25-bbcd-44bd-95b8-4fee915c9dba",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "age",
> "qualifiedName":
> "default.after_changes_13.age@primary",
> "type": "int"
> }
> }
> ],
> "comment": null,
> "createTime": "2016-04-26T09:27:29.000Z",
> "db": {
> "id": "b23cc76a-32e4-4137-860e-8cac2689ba0e",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_db",
> "version": 0
> },
> "description": null,
> "lastAccessTime": "2016-04-26T09:27:29.000Z",
> "name": "default.after_changes_13@primary",
> "owner": "apathan",
> "parameters": {
>
[jira] [Commented] (ATLAS-709) Creating an instance with bi-directional reference does not set the reverse reference
[
https://issues.apache.org/jira/browse/ATLAS-709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257638#comment-15257638
]
Shwetha G S commented on ATLAS-709:
---
ATLAS-499 has the same intentions. Can we resolve this as duplicate of
ATLAS-499? cc [~dkantor]
> Creating an instance with bi-directional reference does not set the reverse
> reference
> -
>
> Key: ATLAS-709
> URL: https://issues.apache.org/jira/browse/ATLAS-709
> Project: Atlas
> Issue Type: Bug
>Reporter: Prasad S Madugundu
>
> Steps to create the issue:
> 1. I have defined 2 classes (DeletePackage_Account and
> DeletePackage_Customer) with a bi-directional reference
> (DeletePackage_Account.primaryCustomer and
> DeletePackage_Customer.ownsAccount).
> 2. I have created instance of the 1st class (DeletePackage_Account) and got
> GUID.
> 3. I have created instance of the 2nd class (DeletePackage_Customer) and set
> the GUID of the first instance (ownsAccount) for establishing the
> bi-directional reference between these instances.
> 4. If I GET the first instance (DeletePackage_Account), I expect that the
> reference (primaryCustomer) is created to the 2nd instance. But, it does not.
> I am sending the JSON for all the above 4 steps.
> Register types request (DeletePackage_Customer and DeletePackage_Account that
> have bi-directional reference as highlighted in bold below)
> -
> uri = http://9.184.184.47:21000/api/atlas/types,
> json =
> {"enumTypes":[],"structTypes":[],"traitTypes":[{"superTypes":[],"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType","typeName":"DeletePackage_frozen","typeDescription":"","attributeDefinitions":[]}],
> "classTypes":[{"superTypes":["OMSBase_OMObject"],"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.ClassType","typeName":"DeletePackage_Customer","typeDescription":"","attributeDefinitions":[{"name":"name","dataTypeName":"string","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"primaryAccount","dataTypeName":"DeletePackage_Account","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"ownsAccount","dataTypeName":"array","multiplicity":"optional","isComposite":false,"reverseAttributeName":"primaryCustomer","isUnique":false,"isIndexable":true}]},
> {"superTypes":["OMSBase_OMObject"],"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.ClassType","typeName":"DeletePackage_Account","typeDescription":"","attributeDefinitions":[{"name":"balance","dataTypeName":"double","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"recentDepositAmounts","dataTypeName":"array","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"lastModified","dataTypeName":"date","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"number","dataTypeName":"string","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"signators","dataTypeName":"array","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"props","dataTypeName":"map","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"recentDeposits","dataTypeName":"array","multiplicity":"optional","isComposite":false,"reverseAttributeName":null,"isUnique":false,"isIndexable":true},{"name":"primaryCustomer","dataTypeName":"array","multiplicity":"optional","isComposite":false,"reverseAttributeName":"ownsAccount","isUnique":false,"isIndexable":true}]}]}
> Create DeletePackage_Account instance request
>
> uri = http://9.184.184.47:21000/api/atlas/entities,
> json =
> [{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-420977068982507","version":0,"typeName":"DeletePackage_Account"},"typeName":"DeletePackage_Account","values":{"balance":1000.0,"omCreationTimestamp":"2016-04-26T05:40:35.008Z","omModifiedByUser":"oms","number":"acc1","omOptimisticLockId":0,"omModificationTimestamp":"2016-04-26T05:40:35.008Z","omCreatedByUser":"oms"},"traitNames":[],"traits":{}}]
> response = {"requestId":"qtp1712669532-12 -
>
[jira] [Updated] (ATLAS-495) Atlas Ranger Authorization Plugin
[ https://issues.apache.org/jira/browse/ATLAS-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-495: -- Attachment: ATLAS-495.5.patch Latest patch from reviewboard > Atlas Ranger Authorization Plugin > - > > Key: ATLAS-495 > URL: https://issues.apache.org/jira/browse/ATLAS-495 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Nixon Rodrigues > Fix For: 0.7-incubating > > Attachments: ATLAS-495.5.patch, ATLAS-495.patch > > > Atlas must support an Authorization Plugin that leverage Ranger: > Requirements > - The Ranger Plugin will authorize all user actions against security policies > defined in Ranger > Example of Policies include: > - Restricting Access to specific actions based on user role i.e. > - Ability to Tag > - Ability to Delete > - Ability to View > - Ability to Modify / Update -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-815) Authentication error message
Shwetha G S created ATLAS-815:
-
Summary: Authentication error message
Key: ATLAS-815
URL: https://issues.apache.org/jira/browse/ATLAS-815
Project: Atlas
Issue Type: Bug
Reporter: Shwetha G S
Fix For: 0.7-incubating
Currently, UI maps username/password errors to invalid credentials. But even
REST APIs should return 'invalid credentials'
{noformat}
apache-atlas-0.7-incubating-SNAPSHOT$ curl -u admin2:admin
http://localhost:21000/api/atlas/admin/version
Error 401 Username not found.admin2
HTTP ERROR 401
Problem accessing /api/atlas/admin/version. Reason:
Username not found.admin2Powered by
Jetty://
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-814) Too many logs from Atlas authorization
Shwetha G S created ATLAS-814:
-
Summary: Too many logs from Atlas authorization
Key: ATLAS-814
URL: https://issues.apache.org/jira/browse/ATLAS-814
Project: Atlas
Issue Type: Bug
Reporter: Shwetha G S
Assignee: Nixon Rodrigues
Fix For: 0.7-incubating
For every request, the following debug logs are printed. I don't think some of
them have any useful info. Please cleanup
{noformat}
2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~ ==>
AuthorizationFilter.doFilter (AtlasAuthorizationFilter:81)
2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~
/api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema is a valid REST
API request!!! (AtlasAuthorizationFilter:88)
2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~ ==>
AtlasAuthorizationFilter getAtlasAction HTTP Method GET mapped to AtlasAction :
READ (AtlasAuthorizationUtils:89)
2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~
UserName ==>> admin
Groups ==>> [ADMIN]
URL ::
http://localhost:21000/api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
Action :: READ
request.getServletPath() ::
/api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
(AtlasAuthorizationFilter:118)
2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~ getAtlasResourceType <===
for /api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
(AtlasAuthorizationUtils:98)
2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~ getApi <=== from
/api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
(AtlasAuthorizationUtils:52)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ Extracted
lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema from path :
/api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
(AtlasAuthorizationUtils:45)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ Returning AtlasResources
[ENTITY] for api lineage (AtlasAuthorizationUtils:133)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ getAtlasResource <===
This implementation will be changed for Resource level Authorization.
(AtlasAuthorizationUtils:143)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ <==
AtlasAccessRequestImpl-- Initializing AtlasAccessRequest (AtlasAccessRequest:41)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ <== SimpleAtlasAuthorizer
isAccessAllowed (SimpleAtlasAuthorizer:116)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~
isAccessAllowd(AtlasAccessRequest [resourceType=[ENTITY], resource=*,
action=READ, user=admin, userGroups=[ADMIN], accessTime=null,
clientIPAddress=null]) (SimpleAtlasAuthorizer:117)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ Checking for ::
User :: admin
Groups :: [ADMIN]
Action :: READ
Resource :: * (SimpleAtlasAuthorizer:125)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ checkAccess for Operation
:: READ on Resource [ENTITY]:* (SimpleAtlasAuthorizer:139)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ <== SimpleAtlasAuthorizer
checkAccess (SimpleAtlasAuthorizer:185)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ Now checking access for
accessor : admin
Resource Types : [ENTITY]
Resource : *
Map : {saqeeb={ENTITY=[*], TERM=[*], TYPE=[*], TAXONOMY=[*]},
admin={OPERATION=[*], ENTITY=[*], TERM=[*], TYPE=[*], TAXONOMY=[*]},
nixon={ENTITY=[*], TERM=[*], TYPE=[*], TAXONOMY=[*]}}
(SimpleAtlasAuthorizer:186)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~
Checking for resource : * in list : [*]
(SimpleAtlasAuthorizer:195)
2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ <== SimpleAtlasAuthorizer
isMatch (SimpleAtlasAuthorizer:272)
2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ <== SimpleAtlasAuthorizer
resourceMatchHelper (SimpleAtlasAuthorizer:239)
2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ ==> SimpleAtlasAuthorizer
resourceMatchHelper (SimpleAtlasAuthorizer:265)
2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ ==> SimpleAtlasAuthorizer
isMatch(*): true (SimpleAtlasAuthorizer:315)
2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ Check for admin :: true
(SimpleAtlasAuthorizer:210)
2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ ==> SimpleAtlasAuthorizer
checkAccess (SimpleAtlasAuthorizer:211)
2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ ==>
+SimpleAtlasAuthorizer isAccessAllowed = true (SimpleAtlasAuthorizer:176)
2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ Authorizer result :: true
(AtlasAuthorizationFilter:134)
2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ Access is allowed so
forwarding the request!!! (AtlasAuthorizationFilter:138)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-716) Entity update/delete notifications
[ https://issues.apache.org/jira/browse/ATLAS-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-716: -- Attachment: ATLAS-716-v3.patch Re-based to trunk. This also has changes for ATLAS-710 - the UTs were conflicting, so merged both the patches. Here are the changes: 1. Create/update/delete APIs return all the entities created/updated/deleted 2. In case of update API, and soft delete, if a struct/trait needs to be deleted, it will be hard deleted - audit will have more details 3. In case of update API, and soft delete, if an entity needs to be deleted, its just marked as deleted, and the parent entity will still have a reference to the deleted entity. For example, in case of hive model, if column has to be deleted, column will be marked as deleted and table will still have reference to this deleted column 4. In case of delete API, and soft delete, all the references(struct, trait, class) of the deleted entity are still maintained > Entity update/delete notifications > -- > > Key: ATLAS-716 > URL: https://issues.apache.org/jira/browse/ATLAS-716 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-716-v2.patch, ATLAS-716-v3.patch, ATLAS-716.patch > > > Entity delete can result in some entity updates(for reverse attributes). The > listeners should be called accordingly. Also all the entities > created/updated/deleted should be returned in the API response -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-811) Entity delete should check for required reverse references - soft deletes
Shwetha G S created ATLAS-811: - Summary: Entity delete should check for required reverse references - soft deletes Key: ATLAS-811 URL: https://issues.apache.org/jira/browse/ATLAS-811 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Assignee: Shwetha G S Fix For: 0.7-incubating DeleteHandler.deleteEdgeBetweenVertices() checks for elements.size(). With soft deletes, even the deleted elements are part of the array. It should instead check for active elements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-810) CTAS tables creation throws exceptions in Atlas ,but the tables are imported and lineage graphs are also rendered.
[ https://issues.apache.org/jira/browse/ATLAS-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S reassigned ATLAS-810: - Assignee: Shwetha G S > CTAS tables creation throws exceptions in Atlas ,but the tables are imported > and lineage graphs are also rendered. > -- > > Key: ATLAS-810 > URL: https://issues.apache.org/jira/browse/ATLAS-810 > Project: Atlas > Issue Type: Bug > Environment: Atlas (commit id : > 98769871e56d9a97792e2dba52345e876908ac63 ) > HBase , External Kafka, Solr >Reporter: Sharmadha Sainath >Assignee: Shwetha G S >Priority: Minor > > Atlas is populated with "n" tables and then CTAS tables are created from > those"n" tables. > Atlas throws the following exceptions : > 2016-05-19 02:59:39,362 ERROR - [NotificationHookConsumer thread-0:] ~ Unable > to persist entity instance (EntityResource:148) > java.lang.NullPointerException > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:124) > at org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:580) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2016-05-19 02:59:39,379 WARN - [NotificationHookConsumer thread-0:] ~ Error > handling message > org.apache.atlas.notification.hook.HookNotification$EntityCreateRequest@24fc6736 > (NotificationHookConsumer:233) > org.apache.atlas.AtlasServiceException: Metadata service API CREATE_ENTITY > failed with status 500(Internal Server Error) Response Body > (java.lang.NullPointerException > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:124) > at org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:580) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:122) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:580) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2016-05-19 02:59:39,380 DEBUG - [NotificationHookConsumer thread-0:] ~ > Committed offset: 2 (KafkaConsumer:96) > No exceptions in Hive logs. > But all CTAS tables are imported and lineage graphs are properly rendered on > Atlas UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-809) JAAS configuration needed for Kafka interaction via Atlas config file
[ https://issues.apache.org/jira/browse/ATLAS-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-809: -- Attachment: 0001-In-memory-JAAS-Configurator.-Addressed-review-commen.patch patch from reviewboard > JAAS configuration needed for Kafka interaction via Atlas config file > - > > Key: ATLAS-809 > URL: https://issues.apache.org/jira/browse/ATLAS-809 > Project: Atlas > Issue Type: Bug >Reporter: Madhan Neethiraj >Assignee: Abhay Kulkarni > Attachments: > 0001-In-memory-JAAS-Configurator.-Addressed-review-commen.patch > > > In a kerberized environment, using Kafka from Atlas requires 'KafkaClient' > section to be configured in application's JAAS config file. Given Atlas hooks > run in host components (like Hive, Falcon, Storm..), this requires updates to > JAAS config file used by the component; and in some components (like Hive) > require update to JVM startup parameters to specify the location of JAAS > configuration file. Instead of requiring to update each component's JAAS > configuration file and startup parameters, it will be cleaner and simpler to > handle JAAS configuration via existing Atlas configuration file > (atlas-application.properties). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-822) Type updates - add supertypes
Shwetha G S created ATLAS-822: - Summary: Type updates - add supertypes Key: ATLAS-822 URL: https://issues.apache.org/jira/browse/ATLAS-822 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Fix For: 0.7-incubating In type updates, we allow adding super types. But set of super types is maintained in every entity, and this type update will require updating all the entities of that type which is not handled currently. We should either update all entities or not allow updating super type information -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-713) Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.
[
https://issues.apache.org/jira/browse/ATLAS-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-713:
--
Attachment: ATLAS-713.patch
Current lineage APIs work on the dataset name. With soft delete, there can be
more than one entities for the given dataset name.
1. Added new lineage APIs(input graph, output graph, schema) which work on
entity id
2. Modified current lineage APIs to work on the active entity for the given
dataset name
3. Modified UI to use the new APIs
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> -
>
> Key: ATLAS-713
> URL: https://issues.apache.org/jira/browse/ATLAS-713
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-713.patch
>
>
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> Create a hive table (CREATE TABLE test_change1 (a int, b int, c int); )
> Change column name to new one (ALTER TABLE test_change1 CHANGE c c1 INT
> FIRST; )
> UI snapshot: https://monosnap.com/file/GVOipYxkcaJPcqgnJuxULBTCZopytR
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/ff37663a-7561-462c-9ec8-2bacaebc0296'
> -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, sdch' -H
> 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh;
> Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/49.0.2623.112 Safari/537.36' -H 'Accept: application/json, text/plain,
> */*' -H 'Referer: http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=13d5gxn515o7l5271gbb0j0ec' -H 'Connection: keep-alive' -H
> 'Cache-Control: no-cache' --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 33900 33900 0 2187 0 --:--:-- 0:00:01 --:--:-- 2188
> {
> "GUID": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "definition": {
> "id": {
> "id": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "9252f481-ca2a-4b31-88d2-d404dfc9d3fd",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "c1",
> "qualifiedName": "default.test_change1.c1@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "a4d14fff-842f-48bc-962f-0e4fbac2a815",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "a",
> "qualifiedName": "default.test_change1.a@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "5c700938-5782-4ae4-8599-7dfe90fecfdc",
> "jsonClass":
>
[jira] [Commented] (ATLAS-713) Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.
[
https://issues.apache.org/jira/browse/ATLAS-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279838#comment-15279838
]
Shwetha G S commented on ATLAS-713:
---
This patch should be applied on top of ATLAS-738 patch
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> -
>
> Key: ATLAS-713
> URL: https://issues.apache.org/jira/browse/ATLAS-713
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-713.patch
>
>
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> Create a hive table (CREATE TABLE test_change1 (a int, b int, c int); )
> Change column name to new one (ALTER TABLE test_change1 CHANGE c c1 INT
> FIRST; )
> UI snapshot: https://monosnap.com/file/GVOipYxkcaJPcqgnJuxULBTCZopytR
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/ff37663a-7561-462c-9ec8-2bacaebc0296'
> -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, sdch' -H
> 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh;
> Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/49.0.2623.112 Safari/537.36' -H 'Accept: application/json, text/plain,
> */*' -H 'Referer: http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=13d5gxn515o7l5271gbb0j0ec' -H 'Connection: keep-alive' -H
> 'Cache-Control: no-cache' --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 33900 33900 0 2187 0 --:--:-- 0:00:01 --:--:-- 2188
> {
> "GUID": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "definition": {
> "id": {
> "id": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "9252f481-ca2a-4b31-88d2-d404dfc9d3fd",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "c1",
> "qualifiedName": "default.test_change1.c1@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "a4d14fff-842f-48bc-962f-0e4fbac2a815",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "a",
> "qualifiedName": "default.test_change1.a@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "5c700938-5782-4ae4-8599-7dfe90fecfdc",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
>
[jira] [Commented] (ATLAS-661) REST API Authentication
[ https://issues.apache.org/jira/browse/ATLAS-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276013#comment-15276013 ] Shwetha G S commented on ATLAS-661: --- Its timing issue and jenkins build has been failing for a while with the same issue. For others, some other test fails. So, its not related to ATLAS-690 > REST API Authentication > --- > > Key: ATLAS-661 > URL: https://issues.apache.org/jira/browse/ATLAS-661 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Nixon Rodrigues >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-661.2.patch, ATLAS-661.3.patch, ATLAS-661.patch > > > Atlas needs to authorize the REST API invocations; as necessary requests will > be forced to authenticate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-738) Add query ability on system properties like guid, state, createdtime etc
[ https://issues.apache.org/jira/browse/ATLAS-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-738: -- Attachment: ATLAS-738-v2.patch > Add query ability on system properties like guid, state, createdtime etc > > > Key: ATLAS-738 > URL: https://issues.apache.org/jira/browse/ATLAS-738 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-738-v2.patch, ATLAS-738.patch > > > Currently, DSL doesn't support querying on internal fields like guid, state, > created time, modified time etc. Some useful queries are: > 1. All entities created after some time > 2. All entities modified after some time > 3. Retrieve entity with given id or state -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-738) Add query ability on system properties like guid, state, createdtime etc
[ https://issues.apache.org/jira/browse/ATLAS-738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276078#comment-15276078 ] Shwetha G S commented on ATLAS-738: --- Got rid of the map, create review board request with the updates. Have moved newAttributeInfo() to TypesUtil as it had other utilities and AttributeInfo already has a type system and would cause conflicts > Add query ability on system properties like guid, state, createdtime etc > > > Key: ATLAS-738 > URL: https://issues.apache.org/jira/browse/ATLAS-738 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-738-v2.patch, ATLAS-738.patch > > > Currently, DSL doesn't support querying on internal fields like guid, state, > created time, modified time etc. Some useful queries are: > 1. All entities created after some time > 2. All entities modified after some time > 3. Retrieve entity with given id or state -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-738) Add query ability on system properties like guid, state, createdtime etc
[ https://issues.apache.org/jira/browse/ATLAS-738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276078#comment-15276078 ] Shwetha G S edited comment on ATLAS-738 at 5/9/16 8:04 AM: --- Got rid of the map, created review board request with the updates. Have moved newAttributeInfo() to TypesUtil as it had other utilities and AttributeInfo already has a type system and would cause conflicts was (Author: shwethags): Got rid of the map, create review board request with the updates. Have moved newAttributeInfo() to TypesUtil as it had other utilities and AttributeInfo already has a type system and would cause conflicts > Add query ability on system properties like guid, state, createdtime etc > > > Key: ATLAS-738 > URL: https://issues.apache.org/jira/browse/ATLAS-738 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-738-v2.patch, ATLAS-738.patch > > > Currently, DSL doesn't support querying on internal fields like guid, state, > created time, modified time etc. Some useful queries are: > 1. All entities created after some time > 2. All entities modified after some time > 3. Retrieve entity with given id or state -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-758) hdfs location of hive table is pointing to old location even after rename.
[
https://issues.apache.org/jira/browse/ATLAS-758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281300#comment-15281300
]
Shwetha G S commented on ATLAS-758:
---
+1
> hdfs location of hive table is pointing to old location even after rename.
> --
>
> Key: ATLAS-758
> URL: https://issues.apache.org/jira/browse/ATLAS-758
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Suma Shivaprasad
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-758.1.patch, ATLAS-758.patch, bug.log
>
>
> hdfs location of hive table is pointing to old location even after rename.
> Steps to repro:
> {noformat}
> 0: jdbc:hive2://localhost:1/default> create database db_temporary;
> OK
> No rows affected (0.281 seconds)
> 0: jdbc:hive2://localhost:1/default> use db_temporary;
> OK
> No rows affected (0.035 seconds)
> 0: jdbc:hive2://localhost:1/default> create table temp_table (name
> String);
> OK
> No rows affected (0.204 seconds)
> 0: jdbc:hive2://localhost:1/default> describe formatted temp_table;
> OK
> +---+---+---+--+
> | col_name| data_type
> |comment|
> +---+---+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | name | string
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | db_temporary
> | NULL |
> | Owner:| apathan
> | NULL |
> | CreateTime: | Sat May 07 18:02:47 IST 2016
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:| 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/db_temporary.db/temp_table | NULL
> |
> | Table Type: | MANAGED_TABLE
> | NULL |
> | Table Parameters: | NULL
> | NULL |
> | | transient_lastDdlTime
> | 1462624367|
> | | NULL
> | NULL |
> | # Storage Information | NULL
> | NULL |
> | SerDe Library:|
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe| NULL
> |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat
> | NULL |
> | OutputFormat: |
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat| NULL
> |
> | Compressed: | No
> | NULL |
> | Num Buckets: | -1
> | NULL |
> | Bucket Columns: | []
> | NULL |
> | Sort Columns: | []
> | NULL |
> | Storage
[jira] [Resolved] (ATLAS-790) persist method on JavaRDDLike
[ https://issues.apache.org/jira/browse/ATLAS-790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-790. --- Resolution: Invalid > persist method on JavaRDDLike > - > > Key: ATLAS-790 > URL: https://issues.apache.org/jira/browse/ATLAS-790 > Project: Atlas > Issue Type: Improvement >Reporter: sumit rastogi > > All SubClass of JavaRDDLike has persist function but JavaRDDLike does not > have it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-645) FieldMapping.output() results in stack overflow when instances reference each other
[
https://issues.apache.org/jira/browse/ATLAS-645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-645:
--
Attachment: rb45948.patch
patch from reveiwboard
> FieldMapping.output() results in stack overflow when instances reference each
> other
> ---
>
> Key: ATLAS-645
> URL: https://issues.apache.org/jira/browse/ATLAS-645
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: David Kantor
> Attachments: rb45948.patch, rb45948.patch
>
>
> FieldMapping.output() results in a stack overflow due to infinite recursion
> when two IReferenceableInstance or IStruct objects reference each other.
> This issue was first exposed by
> GraphBackedMetadataRepositoryDeleteEntitiesTest.testDisconnectMapReferenceFromClassType().
> {noformat}
> SLF4J: Failed toString() invocation on an object of type
> [org.apache.atlas.typesystem.persistence.ReferenceableInstance]
> java.lang.StackOverflowError
> at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
> at java.util.regex.Pattern$Branch.match(Pattern.java:4502)
> at java.util.regex.Pattern$BranchConn.match(Pattern.java:4466)
> at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
> at java.util.regex.Pattern$Curly.match0(Pattern.java:4177)
> at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
> at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
> at java.util.regex.Pattern$Branch.match(Pattern.java:4502)
> at java.util.regex.Pattern$Branch.match(Pattern.java:4500)
> at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
> at java.util.regex.Pattern$Start.match(Pattern.java:3408)
> at java.util.regex.Matcher.search(Matcher.java:1199)
> at java.util.regex.Matcher.find(Matcher.java:618)
> at java.util.Formatter.parse(Formatter.java:2517)
> at java.util.Formatter.format(Formatter.java:2469)
> at java.util.Formatter.format(Formatter.java:2423)
> at java.lang.String.format(String.java:2792)
> at org.apache.atlas.typesystem.persistence.Id.toString(Id.java:98)
> at
> org.apache.atlas.typesystem.types.FieldMapping.output(FieldMapping.java:114)
> at
> org.apache.atlas.typesystem.persistence.ReferenceableInstance.toString(ReferenceableInstance.java:92)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-713) Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.
[
https://issues.apache.org/jira/browse/ATLAS-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-713:
--
Attachment: ATLAS-713-v2.patch
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> -
>
> Key: ATLAS-713
> URL: https://issues.apache.org/jira/browse/ATLAS-713
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-713-v2.patch, ATLAS-713.patch
>
>
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> Create a hive table (CREATE TABLE test_change1 (a int, b int, c int); )
> Change column name to new one (ALTER TABLE test_change1 CHANGE c c1 INT
> FIRST; )
> UI snapshot: https://monosnap.com/file/GVOipYxkcaJPcqgnJuxULBTCZopytR
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/ff37663a-7561-462c-9ec8-2bacaebc0296'
> -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, sdch' -H
> 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh;
> Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/49.0.2623.112 Safari/537.36' -H 'Accept: application/json, text/plain,
> */*' -H 'Referer: http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=13d5gxn515o7l5271gbb0j0ec' -H 'Connection: keep-alive' -H
> 'Cache-Control: no-cache' --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 33900 33900 0 2187 0 --:--:-- 0:00:01 --:--:-- 2188
> {
> "GUID": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "definition": {
> "id": {
> "id": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "9252f481-ca2a-4b31-88d2-d404dfc9d3fd",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "c1",
> "qualifiedName": "default.test_change1.c1@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "a4d14fff-842f-48bc-962f-0e4fbac2a815",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "a",
> "qualifiedName": "default.test_change1.a@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "5c700938-5782-4ae4-8599-7dfe90fecfdc",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
>
[jira] [Commented] (ATLAS-784) Configure config.store.uri for Falcon hook IT
[
https://issues.apache.org/jira/browse/ATLAS-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282616#comment-15282616
]
Shwetha G S commented on ATLAS-784:
---
+1
> Configure config.store.uri for Falcon hook IT
> -
>
> Key: ATLAS-784
> URL: https://issues.apache.org/jira/browse/ATLAS-784
> Project: Atlas
> Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Hemanth Yamijala
> Attachments: ATLAS-784.patch
>
>
> While testing with later versions of Falcon dependencies, we found
> FalconHookIT is failing due to expecting a particular property. The specific
> exception is
> {code}
> java.lang.ExceptionInInitializerError
> at
> org.apache.atlas.falcon.hook.FalconHookIT.(FalconHookIT.java:69)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:28)
> at org.testng.internal.ClassHelper.createInstance1(ClassHelper.java:372)
> at org.testng.internal.ClassHelper.createInstance(ClassHelper.java:284)
> at org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:109)
> at org.testng.internal.ClassImpl.getInstances(ClassImpl.java:181)
> at
> org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:120)
> at org.testng.TestRunner.initMethods(TestRunner.java:402)
> at org.testng.TestRunner.init(TestRunner.java:228)
> at org.testng.TestRunner.init(TestRunner.java:198)
> at org.testng.TestRunner.(TestRunner.java:147)
> at
> org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:525)
> at org.testng.SuiteRunner.init(SuiteRunner.java:156)
> at org.testng.SuiteRunner.(SuiteRunner.java:110)
> at org.testng.TestNG.createSuiteRunner(TestNG.java:1235)
> at org.testng.TestNG.createSuiteRunners(TestNG.java:1222)
> at org.testng.TestNG.runSuitesLocally(TestNG.java:1074)
> at org.testng.TestNG.run(TestNG.java:999)
> at
> org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:132)
> at
> org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
> at
> org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
> at
> org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:147)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
> Caused by: java.lang.IllegalArgumentException: Can not create a Path from a
> null string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.falcon.util.StartupProperties.get(StartupProperties.java:91)
> at
> org.apache.falcon.entity.store.ConfigurationStore.(ConfigurationStore.java:121)
> at
> org.apache.falcon.entity.store.ConfigurationStore.(ConfigurationStore.java:99)
> ... 29 more
> {code}
> Adding this property does not harm the current versions we are building with,
> so filing this ticket to add the same for ease of migrating.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (ATLAS-710) Soft deletes: GET response on a deleted hive tables should have references to hive_db and hive_storagedesc
[
https://issues.apache.org/jira/browse/ATLAS-710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S resolved ATLAS-710.
---
Resolution: Duplicate
Will address it as part of ATLAS-716
> Soft deletes: GET response on a deleted hive tables should have references to
> hive_db and hive_storagedesc
> --
>
> Key: ATLAS-710
> URL: https://issues.apache.org/jira/browse/ATLAS-710
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>
> Soft deletes: GET response on a deleted hive tables does not have references
> to hive_db and hive_storagedesc.
> After creating the hive_table, get response shows references to db, sd and
> columns
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/1b11b234-7360-4679-93f9-d1c01f839f79'
> -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language:
> en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X
> 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112
> Safari/537.36' -H 'Accept: application/json, text/plain, */*' -H 'Referer:
> http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=3frbae3qwzi2xkdjyvz8nyko' -H 'Connection: keep-alive' --compressed
> | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 32710 32710 0 2088 0 --:--:-- 0:00:01 --:--:-- 2088
> {
> "GUID": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "definition": {
> "id": {
> "id": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "2c50426e-7db3-4912-9bbc-13150e962ed5",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "name",
> "qualifiedName":
> "default.after_changes_13.name@primary",
> "type": "string"
> }
> },
> {
> "id": {
> "id": "2af81d25-bbcd-44bd-95b8-4fee915c9dba",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "age",
> "qualifiedName":
> "default.after_changes_13.age@primary",
> "type": "int"
> }
> }
> ],
> "comment": null,
> "createTime": "2016-04-26T09:27:29.000Z",
> "db": {
> "id": "b23cc76a-32e4-4137-860e-8cac2689ba0e",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_db",
> "version": 0
> },
> "description": null,
> "lastAccessTime": "2016-04-26T09:27:29.000Z",
> "name": "default.after_changes_13@primary",
> "owner": "apathan",
> "parameters": {
> "transient_lastDdlTime": "1461662849"
> },
> "partitionKeys": [
> {
> "id": {
> "id":
[jira] [Resolved] (ATLAS-748) UI: UI should use entity id based lineage and schema
[ https://issues.apache.org/jira/browse/ATLAS-748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-748. --- Resolution: Duplicate Handled as part of ATLAS-713 > UI: UI should use entity id based lineage and schema > > > Key: ATLAS-748 > URL: https://issues.apache.org/jira/browse/ATLAS-748 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S > Fix For: 0.7-incubating > > > Currently, lineage and schema APIs use table qualified name as argument. With > soft delete, there can be multiple entities for a given table qualified > name(at most one active entity and >=0 deleted entities). In UI, the lineage > and schema is shown in the context of a single entity. So, even lineage and > schema should use the lineage and schema corresponding to that entity > This requires new REST APIs that will be added as part of ATLAS-713 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-713) Entity lineage based on entity id
[
https://issues.apache.org/jira/browse/ATLAS-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-713:
--
Summary: Entity lineage based on entity id (was: Atlas UI is still showing
the deleted column of a hive table (Entity response of hive table does not have
it). - with browser cache disabled.)
> Entity lineage based on entity id
> -
>
> Key: ATLAS-713
> URL: https://issues.apache.org/jira/browse/ATLAS-713
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-713-v2.patch, ATLAS-713.patch
>
>
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> Create a hive table (CREATE TABLE test_change1 (a int, b int, c int); )
> Change column name to new one (ALTER TABLE test_change1 CHANGE c c1 INT
> FIRST; )
> UI snapshot: https://monosnap.com/file/GVOipYxkcaJPcqgnJuxULBTCZopytR
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/ff37663a-7561-462c-9ec8-2bacaebc0296'
> -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, sdch' -H
> 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh;
> Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/49.0.2623.112 Safari/537.36' -H 'Accept: application/json, text/plain,
> */*' -H 'Referer: http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=13d5gxn515o7l5271gbb0j0ec' -H 'Connection: keep-alive' -H
> 'Cache-Control: no-cache' --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 33900 33900 0 2187 0 --:--:-- 0:00:01 --:--:-- 2188
> {
> "GUID": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "definition": {
> "id": {
> "id": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "9252f481-ca2a-4b31-88d2-d404dfc9d3fd",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "c1",
> "qualifiedName": "default.test_change1.c1@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "a4d14fff-842f-48bc-962f-0e4fbac2a815",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "a",
> "qualifiedName": "default.test_change1.a@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "5c700938-5782-4ae4-8599-7dfe90fecfdc",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
>
[jira] [Commented] (ATLAS-790) persist method on JavaRDDLike
[ https://issues.apache.org/jira/browse/ATLAS-790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288735#comment-15288735 ] Shwetha G S commented on ATLAS-790: --- You have filed the issue under atlas project - http://atlas.incubator.apache.org/. This doesn't look like its related to atlas. Did you mean to create issue for spark? > persist method on JavaRDDLike > - > > Key: ATLAS-790 > URL: https://issues.apache.org/jira/browse/ATLAS-790 > Project: Atlas > Issue Type: Improvement >Reporter: sumit rastogi > > All SubClass of JavaRDDLike has persist function but JavaRDDLike does not > have it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-771) 1 844-307-5701 Outlook pro c.u.s.t.o.m.e.r s.u.p.p.o.r.t n.u.m.b.e.r Outlook T.e.c.h s.u.p.p.o.r.t ph.one n.u.m.b.e
[ https://issues.apache.org/jira/browse/ATLAS-771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-771. --- Resolution: Invalid > 1 844-307-5701 Outlook pro c.u.s.t.o.m.e.r s.u.p.p.o.r.t n.u.m.b.e.r Outlook > T.e.c.h s.u.p.p.o.r.t ph.one n.u.m.b.e > --- > > Key: ATLAS-771 > URL: https://issues.apache.org/jira/browse/ATLAS-771 > Project: Atlas > Issue Type: Bug >Reporter: annaadiwar0123 > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-770) usa Icloud technical support number @1 844-848-1262@ Icloud customer service phone number usa,APPLE Icloud customer support phone number Usa
[ https://issues.apache.org/jira/browse/ATLAS-770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-770. --- Resolution: Invalid > usa Icloud technical support number @1 844-848-1262@ Icloud customer service > phone number usa,APPLE Icloud customer support phone number Usa > > > Key: ATLAS-770 > URL: https://issues.apache.org/jira/browse/ATLAS-770 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.5-incubating > Environment: usa Icloud technical support number @1 844-848-1262@ > Icloud customer service phone number usa,APPLE Icloud customer support phone > number Usausa Icloud technical support number @1 844-848-1262@ Icloud > customer service phone number usa,APPLE Icloud customer support phone number > Usa >Reporter: sinkukumar321 > Labels: github-import > Fix For: 0.7-incubating > > > usa Icloud technical support number @1 844-848-1262@ Icloud customer service > phone number usa,APPLE Icloud customer support phone number Usa > usa Icloud technical support number @1 844-848-1262@ Icloud customer service > phone number usa,APPLE Icloud customer support phone number Usa > NEW YORK LIVE @1 844-848-1262@APPLE Icloud customer service phone number > usa,APPLE Icloud customer support phone number Usa Describe NEW YORK LIVE > +@1844-848-1262@APPLE Icloud customer service phone number usa,APPLE Icloud > customer support phone number Usa here. USACanada Help@@1844-848-1262@ Icloud > customer service phone number usa, Icloud customer support phone number > Canada here. Usa Canada@@Call, 1-844-848-1262 for Icloud customer support > telephone number, Icloud Toll Free, @(1-844-848-1262)@: Icloud Technical > Support Number, Icloud help desk phone number vides online solution for all > USA/CANADA clients. For any help of query call I 844-848-1262 to get all > Icloud account solution. 1844-848-1262 For any help of query call 1 > 844-848-1262 to get all Icloud account solution. @@Call, 1-844-848-1262 for > all type help by Icloud support telephone number, Icloud support phone > number, Icloud support phone number, Icloud help phone number, Icloud > technical support number. Icloud support number, Icloud phone number, Icloud > tech support number, Icloud customer support number, Icloud customer support > phone number, Icloud customer service phone number, Icloud payroll customer > service phone number, Icloud support phone number Icloud help number Icloud > Helpline Number; Icloud help phone number- Icloud Helpline Number, Icloud > Tech Support Toll free Number, Icloud Support Telephone Number, Icloud Tech > Support Telephone number, Icloud Tech Support contact number, Icloud support > contact number, Icloud technical support contact number, Icloud help desk > phone number. Icloud password recovery support phone number. Call, > @(1-844-848-1262)@: Icloud password recovery support phone number. It@@Call, > 1-844-848-1262 for Icloud customer support telephone number, Icloud support > phone number is very popular toll free number which vide by Icloud technical > support, Icloud Customer Service Phone Number, Icloud Customer Service > Number, Icloud Customer Support Phone Number, Icloud Customer Support Number, > Icloud Customer Service Helpline Number, Icloud Customer Care Number, Icloud > support team phone number. Icloud password recovery support phone number. > Call,@(1-844-848-1262)@: Icloud help number- Icloud Helpline Number; Icloud > help phone number, Icloud Helpline Number, Icloud Tech Support Toll free > Number, Icloud Support Telephone Number, Icloud Tech Support Telephone > number, Icloud Tech Support contact number, Icloud support contact number, > Icloud technical support contact number, Icloud support phone number, Icloud > payroll support phone number. Icloud payroll customer support phone number.. > Icloud password reset support phone number. for all type help by Icloud > support telephone number, Icloud help phone number, Icloud technical support > number. Icloud support number, Icloud phone number, Icloud tech support > number, Icloud customer support number, Icloud customer support phone number, > Icloud customer service phone number, Icloud payroll customer service phone > number Icloud Toll Free, @(1-844-848-1262)@: Icloud Technical Support number > usa, Icloud help desk phone number usa vides online solution for all > USA/CANADA clients. For any help of query call I 844-848-1262 to get all > Icloud account solution. 1844-848-1262 For any help of query call 1 > 844-848-1262 to get all Icloud account solution. @@Call, 1-844-848-1262 for > all type help by Icloud support phone number usa, Icloud support phone number > usa,
[jira] [Commented] (ATLAS-602) Hooks stuck in case of failure
[ https://issues.apache.org/jira/browse/ATLAS-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268093#comment-15268093 ] Shwetha G S commented on ATLAS-602: --- The failure is in hooks, because of issues in sending messages to kafka. So, writing to kafka failed queue may not work as well > Hooks stuck in case of failure > -- > > Key: ATLAS-602 > URL: https://issues.apache.org/jira/browse/ATLAS-602 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > > in the case of whoever uses AtlasHook.notifyEntities(), the retry loop > doesn't ever exit, which is a problem as well. Thanks [~yhemanth] for > reporting it -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-738) Add query ability on internal properties like guid, state, createdtime etc
Shwetha G S created ATLAS-738: - Summary: Add query ability on internal properties like guid, state, createdtime etc Key: ATLAS-738 URL: https://issues.apache.org/jira/browse/ATLAS-738 Project: Atlas Issue Type: Improvement Reporter: Shwetha G S Assignee: Shwetha G S Currently, DSL doesn't support querying on internal fields like guid, state, created time, modified time etc. Some useful queries are: 1. All entities created after some time 2. All entities modified after some time 3. Retrieve entity with given id or state -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-741) atlas_stop fails if embedded hmaster that was started doesn't exist
Shwetha G S created ATLAS-741:
-
Summary: atlas_stop fails if embedded hmaster that was started
doesn't exist
Key: ATLAS-741
URL: https://issues.apache.org/jira/browse/ATLAS-741
Project: Atlas
Issue Type: Bug
Reporter: Shwetha G S
Fix For: 0.7-incubating
{noformat}
apache-atlas-0.7-incubating-SNAPSHOT$ ./bin/atlas_stop.py
stopping atlas
no master to stop because no pid file /tmp/hbase-sshivalingamurthy-master.pid
{noformat}
atlas_stop fails to shutdown atlas if it can't find the hbase master that it
had started earlier
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-742) Avoid downloading hbase multiple times
Shwetha G S created ATLAS-742: - Summary: Avoid downloading hbase multiple times Key: ATLAS-742 URL: https://issues.apache.org/jira/browse/ATLAS-742 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Assignee: Shwetha G S In distro module, hbase package which is ~100mb is downloaded to target. mvn clean is required during development for clean build and target gets cleaned up and hbase is downloaded everytime. Instead, download hbase to a location outside target and skip download is exists -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-742) Avoid downloading hbase multiple times
[ https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-742: -- Description: In distro module, hbase package which is ~100mb is downloaded to target. mvn clean is required during development for clean build and target gets cleaned up and hbase is downloaded everytime. Instead, download hbase to a location outside target and skip download if exists (was: In distro module, hbase package which is ~100mb is downloaded to target. mvn clean is required during development for clean build and target gets cleaned up and hbase is downloaded everytime. Instead, download hbase to a location outside target and skip download is exists) > Avoid downloading hbase multiple times > -- > > Key: ATLAS-742 > URL: https://issues.apache.org/jira/browse/ATLAS-742 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > > In distro module, hbase package which is ~100mb is downloaded to target. mvn > clean is required during development for clean build and target gets cleaned > up and hbase is downloaded everytime. Instead, download hbase to a location > outside target and skip download if exists -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-743) Soft Delete: Modifying a deleted entity should not be allowed.
[ https://issues.apache.org/jira/browse/ATLAS-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-743. --- Resolution: Duplicate > Soft Delete: Modifying a deleted entity should not be allowed. > -- > > Key: ATLAS-743 > URL: https://issues.apache.org/jira/browse/ATLAS-743 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan >Assignee: Shwetha G S >Priority: Critical > > Modifying a deleted entity is still allowed. For example: attaching a trait > to a deleted entity is still possible. I dont know a use-case why atlas > should allow this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-745) Set heapsize for embedded hbase
Shwetha G S created ATLAS-745: - Summary: Set heapsize for embedded hbase Key: ATLAS-745 URL: https://issues.apache.org/jira/browse/ATLAS-745 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Fix For: 0.7-incubating Embedded hbase thats started as part of atlas start, doesn't have any max heap set and fails with GC. Will a heap of 1GB help? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-713) Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.
[
https://issues.apache.org/jira/browse/ATLAS-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S reassigned ATLAS-713:
-
Assignee: Shwetha G S (was: Keval Bhatt)
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> -
>
> Key: ATLAS-713
> URL: https://issues.apache.org/jira/browse/ATLAS-713
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
>
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> Create a hive table (CREATE TABLE test_change1 (a int, b int, c int); )
> Change column name to new one (ALTER TABLE test_change1 CHANGE c c1 INT
> FIRST; )
> UI snapshot: https://monosnap.com/file/GVOipYxkcaJPcqgnJuxULBTCZopytR
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/ff37663a-7561-462c-9ec8-2bacaebc0296'
> -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, sdch' -H
> 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh;
> Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/49.0.2623.112 Safari/537.36' -H 'Accept: application/json, text/plain,
> */*' -H 'Referer: http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=13d5gxn515o7l5271gbb0j0ec' -H 'Connection: keep-alive' -H
> 'Cache-Control: no-cache' --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 33900 33900 0 2187 0 --:--:-- 0:00:01 --:--:-- 2188
> {
> "GUID": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "definition": {
> "id": {
> "id": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "9252f481-ca2a-4b31-88d2-d404dfc9d3fd",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "c1",
> "qualifiedName": "default.test_change1.c1@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "a4d14fff-842f-48bc-962f-0e4fbac2a815",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "a",
> "qualifiedName": "default.test_change1.a@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "5c700938-5782-4ae4-8599-7dfe90fecfdc",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
[jira] [Commented] (ATLAS-713) Atlas UI is still showing the deleted column of a hive table (Entity response of hive table does not have it). - with browser cache disabled.
[
https://issues.apache.org/jira/browse/ATLAS-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268925#comment-15268925
]
Shwetha G S commented on ATLAS-713:
---
This is a backend issue that I need to fix. Will create another jira for UI
changes
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> -
>
> Key: ATLAS-713
> URL: https://issues.apache.org/jira/browse/ATLAS-713
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Keval Bhatt
> Fix For: 0.7-incubating
>
>
> Atlas UI is still showing the deleted column of a hive table (Entity response
> of hive table does not have it). - with browser cache disabled.
> Create a hive table (CREATE TABLE test_change1 (a int, b int, c int); )
> Change column name to new one (ALTER TABLE test_change1 CHANGE c c1 INT
> FIRST; )
> UI snapshot: https://monosnap.com/file/GVOipYxkcaJPcqgnJuxULBTCZopytR
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/entities/ff37663a-7561-462c-9ec8-2bacaebc0296'
> -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, sdch' -H
> 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh;
> Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/49.0.2623.112 Safari/537.36' -H 'Accept: application/json, text/plain,
> */*' -H 'Referer: http://localhost:21000/index.html' -H 'Cookie:
> JSESSIONID=13d5gxn515o7l5271gbb0j0ec' -H 'Connection: keep-alive' -H
> 'Cache-Control: no-cache' --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 33900 33900 0 2187 0 --:--:-- 0:00:01 --:--:-- 2188
> {
> "GUID": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "definition": {
> "id": {
> "id": "ff37663a-7561-462c-9ec8-2bacaebc0296",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "9252f481-ca2a-4b31-88d2-d404dfc9d3fd",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "c1",
> "qualifiedName": "default.test_change1.c1@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "a4d14fff-842f-48bc-962f-0e4fbac2a815",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "a",
> "qualifiedName": "default.test_change1.a@primary",
> "type": "int"
> }
> },
> {
> "id": {
> "id": "5c700938-5782-4ae4-8599-7dfe90fecfdc",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
>
[jira] [Resolved] (ATLAS-750) docker_operator - Improve the integration with swarm
[ https://issues.apache.org/jira/browse/ATLAS-750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-750. --- Resolution: Invalid Not related to Atlas project > docker_operator - Improve the integration with swarm > > > Key: ATLAS-750 > URL: https://issues.apache.org/jira/browse/ATLAS-750 > Project: Atlas > Issue Type: Improvement >Reporter: Amikam Snir >Priority: Minor > > Swarm is not well supported by docker_operator, due to this issue: > https://github.com/docker/swarm/issues/475 > In order to fix it, we will use cpu_shares instead of cpus. > p.s. The default value is None. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-730) Change titan hbase table name
Shwetha G S created ATLAS-730: - Summary: Change titan hbase table name Key: ATLAS-730 URL: https://issues.apache.org/jira/browse/ATLAS-730 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Fix For: 0.7-incubating By default, titan uses hbase table name as 'titan' with hbase backend. Change the name to something atlas specific -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-682) Set HBase root dir to be relative to test target directory for HBaseBasedAuditRepositoryTest
[
https://issues.apache.org/jira/browse/ATLAS-682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S reassigned ATLAS-682:
-
Assignee: Shwetha G S
> Set HBase root dir to be relative to test target directory for
> HBaseBasedAuditRepositoryTest
>
>
> Key: ATLAS-682
> URL: https://issues.apache.org/jira/browse/ATLAS-682
> Project: Atlas
> Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
>
> Sometimes when running HBaseBasedAuditRepositoryTest, the test is failing
> with a timeout. Logs reveal a specific kind of stack trace:
> {code}
> 2016-04-18 15:11:44,437 FATAL - [10.200.4.44:49249.activeMasterManager:] ~
> Unhandled exception. Starting shutdown. (HMaster:2098)
> org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout
> needs to be upgraded. You have version null and I want version 8. Consult
> http://hbase.apache.org/book.html for further information about upgrading
> HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck
> -fixVersionFile'.
> at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:677)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:455)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:146)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:126)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:649)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It appears that the HBase root directory is being set to
> /tmp/hbase-{username} and if this has left over data it is failing. Under Mac
> OSX - this is resolving to a very hard to find directory by itself. Filing
> this to see if we can fix the temporary directories to be under the target
> folder of the module so that mvn clean actually will fix problems.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-682) Set HBase root dir to be relative to test target directory for HBaseBasedAuditRepositoryTest
[
https://issues.apache.org/jira/browse/ATLAS-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272113#comment-15272113
]
Shwetha G S edited comment on ATLAS-682 at 5/5/16 9:43 AM:
---
HBaseTestingUtility.createLocalHTU() sets hbase.rootdir to target/test-data
was (Author: shwethags):
HBaseTestingUtility.createLocalHTU() sets hbase.rootdir
> Set HBase root dir to be relative to test target directory for
> HBaseBasedAuditRepositoryTest
>
>
> Key: ATLAS-682
> URL: https://issues.apache.org/jira/browse/ATLAS-682
> Project: Atlas
> Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-682.patch
>
>
> Sometimes when running HBaseBasedAuditRepositoryTest, the test is failing
> with a timeout. Logs reveal a specific kind of stack trace:
> {code}
> 2016-04-18 15:11:44,437 FATAL - [10.200.4.44:49249.activeMasterManager:] ~
> Unhandled exception. Starting shutdown. (HMaster:2098)
> org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout
> needs to be upgraded. You have version null and I want version 8. Consult
> http://hbase.apache.org/book.html for further information about upgrading
> HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck
> -fixVersionFile'.
> at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:677)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:455)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:146)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:126)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:649)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It appears that the HBase root directory is being set to
> /tmp/hbase-{username} and if this has left over data it is failing. Under Mac
> OSX - this is resolving to a very hard to find directory by itself. Filing
> this to see if we can fix the temporary directories to be under the target
> folder of the module so that mvn clean actually will fix problems.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-742) Avoid downloading hbase multiple times
[ https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-742: -- Attachment: ATLAS-742.patch > Avoid downloading hbase multiple times > -- > > Key: ATLAS-742 > URL: https://issues.apache.org/jira/browse/ATLAS-742 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Minor > Fix For: 0.7-incubating > > Attachments: ATLAS-742.patch > > > In distro module, hbase package which is ~100mb is downloaded to target. mvn > clean is required during development for clean build and target gets cleaned > up and hbase is downloaded everytime. Instead, download hbase to a location > outside target and skip download if exists -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-682) Set HBase root dir to be relative to test target directory for HBaseBasedAuditRepositoryTest
[
https://issues.apache.org/jira/browse/ATLAS-682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-682:
--
Attachment: ATLAS-682.patch
> Set HBase root dir to be relative to test target directory for
> HBaseBasedAuditRepositoryTest
>
>
> Key: ATLAS-682
> URL: https://issues.apache.org/jira/browse/ATLAS-682
> Project: Atlas
> Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Shwetha G S
> Attachments: ATLAS-682.patch
>
>
> Sometimes when running HBaseBasedAuditRepositoryTest, the test is failing
> with a timeout. Logs reveal a specific kind of stack trace:
> {code}
> 2016-04-18 15:11:44,437 FATAL - [10.200.4.44:49249.activeMasterManager:] ~
> Unhandled exception. Starting shutdown. (HMaster:2098)
> org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout
> needs to be upgraded. You have version null and I want version 8. Consult
> http://hbase.apache.org/book.html for further information about upgrading
> HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck
> -fixVersionFile'.
> at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:677)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:455)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:146)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:126)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:649)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It appears that the HBase root directory is being set to
> /tmp/hbase-{username} and if this has left over data it is failing. Under Mac
> OSX - this is resolving to a very hard to find directory by itself. Filing
> this to see if we can fix the temporary directories to be under the target
> folder of the module so that mvn clean actually will fix problems.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-747) Hive CTAS entity registration fails because userName is null
[ https://issues.apache.org/jira/browse/ATLAS-747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-747: -- Attachment: ATLAS-747.patch > Hive CTAS entity registration fails because userName is null > > > Key: ATLAS-747 > URL: https://issues.apache.org/jira/browse/ATLAS-747 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-747.patch > > > With apache hive 1.2.x and hive CLI, hookContext.getUserName() is null and > hive_process entity registration fails as userName is required parameter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-670) Entity update/delete operations shouldn't be supported on deleted entities
[ https://issues.apache.org/jira/browse/ATLAS-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-670. --- Resolution: Not A Problem Updates to entities like add some comments(if there is any comments attribute), add/delete tag should still be supported on deleted entities as they are useful. > Entity update/delete operations shouldn't be supported on deleted entities > -- > > Key: ATLAS-670 > URL: https://issues.apache.org/jira/browse/ATLAS-670 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-748) UI: UI should use entity id based lineage and schema
[ https://issues.apache.org/jira/browse/ATLAS-748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-748: -- Summary: UI: UI should use entity id based lineage and schema (was: UI: UI should use entity id based lineage and schema0) > UI: UI should use entity id based lineage and schema > > > Key: ATLAS-748 > URL: https://issues.apache.org/jira/browse/ATLAS-748 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S > Fix For: 0.7-incubating > > > Currently, lineage and schema APIs use table qualified name as argument. With > soft delete, there can be multiple entities for a given table qualified > name(at most one active entity and >=0 deleted entities). In UI, the lineage > and schema is shown in the context of a single entity. So, even lineage and > schema should use the lineage and schema corresponding to that entity > This requires new REST APIs that will be added as part of ATLAS-713 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-748) UI: UI should use entity id based lineage and schema0
Shwetha G S created ATLAS-748: - Summary: UI: UI should use entity id based lineage and schema0 Key: ATLAS-748 URL: https://issues.apache.org/jira/browse/ATLAS-748 Project: Atlas Issue Type: Improvement Reporter: Shwetha G S Fix For: 0.7-incubating Currently, lineage and schema APIs use table qualified name as argument. With soft delete, there can be multiple entities for a given table qualified name(at most one active entity and >=0 deleted entities). In UI, the lineage and schema is shown in the context of a single entity. So, even lineage and schema should use the lineage and schema corresponding to that entity This requires new REST APIs that will be added as part of ATLAS-713 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-443) Add test for hive create table without columns
[ https://issues.apache.org/jira/browse/ATLAS-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-443: -- Assignee: (was: Shwetha G S) > Add test for hive create table without columns > -- > > Key: ATLAS-443 > URL: https://issues.apache.org/jira/browse/ATLAS-443 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S > Fix For: trunk > > > Add test for issue fixed in ATLAS-409 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-443) Add test for hive create table without columns
[ https://issues.apache.org/jira/browse/ATLAS-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-443: -- Attachment: (was: ATLAS-443.patch) > Add test for hive create table without columns > -- > > Key: ATLAS-443 > URL: https://issues.apache.org/jira/browse/ATLAS-443 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: trunk > > > Add test for issue fixed in ATLAS-409 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-742) Avoid downloading hbase multiple times
[ https://issues.apache.org/jira/browse/ATLAS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272084#comment-15272084 ] Shwetha G S commented on ATLAS-742: --- Instead of modifying global state, outside of the project(which can have concurrency issues), I was planning to write to /distro/hbase(outside of target) > Avoid downloading hbase multiple times > -- > > Key: ATLAS-742 > URL: https://issues.apache.org/jira/browse/ATLAS-742 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Minor > > In distro module, hbase package which is ~100mb is downloaded to target. mvn > clean is required during development for clean build and target gets cleaned > up and hbase is downloaded everytime. Instead, download hbase to a location > outside target and skip download if exists -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-585) NotificationHookConsumer creates new AtlasClient for every message
[ https://issues.apache.org/jira/browse/ATLAS-585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-585: -- Attachment: ATLAS-585-v2.patch > NotificationHookConsumer creates new AtlasClient for every message > -- > > Key: ATLAS-585 > URL: https://issues.apache.org/jira/browse/ATLAS-585 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-585-v2.patch, ATLAS-585.patch > > > Every AtlasClient instance creates com.sun.jersey.api.client.Client which is > expensive. As part of ATLAS-577, NotificationHookConsumer creates new atlas > client for every message. This is done to pass the user information(the user > who created the message/entities) through the APIs. This wouldn't work in > kerberos environment where the credentials are not available on the atlas > server. We should probably make the API call as atlas user and pass in the > actual user information in another query param(like doas user) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-747) Hive CTAS entity registration fails because userName is null
Shwetha G S created ATLAS-747: - Summary: Hive CTAS entity registration fails because userName is null Key: ATLAS-747 URL: https://issues.apache.org/jira/browse/ATLAS-747 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Assignee: Shwetha G S Fix For: 0.7-incubating With apache hive 1.2.x and hive CLI, hookContext.getUserName() is null and hive_process entity registration fails as userName is required parameter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
