[jira] [Commented] (ATLAS-3475) Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954243#comment-16954243
 ] 

ASF subversion and git services commented on ATLAS-3475:


Commit 686c23d6a7e85575f452116b240f9fa0cd098dae in atlas's branch 
refs/heads/branch-2.0 from Sarath Subramanian
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=686c23d ]

ATLAS-3475: Invalid instance creation/updation parameters passed : 
hive_process.qualifiedName

(cherry picked from commit f9b483175a0a54c2cde417cde42a04f0e4e4cf44)


> Invalid instance creation/updation parameters passed : 
> hive_process.qualifiedName: mandatory attribute value missing in type 
> Referenceable
> --
>
> Key: ATLAS-3475
> URL: https://issues.apache.org/jira/browse/ATLAS-3475
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: ATLAS-3475.001.patch
>
>
> While running CTAS query via spark.sql, Atlas is throwing below error
> {code:java}
> org.apache.atlas.exception.AtlasBaseException: Invalid instance 
> creation/updation parameters passed : hive_process.qualifiedName: mandatory 
> attribute value missing in type 
> Referenceableorg.apache.atlas.exception.AtlasBaseException: Invalid instance 
> creation/updation parameters passed : hive_process.qualifiedName: mandatory 
> attribute value missing in type Referenceable at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:103)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:938)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:748)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:279)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke()
>  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
>  at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>  at 
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
>  at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$91e799f5.createOrUpdate()
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:802)
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:700)
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:521)
>  at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:89) at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748){code}
>  The above error comes even in a newly created cluster, where there are 
> (were) no shell entities present.
> the error message is mostly harmless since the entities are getting created 
> properly. Shell entities too are getting resolved correctly.
> But in the metrics UI, each CTAS query shows up as a failure under 
> ATLAS_HOOK-0 failed messages 
> Steps to reproduce:
>  # spark-shell
>  # spark.sql("create table t1(name string)")
>  # spark.sql("create table t2 as select * from t1")



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3475) Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954240#comment-16954240
 ] 

ASF subversion and git services commented on ATLAS-3475:


Commit f9b483175a0a54c2cde417cde42a04f0e4e4cf44 in atlas's branch 
refs/heads/master from Sarath Subramanian
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=f9b4831 ]

ATLAS-3475: Invalid instance creation/updation parameters passed : 
hive_process.qualifiedName


> Invalid instance creation/updation parameters passed : 
> hive_process.qualifiedName: mandatory attribute value missing in type 
> Referenceable
> --
>
> Key: ATLAS-3475
> URL: https://issues.apache.org/jira/browse/ATLAS-3475
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: ATLAS-3475.001.patch
>
>
> While running CTAS query via spark.sql, Atlas is throwing below error
> {code:java}
> org.apache.atlas.exception.AtlasBaseException: Invalid instance 
> creation/updation parameters passed : hive_process.qualifiedName: mandatory 
> attribute value missing in type 
> Referenceableorg.apache.atlas.exception.AtlasBaseException: Invalid instance 
> creation/updation parameters passed : hive_process.qualifiedName: mandatory 
> attribute value missing in type Referenceable at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:103)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:938)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:748)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:279)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke()
>  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
>  at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>  at 
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
>  at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$91e799f5.createOrUpdate()
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:802)
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:700)
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:521)
>  at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:89) at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748){code}
>  The above error comes even in a newly created cluster, where there are 
> (were) no shell entities present.
> the error message is mostly harmless since the entities are getting created 
> properly. Shell entities too are getting resolved correctly.
> But in the metrics UI, each CTAS query shows up as a failure under 
> ATLAS_HOOK-0 failed messages 
> Steps to reproduce:
>  # spark-shell
>  # spark.sql("create table t1(name string)")
>  # spark.sql("create table t2 as select * from t1")



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-2682) Atlas startup fails when configuring External HBase.

[cdmikechen (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954235#comment-16954235
 ] 

cdmikechen commented on ATLAS-2682:
---

You need to copy core-site.xml and hdfs-site.xml to hbase conf dir.

> Atlas startup fails when configuring External HBase.
> 
>
> Key: ATLAS-2682
> URL: https://issues.apache.org/jira/browse/ATLAS-2682
> Project: Atlas
>  Issue Type: Bug
>Reporter: Kwangsik Lee
>Priority: Major
>
> Hi, guys. 
> I'm trying to set Atlas with our existing Hbase which is 1.0.0.
> I used Atlas-1.0.0-SNAPSHOT, but I encountered the error message with below 
> settings.
> Can you help me out to figure out what I can fix this issue?
> Thanks.
> {code:java}
> atlas.graph.storage.backend=hbase
> atlas.graph.storage.hbase.table=apache_atlas_janus
> #Hbase
> #For standalone mode , specify localhost
> #for distributed mode, specify zookeeper quorum here
> #atlas.graph.storage.hostname=localhost
> atlas.graph.storage.hostname=server:2181,server:2181,server:2181,server:2181,server:2181
> atlas.graph.storage.hbase.regions-per-server=1
> atlas.graph.storage.lock.wait-time=1
> {code}
> {code:java}
> export HBASE_CONF_DIR=/home/user/hbase/conf
> {code}
> {code:java}
> 2018-05-15 12:29:42,838 ERROR - [main:] ~ Context initialization failed 
> (ContextLoader:350) org.springframework.beans.factory.BeanCreationException: 
> Error creating bean with name 
> 'org.springframework.context.event.internalEventListenerProcessor': 
> BeanPostProcessor before instantiation of bean failed; nested exception is 
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
> creating bean with name 'graphTransactionAdvisor' defined in URL 
> [jar:file:/home/suser/apache-atlas-1.0.0-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0-SNAPSHOT.jar!/org/apache/atlas/GraphTransactionAdvisor.class]:
>  Unsatisfied dependency expressed through constructor parameter 0; nested 
> exception is 
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
> creating bean with name 'graphTransactionInterceptor' defined in URL 
> [jar:file:/home/suser/apache-atlas-1.0.0-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0-SNAPSHOT.jar!/org/apache/atlas/GraphTransactionInterceptor.class]:
>  Unsatisfied dependency expressed through constructor parameter 0; nested 
> exception is org.springframework.beans.factory.BeanCreationException: Error 
> creating bean with name 'get' defined in 
> org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiation via 
> factory method failed; nested exception is 
> org.springframework.beans.BeanInstantiationException: Failed to instantiate 
> [org.apache.atlas.repository.graphdb.AtlasGraph]: Factory method 'get' threw 
> exception; nested exception is java.lang.IllegalArgumentException: Could not 
> instantiate implementation: 
> org.janusgraph.diskstorage.hbase.HBaseStoreManager at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:479)
>  at 
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
>  at 
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
>  at 
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
>  at 
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
>  at 
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
>  at 
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:866)
>  at 
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
>  at 
> org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
>  at 
> org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
>  at 
> org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
>  at 
> org.apache.atlas.web.setup.KerberosAwareListener.contextInitialized(KerberosAwareListener.java:31)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:843)
>  at 
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:816)
>  at 
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
>  at 
> 

Re: Review Request 71619: ATLAS-3464: Define Entities stored in Atlas for ML Governance

[Na Li via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/
---

(Updated Oct. 18, 2019, 2 a.m.)


Review request for atlas, Austin Nobis, Ashutosh Mestry, Karthik Manamcheri, 
Sridhar K, Madhan Neethiraj, and Sarath Subramanian.


Bugs: atlas-3464
https://issues.apache.org/jira/browse/atlas-3464


Repository: atlas


Description
---

Define entities used for Machine Learning Governance


Diffs (updated)
-

  addons/models/1000-Hadoop/-ml_model.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71619/diff/3/

Changes: https://reviews.apache.org/r/71619/diff/2-3/


Testing
---

verified it is valid json file


Thanks,

Na Li

[jira] [Assigned] (ATLAS-3447) Basic Search API does not return child classifications

[Le Ma (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Le Ma reassigned ATLAS-3447:


Assignee: Le Ma

> Basic Search API does not return child classifications
> --
>
> Key: ATLAS-3447
> URL: https://issues.apache.org/jira/browse/ATLAS-3447
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Le Ma
>Priority: Major
> Attachments: 
> child_classification_does_not_appear_after_parent_classification_is_reapplied.webm,
>  
> childclassification_vanishes_after_parent_classification_association_is_empty.webm
>
>
> Detailed explanation of the issue along with an additional related issue(with 
> videos attached).
> Scenario I:  
> [^childclassification_vanishes_after_parent_classification_association_is_empty.webm]
> Entity E1 is associated with classification 'Parent' and entity E2 is 
> associated with classification 'Child' where 'Child' is subclassification of 
> 'Parent'.
>  * In the 'Parent' classification details page, E1 and E2 are visible.
>  * 'Parent' classification is removed from E1.
>  * When we view Parent classification details, E1 no longer appears, but even 
> E2 is removed, which is incorrect as the 'Child' is a subclassification of 
> 'Parent' and should be visible on the page as the option "Exclude 
> subclassifications" is not selected.
>  
> Scenario II: 
> [^child_classification_does_not_appear_after_parent_classification_is_reapplied.webm]
> After performing steps from scenario I, 
>  *  Reassign 'Parent' classification to E1
>  * Navigate to 'Parent' classification details page.
>  * E1 is still the only entity visible, even E2 should also be visible as per 
> step 1 in Scenario I.
>  * Now, remove 'Child' classification from E2 and then reapply 'Child' to E2.
>  * Navigate again to 'Parent' classification details page.
>  * You will see that now E1 and E2 appear as expected. This seems to be a 
> result of a stale connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71619: ATLAS-3464: Define Entities stored in Atlas for ML Governance

[Na Li via Review Board]

> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 336 (patched)
> > 
> >
> > Like model builds, recommend adding an attribute for number of GPU's.

done


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 339 (patched)
> > 
> >
> > Would this be things like pod ID's? If we want to expose this kind of 
> > low-level, debugging information then let's make it complete enough to 
> > support most debugging use cases. 
> > 
> > For example, in order to take action based on the pod ID's, a cluster 
> > admin would need to know the k8s API server API, the kubeconfig, and the 
> > namespace in which the model is running.
> > 
> > I think we could leave this field out for now, add an unstructured 
> > 'metadata' field and tackle the debugging flows later.

I removed it


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 346 (patched)
> > 
> >
> > Should we add number of replicas?

done


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 361 (patched)
> > 
> >
> > Should this be optional?

it is removed


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 373 (patched)
> > 
> >
> > Is this for things like admin/full user/viewer only? If so, why make it 
> > non-optional?

changed it to optional. I am thinking the type is admin/user, mlgov/service etc.


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 384-385 (patched)
> > 
> >
> > There currently is no such process in CML, projects are always created 
> > by user clicks in the UI.
> > 
> > Oh, is this just here to define lineage relationships betwene users and 
> > projects?

yes. The user is associated with a process, and the process creates a project. 
A user can have a lot of metadata. We don't want to duplicate the info to a 
project.


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 415 (patched)
> > 
> >
> > Same question about whether uniqueId should really be optional.

removed.


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 442 (patched)
> > 
> >
> > The name above is just ml_project_create_process.

This is relationship between ml_user and ml_project_create_process


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 480 (patched)
> > 
> >
> > What's the reason for the ml_user prefix here and on 
> > ml_user_model_deploy_process?

it is a relationship involves two types


- Na


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/#review218231
---


On Oct. 16, 2019, 12:30 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71619/
> ---
> 
> (Updated Oct. 16, 2019, 12:30 a.m.)
> 
> 
> Review request for atlas, Austin Nobis, Ashutosh Mestry, Karthik Manamcheri, 
> Sridhar K, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: atlas-3464
> https://issues.apache.org/jira/browse/atlas-3464
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Define entities used for Machine Learning Governance
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/-ml_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71619/diff/1/
> 
> 
> Testing
> ---
> 
> verified it is valid json file
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 71619: ATLAS-3464: Define Entities stored in Atlas for ML Governance

[Na Li via Review Board]

> On Oct. 16, 2019, 4:45 p.m., Karthik Manamcheri wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 286 (patched)
> > 
> >
> > What is the purpose of "updatedAt"?

I removed it. if user wants it, can add it in attribute "metadata"


> On Oct. 16, 2019, 4:45 p.m., Karthik Manamcheri wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 349 (patched)
> > 
> >
> > How is a ML User different from a regular user?

it is a user involved in ML actions. There is no base type we can re-use.

__AtlasUserProfile is used internally, totally different purpose from ours.


> On Oct. 16, 2019, 4:45 p.m., Karthik Manamcheri wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 447 (patched)
> > 
> >
> > I question the need for a ml_user at this point. I don't completely 
> > understand how or why a ml_user is different from any other user in the 
> > system?
> > 
> > Should this just be "user"?

I prefix each type with "ml_" to avoid type conflict. Other integration may 
change to use "user" for a totally different purpose. We do need to have a type 
for user.


- Na


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/#review218235
---


On Oct. 16, 2019, 12:30 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71619/
> ---
> 
> (Updated Oct. 16, 2019, 12:30 a.m.)
> 
> 
> Review request for atlas, Austin Nobis, Ashutosh Mestry, Karthik Manamcheri, 
> Sridhar K, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: atlas-3464
> https://issues.apache.org/jira/browse/atlas-3464
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Define entities used for Machine Learning Governance
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/-ml_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71619/diff/1/
> 
> 
> Testing
> ---
> 
> verified it is valid json file
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 71619: ATLAS-3464: Define Entities stored in Atlas for ML Governance

[Na Li via Review Board]

> On Oct. 16, 2019, 7:36 p.m., Na Li wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 51 (patched)
> > 
> >
> > Those fields are optional. So if some projects cannot provide those 
> > fields, that is OK. The attributes I put here are used in several projects.
> > 
> > If we put several "key:value" pairs in a single string, we risk losing 
> > some info. 
> > 
> > I think the big drawback of putting several key:value pairs in a single 
> > json string is that:some can be lost by accident. For example the 
> > customAttributes = "{githubRepoURL:https://host1/project1 }". Then later 
> > on, an update comes in with customAttributes = "{mlFramework:tensorFlow}" 
> > because that application has no idea someone is tracking githubRepoURL, or 
> > did not set githubRepoURL because only mlFramework has changed. Then the 
> > previous info on where the source code is got lost by accident. It is hard 
> > to debug. Besides, each time someone wants to use the info, has to parse 
> > the string.
> > 
> > The counter argument can be how about putting these attributes here. If 
> > they are not popular, we just don't use them.
> > 
> > The following info is from Ashutosh. 
> > Having an attribute that contains json blob is not searchable at Atlas 
> > in general. 
> > 
> > Other benefit of adding an attribute to the model is that entities 
> > created will get validated for the type
> > because of validation, we can be certain about the data present in that 
> > field
> > 
> > A hacky approach to add json blob and make it searchable is to set them 
> > in customAttributes, which is system property. We cannot see what keys are 
> > used in customAttributes at model files. it is set by whoever creates the 
> > Atlas entity.
> 
> Anand Patil wrote:
> Instead of a string type containing json, what about a 
> map? I see that type used in some other attributes.

That is a really good idea without the risk of lossing info!

I tested Atlas behavior and Atlas can update part of an attribute's value if 
its type is map<>. I modified a HMS hook test in HiveMetastoreHookIT. 
Basically, the hive_table.parameters attribute's type is map. 
When I add another parameter after it has two parameters, all three parameters 
were stored and retrieved successfully.

I will reduce the attributes and have a general purpose attribute of 
map. So users can experiment and we can know what attributes 
are universally useful.


- Na


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/#review218237
---


On Oct. 16, 2019, 12:30 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71619/
> ---
> 
> (Updated Oct. 16, 2019, 12:30 a.m.)
> 
> 
> Review request for atlas, Austin Nobis, Ashutosh Mestry, Karthik Manamcheri, 
> Sridhar K, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: atlas-3464
> https://issues.apache.org/jira/browse/atlas-3464
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Define entities used for Machine Learning Governance
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/-ml_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71619/diff/1/
> 
> 
> Testing
> ---
> 
> verified it is valid json file
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 71619: ATLAS-3464: Define Entities stored in Atlas for ML Governance

[Na Li via Review Board]

> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 165 (patched)
> > 
> >
> > Same comment about metadata: gitCommitId, hyperParameters, 
> > outcomeTypeDescription feel prematurely standardized. I feel that a 
> > map metadata field would be better suited to where we are 
> > now.

I will remove those attributes and add a general purpose attribute "metadata"


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 175 (patched)
> > 
> >
> > It's slightly awkward to have to stringify all hyperparameter values. 
> > Does Atlas have option types?

it is removed


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 197 (patched)
> > 
> >
> > If status is non-optional, it should probably be an enum or Altus 
> > equivalent.
> > 
> > Also, I'm not sure this field is required for model builds as opposed 
> > to model deployments. Builds' status is very simple - building, built or 
> > errored I suppose. We don't necessarily need to represent building or 
> > errored builds in Atlas.

It is removed.


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 220 (patched)
> > 
> >
> > I'm not sure defaultReplicas belongs on the model build. The correct 
> > number of replicas depends on usage, it's not really a property of the 
> > model itself.

It is removed.


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 228 (patched)
> > 
> >
> > "imageTag" would be more standard terminology. The tag typically 
> > includes the entire image URL.
> > 
> > Consider also adding the image hash, as tags/url's are not necessarily 
> > unique.

fixed


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 268 (patched)
> > 
> >
> > We don't need a uniqueID for model deployments?

"qualifiedName" contains unique ID. no need for this.


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 306 (patched)
> > 
> >
> > I don't think we shoud expose serviceIP. It may not be distinct from 
> > modelEndpointURL in non-Kubernetes serving environments and when it is, we 
> > may not want to encourage use of it. For example, when a model is being 
> > called from outside the k8s cluster where it is running, the service IP is 
> > not usable.

it is removed.


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 319 (patched)
> > 
> >
> > If this is required I think it should be an enum or equivalent in Atlas.

changed to enum


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 323 (patched)
> > 
> >
> > Use the same units for CPU and memory here as in the defaults for model 
> > build. I would go with millicores and bytes.

I changed the CPU unit to millicores. and still keep memory unit in Mb. Is a 
unit of bytes too small?


- Na


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/#review218231
---


On Oct. 16, 2019, 12:30 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71619/
> ---
> 
> (Updated Oct. 16, 2019, 12:30 a.m.)
> 
> 
> Review request for atlas, Austin Nobis, Ashutosh Mestry, Karthik Manamcheri, 
> Sridhar K, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: atlas-3464
> https://issues.apache.org/jira/browse/atlas-3464
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Define entities used for Machine Learning Governance
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/-ml_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71619/diff/1/
> 
> 
> Testing
> ---
> 
> verified it is valid json file
> 
> 
> Thanks,
> 
> Na 

Re: Review Request 71619: ATLAS-3464: Define Entities stored in Atlas for ML Governance

[Na Li via Review Board]

> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 10 (patched)
> > 
> >
> > Why make project a subtype of dataset?

Atlas server has a lot of code that build lineage based on Process and DataSet 
types

1) Process only handle types derives from `DataSet`
   {
  "name": "Process",
  "superTypes": [
"Asset"
  ],
  "serviceType": "atlas_core",
  "typeVersion": "1.1",
  "attributeDefs": [
{
  "name": "inputs",
  "typeName": "array",
  "cardinality": "SET",
  "isIndexable": false,
  "isOptional": true,
  "isUnique": false
},
{
  "name": "outputs",
  "typeName": "array",
  "cardinality": "SET",
  "isIndexable": false,
  "isOptional": true,
  "isUnique": false
}
  ]
}

2) Lineage is only built between Process type and DataSet type. 
https://github.com/apache/atlas/blob/master/repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java#L117
3) If we don't want to re-implement the lineage in Atlas, it is better to 
deriver action type from Process type, and data type from DataSet.


> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 51 (patched)
> > 
> >
> > businessUseCase, modelFramework, modelAlgorithms, githubRepoURL, 
> > notebookURL and resourceURL feel prematurely opinionated to me. Some 
> > projects will not use all these fields, and for some projects fields other 
> > than these will probably be more important.
> > 
> > Would it be possible in Atlas to move these to a single "metadata" 
> > attribute whose value is key-value pairs? We can then move to stronger 
> > typing as common patterns emerge.

replace those attributes by

{
  "name": "metadata",
  "description": "Contains key-value pairs that provide project 
metadata",
  "typeName": "map",
  "cardinality": "SINGLE",
  "isIndexable": false,
  "isOptional": true,
  "isUnique": false
}


- Na


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/#review218231
---


On Oct. 16, 2019, 12:30 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71619/
> ---
> 
> (Updated Oct. 16, 2019, 12:30 a.m.)
> 
> 
> Review request for atlas, Austin Nobis, Ashutosh Mestry, Karthik Manamcheri, 
> Sridhar K, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: atlas-3464
> https://issues.apache.org/jira/browse/atlas-3464
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Define entities used for Machine Learning Governance
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/-ml_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71619/diff/1/
> 
> 
> Testing
> ---
> 
> verified it is valid json file
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 71619: ATLAS-3464: Define Entities stored in Atlas for ML Governance

[Na Li via Review Board]

> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 18 (patched)
> > 
> >
> > Can this really be optional?

I will drop this attribute, and user "qualifiedName", which must be unique.


- Na


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/#review218231
---


On Oct. 16, 2019, 12:30 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71619/
> ---
> 
> (Updated Oct. 16, 2019, 12:30 a.m.)
> 
> 
> Review request for atlas, Austin Nobis, Ashutosh Mestry, Karthik Manamcheri, 
> Sridhar K, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: atlas-3464
> https://issues.apache.org/jira/browse/atlas-3464
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Define entities used for Machine Learning Governance
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/-ml_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71619/diff/1/
> 
> 
> Testing
> ---
> 
> verified it is valid json file
> 
> 
> Thanks,
> 
> Na Li
> 
>

[jira] [Updated] (ATLAS-3477) Introduce purging of entity in Atlas

[Sidharth Kumar Mishra (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sidharth Kumar Mishra updated ATLAS-3477:
-
Description: 
In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there is 
no way to do cleanup if user wants to delete those permanently. Right now we 
can change the configuration 

atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.

HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
desired. But what happens to existing soft deleted Entities. There is no way in 
that case.

There are two cases which needs to be addressed as part of this problem.
 # There is an active entity which has to be purged in case the entity is 
dropped.
 # There might be existing soft deleted entities and which has to be purge 
deleted if user desires

In both the above cases we also need to think about the auditing and its impact.

  was:
In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there is 
no way to do cleanup if user wants to delete those permanently. Right now we 
can change the configuration 

atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.

HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
desired. But what happens to existing soft deleted Entities. There is no way in 
that case.

There are two cases which needs to be addressed as part of this problem.
 # There are active entity which has to be purged in case the entity is dropped.
 # There might be existing soft deleted entities and which has to be purge 
deleted

In both the above cases we also need to think about the auditing and its impact.


> Introduce purging of entity in Atlas
> 
>
> Key: ATLAS-3477
> URL: https://issues.apache.org/jira/browse/ATLAS-3477
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Sidharth Kumar Mishra
>Assignee: Sidharth Kumar Mishra
>Priority: Major
>
> In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there 
> is no way to do cleanup if user wants to delete those permanently. Right now 
> we can change the configuration 
> atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.
> HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
> desired. But what happens to existing soft deleted Entities. There is no way 
> in that case.
> There are two cases which needs to be addressed as part of this problem.
>  # There is an active entity which has to be purged in case the entity is 
> dropped.
>  # There might be existing soft deleted entities and which has to be purge 
> deleted if user desires
> In both the above cases we also need to think about the auditing and its 
> impact.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3476) Regression : Case insensitivity in tag filters in search

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3476:
--
Labels: basic-search  (was: )

> Regression : Case insensitivity in tag filters in search
> 
>
> Key: ATLAS-3476
> URL: https://issues.apache.org/jira/browse/ATLAS-3476
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
>  Labels: basic-search
> Fix For: 2.1.0
>
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` fo `attr1`
> 3. Fire basic search for tag1 with filters attr1 = VALUE1
> Entity is returned though there is case mismatch.
>  
> This doesn't happen with entity filters.
> for example , for hive_table sample_08 , 
> basic search :
> type = hive_table,
> filters , name = SAMPLE_08 doesn't fetch the entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3476) Regression : Case insensitivity in tag filters in search

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3476:
--
Component/s:  atlas-core

> Regression : Case insensitivity in tag filters in search
> 
>
> Key: ATLAS-3476
> URL: https://issues.apache.org/jira/browse/ATLAS-3476
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
> Fix For: 2.1.0
>
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` fo `attr1`
> 3. Fire basic search for tag1 with filters attr1 = VALUE1
> Entity is returned though there is case mismatch.
>  
> This doesn't happen with entity filters.
> for example , for hive_table sample_08 , 
> basic search :
> type = hive_table,
> filters , name = SAMPLE_08 doesn't fetch the entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3476) Regression : Case insensitivity in tag filters in search

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3476:
--
Affects Version/s: 2.0.0

> Regression : Case insensitivity in tag filters in search
> 
>
> Key: ATLAS-3476
> URL: https://issues.apache.org/jira/browse/ATLAS-3476
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` fo `attr1`
> 3. Fire basic search for tag1 with filters attr1 = VALUE1
> Entity is returned though there is case mismatch.
>  
> This doesn't happen with entity filters.
> for example , for hive_table sample_08 , 
> basic search :
> type = hive_table,
> filters , name = SAMPLE_08 doesn't fetch the entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3476) Regression : Case insensitivity in tag filters in search

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3476:
--
Fix Version/s: 2.1.0

> Regression : Case insensitivity in tag filters in search
> 
>
> Key: ATLAS-3476
> URL: https://issues.apache.org/jira/browse/ATLAS-3476
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
> Fix For: 2.1.0
>
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` fo `attr1`
> 3. Fire basic search for tag1 with filters attr1 = VALUE1
> Entity is returned though there is case mismatch.
>  
> This doesn't happen with entity filters.
> for example , for hive_table sample_08 , 
> basic search :
> type = hive_table,
> filters , name = SAMPLE_08 doesn't fetch the entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3477) Introduce purging of entity in Atlas

[Sidharth Kumar Mishra (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sidharth Kumar Mishra updated ATLAS-3477:
-
Description: 
In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there is 
no way to do cleanup if user wants to delete those permanently. Right now we 
can change the configuration 

atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.

HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
desired. But what happens to existing soft deleted Entities. There is no way in 
that case.

There are two cases which needs to be addressed as part of this problem.
 # There are active entity which has to be purged in case the entity is dropped.
 # There might be existing soft deleted entities and which has to be purge 
deleted

In both the above cases we also need to think about the auditing and its impact.

  was:
In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there is 
no way to do cleanup if user wants to delete those permanently. Right now we 
can change the configuration 

atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.

HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
desired. But what happens to existing soft deleted Entities. There is no way in 
that case.


> Introduce purging of entity in Atlas
> 
>
> Key: ATLAS-3477
> URL: https://issues.apache.org/jira/browse/ATLAS-3477
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Sidharth Kumar Mishra
>Assignee: Sidharth Kumar Mishra
>Priority: Major
>
> In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there 
> is no way to do cleanup if user wants to delete those permanently. Right now 
> we can change the configuration 
> atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.
> HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
> desired. But what happens to existing soft deleted Entities. There is no way 
> in that case.
> There are two cases which needs to be addressed as part of this problem.
>  # There are active entity which has to be purged in case the entity is 
> dropped.
>  # There might be existing soft deleted entities and which has to be purge 
> deleted
> In both the above cases we also need to think about the auditing and its 
> impact.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3467) Regression : Tag filters (begins_with , ends_with , contains) and space fetches incorrect results

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3467:
--
Affects Version/s: 2.0.0

> Regression : Tag filters (begins_with , ends_with , contains) and space 
> fetches incorrect results
> -
>
> Key: ATLAS-3467
> URL: https://issues.apache.org/jira/browse/ATLAS-3467
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` for `attr1`
> 3. Fire basic search for tag1 with filters:
> "attr1" contains "new comment"
>  
> Entity is returned , though "value1" is not related to "new comment" at all.
>  
> Happens with contains/begins_with/ends_with filters and not with =/!=
> Happens with filters with space (ex : new comment and not with newcomment or 
> new_comment)
> Happens only with tag filters and not with entity filters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3467) Regression : Tag filters (begins_with , ends_with , contains) and space fetches incorrect results

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3467:
--
Component/s:  atlas-core

> Regression : Tag filters (begins_with , ends_with , contains) and space 
> fetches incorrect results
> -
>
> Key: ATLAS-3467
> URL: https://issues.apache.org/jira/browse/ATLAS-3467
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
> Fix For: 2.1.0
>
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` for `attr1`
> 3. Fire basic search for tag1 with filters:
> "attr1" contains "new comment"
>  
> Entity is returned , though "value1" is not related to "new comment" at all.
>  
> Happens with contains/begins_with/ends_with filters and not with =/!=
> Happens with filters with space (ex : new comment and not with newcomment or 
> new_comment)
> Happens only with tag filters and not with entity filters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3467) Regression : Tag filters (begins_with , ends_with , contains) and space fetches incorrect results

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3467:
--
Fix Version/s: 2.1.0

> Regression : Tag filters (begins_with , ends_with , contains) and space 
> fetches incorrect results
> -
>
> Key: ATLAS-3467
> URL: https://issues.apache.org/jira/browse/ATLAS-3467
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
> Fix For: 2.1.0
>
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` for `attr1`
> 3. Fire basic search for tag1 with filters:
> "attr1" contains "new comment"
>  
> Entity is returned , though "value1" is not related to "new comment" at all.
>  
> Happens with contains/begins_with/ends_with filters and not with =/!=
> Happens with filters with space (ex : new comment and not with newcomment or 
> new_comment)
> Happens only with tag filters and not with entity filters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3467) Regression : Tag filters (begins_with , ends_with , contains) and space fetches incorrect results

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954106#comment-16954106
 ] 

ASF subversion and git services commented on ATLAS-3467:


Commit b064d8c3aad588c4975ddd0ad5bc24e12806e171 in atlas's branch 
refs/heads/master from Le Ma
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=b064d8c ]

ATLAS-3467 - Regression: Tagfilters (begins_with, ends_with, contains) and 
space fetches incorrect results
ATLAS-3476 - Regression: Case insensitivity in tag filters in search

Signed-off-by: Sarath Subramanian 


> Regression : Tag filters (begins_with , ends_with , contains) and space 
> fetches incorrect results
> -
>
> Key: ATLAS-3467
> URL: https://issues.apache.org/jira/browse/ATLAS-3467
> Project: Atlas
>  Issue Type: Bug
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` for `attr1`
> 3. Fire basic search for tag1 with filters:
> "attr1" contains "new comment"
>  
> Entity is returned , though "value1" is not related to "new comment" at all.
>  
> Happens with contains/begins_with/ends_with filters and not with =/!=
> Happens with filters with space (ex : new comment and not with newcomment or 
> new_comment)
> Happens only with tag filters and not with entity filters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3476) Regression : Case insensitivity in tag filters in search

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954107#comment-16954107
 ] 

ASF subversion and git services commented on ATLAS-3476:


Commit b064d8c3aad588c4975ddd0ad5bc24e12806e171 in atlas's branch 
refs/heads/master from Le Ma
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=b064d8c ]

ATLAS-3467 - Regression: Tagfilters (begins_with, ends_with, contains) and 
space fetches incorrect results
ATLAS-3476 - Regression: Case insensitivity in tag filters in search

Signed-off-by: Sarath Subramanian 


> Regression : Case insensitivity in tag filters in search
> 
>
> Key: ATLAS-3476
> URL: https://issues.apache.org/jira/browse/ATLAS-3476
> Project: Atlas
>  Issue Type: Bug
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` fo `attr1`
> 3. Fire basic search for tag1 with filters attr1 = VALUE1
> Entity is returned though there is case mismatch.
>  
> This doesn't happen with entity filters.
> for example , for hive_table sample_08 , 
> basic search :
> type = hive_table,
> filters , name = SAMPLE_08 doesn't fetch the entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3476) Regression : Case insensitivity in tag filters in search

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954109#comment-16954109
 ] 

ASF subversion and git services commented on ATLAS-3476:


Commit 0cd458b1a8e4c2fd1e88ad1a986c753294084027 in atlas's branch 
refs/heads/branch-2.0 from Le Ma
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=0cd458b ]

ATLAS-3467 - Regression: Tagfilters (begins_with, ends_with, contains) and 
space fetches incorrect results
ATLAS-3476 - Regression: Case insensitivity in tag filters in search

Signed-off-by: Sarath Subramanian 
(cherry picked from commit b064d8c3aad588c4975ddd0ad5bc24e12806e171)


> Regression : Case insensitivity in tag filters in search
> 
>
> Key: ATLAS-3476
> URL: https://issues.apache.org/jira/browse/ATLAS-3476
> Project: Atlas
>  Issue Type: Bug
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` fo `attr1`
> 3. Fire basic search for tag1 with filters attr1 = VALUE1
> Entity is returned though there is case mismatch.
>  
> This doesn't happen with entity filters.
> for example , for hive_table sample_08 , 
> basic search :
> type = hive_table,
> filters , name = SAMPLE_08 doesn't fetch the entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3477) Introduce purging of entity in Atlas

[Sidharth Kumar Mishra (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sidharth Kumar Mishra updated ATLAS-3477:
-
Summary: Introduce purging of entity in Atlas  (was: No way to purge the 
soft deleted entity from Atlas)

> Introduce purging of entity in Atlas
> 
>
> Key: ATLAS-3477
> URL: https://issues.apache.org/jira/browse/ATLAS-3477
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Sidharth Kumar Mishra
>Assignee: Sidharth Kumar Mishra
>Priority: Major
>
> In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there 
> is no way to do cleanup if user wants to delete those permanently. Right now 
> we can change the configuration 
> atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.
> HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
> desired. But what happens to existing soft deleted Entities. There is no way 
> in that case.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3477) No way to purge the soft deleted entity from Atlas

[Sidharth Kumar Mishra (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sidharth Kumar Mishra updated ATLAS-3477:
-
Issue Type: New Feature  (was: Bug)

> No way to purge the soft deleted entity from Atlas
> --
>
> Key: ATLAS-3477
> URL: https://issues.apache.org/jira/browse/ATLAS-3477
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Sidharth Kumar Mishra
>Assignee: Sidharth Kumar Mishra
>Priority: Major
>
> In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there 
> is no way to do cleanup if user wants to delete those permanently. Right now 
> we can change the configuration 
> atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.
> HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
> desired. But what happens to existing soft deleted Entities. There is no way 
> in that case.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (ATLAS-3477) No way to purge the soft deleted entity from Atlas

[Sidharth Kumar Mishra (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sidharth Kumar Mishra reassigned ATLAS-3477:


Assignee: Sidharth Kumar Mishra

> No way to purge the soft deleted entity from Atlas
> --
>
> Key: ATLAS-3477
> URL: https://issues.apache.org/jira/browse/ATLAS-3477
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sidharth Kumar Mishra
>Assignee: Sidharth Kumar Mishra
>Priority: Major
>
> In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there 
> is no way to do cleanup if user wants to delete those permanently. Right now 
> we can change the configuration 
> atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.
> HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
> desired. But what happens to existing soft deleted Entities. There is no way 
> in that case.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3477) No way to purge the soft deleted entity from Atlas

[Sidharth Kumar Mishra (Jira)]

Sidharth Kumar Mishra created ATLAS-3477:


 Summary: No way to purge the soft deleted entity from Atlas
 Key: ATLAS-3477
 URL: https://issues.apache.org/jira/browse/ATLAS-3477
 Project: Atlas
  Issue Type: Bug
Reporter: Sidharth Kumar Mishra


In case of soft delete at Atlas i.e. the entities marked as 'DELETED' there is 
no way to do cleanup if user wants to delete those permanently. Right now we 
can change the configuration 

atlas.DeleteHandlerV1.impl=org.apache.atlas.repository.store.graph.v1.

HardDeleteHandlerV1 and make the new delete of entity to be cleaned up as 
desired. But what happens to existing soft deleted Entities. There is no way in 
that case.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71619: ATLAS-3464: Define Entities stored in Atlas for ML Governance

[Na Li via Review Board]

> On Oct. 16, 2019, 7:36 p.m., Na Li wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 10 (patched)
> > 
> >
> > "DataSet" in Atlas has different meaning from the meaning in ML.  
> > 
> > DataSet: This type extends Referenceable and Asset. Conceptually, it 
> > can be used to represent an type that stores data. In Atlas, hive tables, 
> > Sqoop RDBMS tables etc are all types that extend from DataSet. Types that 
> > extend DataSet can be expected to have a Schema in the sense that they 
> > would have an attribute that defines attributes of that dataset. For e.g. 
> > the columns attribute in a hive_table. Also entities of types that extend 
> > DataSet participate in data transformation and this transformation can be 
> > captured by Atlas via lineage (or provenance) graphs. 
> > https://atlas.apache.org/0.8.1/TypeSystem.html
> 
> Anand Patil wrote:
> Hmm, it still feels like an odd fit to me, eg model builds and users 
> don't really have a schema. What would you think about instead making our 
> types inherit from Referenceable and Asset?

The type hierarchy is below

1) Asset derives from Referenceable
2) DataSet derives from Asset
3) Infrastructure derives from Asset
4) Process derives from Asset
5) ProcessExecution derives from Asset

It seems to me 
a) DataSet is used for storing data, 
b) Infrastructure is used as container, 
c) Process and ProcessExecution represent any data transformation operation or 
action

Deriving "model builds" or "users" from Asset directly violate this convention. 
I search the integration code from other modules, no type is derived from 
Referenceable or Asset directly.

{
  "name": "Referenceable",
  "superTypes": [],
  "serviceType": "atlas_core",
  "typeVersion": "1.0",
  "attributeDefs": [
{
  "name": "qualifiedName",
  "typeName": "string",
  "cardinality": "SINGLE",
  "isIndexable": true,
  "isOptional": false,
  "isUnique": true
}
  ]
},
{
  "name": "Asset",
  "superTypes": [
"Referenceable"
  ],
  "serviceType": "atlas_core",
  "typeVersion": "1.1",
  "attributeDefs": [
{
  "name": "name",
  "typeName": "string",
  "cardinality": "SINGLE",
  "isIndexable": true,
  "isOptional": false,
  "isUnique": false,
  "indexType": "STRING"
},
{
  "name": "DataSet",
  "superTypes": [
"Asset"
  ],
  "serviceType": "atlas_core",
  "typeVersion": "1.1",
  "attributeDefs": []
},
{
  "name": "Infrastructure",
  "description": "Infrastructure can be IT infrastructure, which contains 
hosts and servers. Infrastructure might not be IT orientated, such as 'Car' for 
IoT applications.",
  "superTypes": [
"Asset"
  ],
  "serviceType": "atlas_core",
  "typeVersion": "1.1",
  "attributeDefs": []
},
{
  "name": "Process",
  "superTypes": [
"Asset"
  ],
  "serviceType": "atlas_core",
  "typeVersion": "1.1",
  "attributeDefs": [
{
  "name": "inputs",
  "typeName": "array",
  "cardinality": "SET",
  "isIndexable": false,
  "isOptional": true,
  "isUnique": false
},
{
  "name": "outputs",
  "typeName": "array",
  "cardinality": "SET",
  "isIndexable": false,
  "isOptional": true,
  "isUnique": false
}
  ]
},
{
  "name": "ProcessExecution",
  "superTypes": [
"Asset"
  ],
  "serviceType": "atlas_core",
  "typeVersion": "1.0",
  "attributeDefs": []
}


> On Oct. 16, 2019, 7:36 p.m., Na Li wrote:
> > addons/models/1000-Hadoop/-ml_model.json
> > Lines 18 (patched)
> > 
> >
> > I don't want this to be optional. However, right now, some data source 
> > does not provide this info.
> 
> Anand Patil wrote:
> Could you give an example?

every Referenceable has an attribute "qualifiedName", which has to be unique. I 
am thinking we should get rid of this attribute "uniqueId", which duplicates 
the purpose of "qualifiedName". the attibute "name" can contain user-friendly 
name, and may not be unique.


- Na


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/#review218237
---


On Oct. 16, 2019, 12:30 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71619/
> ---
> 
> (Updated Oct. 16, 

[jira] [Commented] (ATLAS-3467) Regression : Tag filters (begins_with , ends_with , contains) and space fetches incorrect results

[Le Ma (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954001#comment-16954001
 ] 

Le Ma commented on ATLAS-3467:
--

RC : [https://reviews.apache.org/r/71625/]

> Regression : Tag filters (begins_with , ends_with , contains) and space 
> fetches incorrect results
> -
>
> Key: ATLAS-3467
> URL: https://issues.apache.org/jira/browse/ATLAS-3467
> Project: Atlas
>  Issue Type: Bug
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
>
> 1. Create a tag `tag1` with attribute `attr1`.
> 2. Associate `tag1` to an entity with value `value1` for `attr1`
> 3. Fire basic search for tag1 with filters:
> "attr1" contains "new comment"
>  
> Entity is returned , though "value1" is not related to "new comment" at all.
>  
> Happens with contains/begins_with/ends_with filters and not with =/!=
> Happens with filters with space (ex : new comment and not with newcomment or 
> new_comment)
> Happens only with tag filters and not with entity filters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3425) gremlin Script Execution Failed

[Le Ma (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953998#comment-16953998
 ] 

Le Ma commented on ATLAS-3425:
--

RC : [https://reviews.apache.org/r/71624/]

> gremlin Script Execution Failed
> ---
>
> Key: ATLAS-3425
> URL: https://issues.apache.org/jira/browse/ATLAS-3425
> Project: Atlas
>  Issue Type: Bug
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
> Attachments: error log, image.png
>
>
> In basic search, query parameters are:
>  * typeName: 
>  * classificationName
>  * tag attribute is boolean.
>  * fulltext query is tag attribute name
> No results is found. below error message is found in the backend log
> Please check the attached image for basic seach parameters. 
> javax.script.ScriptException: javax.script.ScriptException: 
> groovy.lang.MissingPropertyException: No such property: in for class: 
> org.apache.tinkerpop.gremlin.structure.T at 
> org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:378)
>  at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
>  at 
> org.apache.atlas.repository.graphdb.janus.AtlasJanusGraph.executeGremlinScript(AtlasJanusGraph.java:368)
>  at 
> org.apache.atlas.discovery.ClassificationSearchProcessor.filter(ClassificationSearchProcessor.java:339)
>  at 
> org.apache.atlas.discovery.SearchProcessor.filter(SearchProcessor.java:156)
>  at 
> org.apache.atlas.discovery.FreeTextSearchProcessor.execute(FreeTextSearchProcessor.java:156)
>  at 
> org.apache.atlas.discovery.EntityDiscoveryService.searchWithSearchContext(EntityDiscoveryService.java:471)
>  at 
> org.apache.atlas.discovery.EntityDiscoveryService.searchWithParameters(EntityDiscoveryService.java:458)
>  at 
> org.apache.atlas.discovery.EntityDiscoveryService$$FastClassBySpringCGLIB$$1af2cf1f.invoke()
>  at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
>  at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>  at 
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
>  at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
>  at 
> org.apache.atlas.discovery.EntityDiscoveryService$$EnhancerBySpringCGLIB$$95d844c8.searchWithParameters()
>  at 
> org.apache.atlas.web.rest.DiscoveryREST.searchWithParameters(DiscoveryREST.java:342)
>  at sun.reflect.GeneratedMethodAccessor181.invoke(Unknown Source)
>  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498)
>  at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>  at 
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
>  at 
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
>  at 
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
>  at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>  at 
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
>  at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>  at 
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
>  at 
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
>  at 
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
>  at 
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
>  at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>  at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1712)
>  at 

[jira] [Commented] (ATLAS-3457) Support Multi-Classification search without attributes

[Le Ma (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953999#comment-16953999
 ] 

Le Ma commented on ATLAS-3457:
--

RC : [https://reviews.apache.org/r/71612/]

> Support Multi-Classification search without attributes
> --
>
> Key: ATLAS-3457
> URL: https://issues.apache.org/jira/browse/ATLAS-3457
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Le Ma
>Assignee: Le Ma
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71623: ATLAS-3472 Port jsp's to Servlet

[Ashutosh Mestry via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71623/#review218270
---




webapp/src/main/java/org/apache/atlas/web/servlets/AtlasHttpServlet.java
Lines 50 (patched)


Refactor: Avoid 'if' clause.



webapp/src/main/webapp/WEB-INF/web.xml
Lines 40 (patched)


Relace AtlasHttpServlet with concrete class, say LoginServlet and 
ErrorServlet.


- Ashutosh Mestry


On Oct. 17, 2019, 10:51 a.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71623/
> ---
> 
> (Updated Oct. 17, 2019, 10:51 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3472
> https://issues.apache.org/jira/browse/ATLAS-3472
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch uses Servlets instead of JSP files to render login and error page.
> 
> 
> Diffs
> -
> 
>   webapp/src/main/java/org/apache/atlas/web/servlets/AtlasHttpServlet.java 
> PRE-CREATION 
>   webapp/src/main/resources/spring-security.xml 1295244 
>   webapp/src/main/webapp/WEB-INF/web.xml 23dc063 
>   webapp/src/main/webapp/error.html.template PRE-CREATION 
>   webapp/src/main/webapp/error.jsp 29ef640 
>   webapp/src/main/webapp/login.html.template PRE-CREATION 
>   webapp/src/main/webapp/login.jsp 70ec06d 
> 
> 
> Diff: https://reviews.apache.org/r/71623/diff/2/
> 
> 
> Testing
> ---
> 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1475/console
> Tested in browser with simple mode.
> Tested in browser with Knox proxy on kerberos mode.
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>

[jira] [Created] (ATLAS-3476) Regression : Case insensitivity in tag filters in search

[Le Ma (Jira)]

Le Ma created ATLAS-3476:


 Summary: Regression : Case insensitivity in tag filters in search
 Key: ATLAS-3476
 URL: https://issues.apache.org/jira/browse/ATLAS-3476
 Project: Atlas
  Issue Type: Bug
Reporter: Le Ma
Assignee: Le Ma


1. Create a tag `tag1` with attribute `attr1`.

2. Associate `tag1` to an entity with value `value1` fo `attr1`

3. Fire basic search for tag1 with filters attr1 = VALUE1

Entity is returned though there is case mismatch.

 

This doesn't happen with entity filters.

for example , for hive_table sample_08 , 

basic search :

type = hive_table,

filters , name = SAMPLE_08 doesn't fetch the entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71634: ATLAS-3475: Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71634/#review218269
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 17, 2019, 4:39 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71634/
> ---
> 
> (Updated Oct. 17, 2019, 4:39 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, 
> Madhan Neethiraj, and Nixon Rodrigues.
> 
> 
> Bugs: ATLAS-3475
> https://issues.apache.org/jira/browse/ATLAS-3475
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> The reason for error message in log is we attempt to create a process entity 
> for alter table properties operation. process entity for external tables 
> should only be created for CREATE_TABLE and CREATE_TABLE_AS_SELECT queries.
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
>  466167aa1 
> 
> 
> Diff: https://reviews.apache.org/r/71634/diff/1/
> 
> 
> Testing
> ---
> 
> Manually validated and no error messages are seen.
> 
> Precommit: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1478/console
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Review Request 71634: ATLAS-3475: Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71634/
---

Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, 
Madhan Neethiraj, and Nixon Rodrigues.


Bugs: ATLAS-3475
https://issues.apache.org/jira/browse/ATLAS-3475


Repository: atlas


Description
---

The reason for error message in log is we attempt to create a process entity 
for alter table properties operation. process entity for external tables should 
only be created for CREATE_TABLE and CREATE_TABLE_AS_SELECT queries.


Diffs
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
 466167aa1 


Diff: https://reviews.apache.org/r/71634/diff/1/


Testing
---

Manually validated and no error messages are seen.

Precommit: 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1478/console


Thanks,

Sarath Subramanian

[jira] [Updated] (ATLAS-3475) Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3475:
--
Description: 
While running CTAS query via spark.sql, Atlas is throwing below error
{code:java}
org.apache.atlas.exception.AtlasBaseException: Invalid instance 
creation/updation parameters passed : hive_process.qualifiedName: mandatory 
attribute value missing in type 
Referenceableorg.apache.atlas.exception.AtlasBaseException: Invalid instance 
creation/updation parameters passed : hive_process.qualifiedName: mandatory 
attribute value missing in type Referenceable at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:103)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:938)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:748)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:279)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke()
 at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
 at 
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
 at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$91e799f5.createOrUpdate()
 at 
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:802)
 at 
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:700)
 at 
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:521)
 at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:89) at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
at java.lang.Thread.run(Thread.java:748){code}

 The above error comes even in a newly created cluster, where there are (were) 
no shell entities present.

the error message is mostly harmless since the entities are getting created 
properly. Shell entities too are getting resolved correctly.

But in the metrics UI, each CTAS query shows up as a failure under ATLAS_HOOK-0 
failed messages 

Steps to reproduce:
 # spark-shell
 # spark.sql("create table t1(name string)")
 # spark.sql("create table t2 as select * from t1")

  was:
While running CTAS query via spark.sql, Atlas is throwing below error

{code:java}
org.apache.atlas.exception.AtlasBaseException: Invalid instance 
creation/updation parameters passed : hive_process.qualifiedName: mandatory 
attribute value missing in type 
Referenceableorg.apache.atlas.exception.AtlasBaseException: Invalid instance 
creation/updation parameters passed : hive_process.qualifiedName: mandatory 
attribute value missing in type Referenceable at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:103)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:938)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:748)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:279)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke()
 at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
 at 
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
 at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
 at 

[jira] [Updated] (ATLAS-3475) Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3475:
--
Attachment: ATLAS-3475.001.patch

> Invalid instance creation/updation parameters passed : 
> hive_process.qualifiedName: mandatory attribute value missing in type 
> Referenceable
> --
>
> Key: ATLAS-3475
> URL: https://issues.apache.org/jira/browse/ATLAS-3475
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: ATLAS-3475.001.patch
>
>
> While running CTAS query via spark.sql, Atlas is throwing below error
> {code:java}
> org.apache.atlas.exception.AtlasBaseException: Invalid instance 
> creation/updation parameters passed : hive_process.qualifiedName: mandatory 
> attribute value missing in type 
> Referenceableorg.apache.atlas.exception.AtlasBaseException: Invalid instance 
> creation/updation parameters passed : hive_process.qualifiedName: mandatory 
> attribute value missing in type Referenceable at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:103)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:938)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:748)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:279)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke()
>  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) 
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
>  at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>  at 
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
>  at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>  at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
>  at 
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$91e799f5.createOrUpdate()
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:802)
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:700)
>  at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:521)
>  at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:89) at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748) \{code}
> The above error comes even in a newly created cluster, where there are (were) 
> no shell entities present.
> the error message is mostly harmless since the entities are getting created 
> properly. Shell entities too are getting resolved correctly.
> But in the metrics UI, each CTAS query shows up as a failure under 
> ATLAS_HOOK-0 failed messages 
> Steps to reproduce:
>  # spark-shell
>  # spark.sql("create table t1(name string)")
>  # spark.sql("create table t2 as select * from t1")



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3475) Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable

[Sarath Subramanian (Jira)]

Sarath Subramanian created ATLAS-3475:
-

 Summary: Invalid instance creation/updation parameters passed : 
hive_process.qualifiedName: mandatory attribute value missing in type 
Referenceable
 Key: ATLAS-3475
 URL: https://issues.apache.org/jira/browse/ATLAS-3475
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 2.0.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.1.0


While running CTAS query via spark.sql, Atlas is throwing below error

{code:java}
org.apache.atlas.exception.AtlasBaseException: Invalid instance 
creation/updation parameters passed : hive_process.qualifiedName: mandatory 
attribute value missing in type 
Referenceableorg.apache.atlas.exception.AtlasBaseException: Invalid instance 
creation/updation parameters passed : hive_process.qualifiedName: mandatory 
attribute value missing in type Referenceable at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:103)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:938)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:748)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:279)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke()
 at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
 at 
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
 at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
 at 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$91e799f5.createOrUpdate()
 at 
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:802)
 at 
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:700)
 at 
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:521)
 at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:89) at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
at java.lang.Thread.run(Thread.java:748) \{code}
The above error comes even in a newly created cluster, where there are (were) 
no shell entities present.

the error message is mostly harmless since the entities are getting created 
properly. Shell entities too are getting resolved correctly.

But in the metrics UI, each CTAS query shows up as a failure under ATLAS_HOOK-0 
failed messages 

Steps to reproduce:
 # spark-shell
 # spark.sql("create table t1(name string)")
 # spark.sql("create table t2 as select * from t1")



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 71631: ATLAS-3427 - Atlas hooks enhancements for better fault-tolerance i.e. Kafka unavailability

[Nixon Rodrigues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71631/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3427
https://issues.apache.org/jira/browse/ATLAS-3427


Repository: atlas


Description
---

This patch includes changes to add file spooling mechanism to make atlas Hooks 
fault tolerant when Kafka destination is down.


Configuration required.

atlas.hook.spool.filecache.is.enabled = true [By Default false]

atlas.hook.spool.filecache.filespool.dir =  "/tmp"  [Directory where spool file 
 are stored, require proper write permissions for hook application]
atlas.hook.spool.filecache.filespool.archive.dir =  "/tmp/archive" [Directory 
where spool file  are archieved after syncing]
atlas.hook.spool.filecache.filespool.index.filename =  "indexFile" [Name of 
index file which has info of spool file and its status]
atlas.hook.spool.filecache.filespool.file.prefix = "atlas-hook"  [Prefix of 
index file]
atlas.hook.spool.filecache.filespool.file.rollover.sec  = 60 [ Roll over time 
for closing the spool file in sec]
atlas.hook.spool.filecache.filespool.destination.retry.ms [Retry time for 
Syncing the message from spool to kafka]  (Default 30 seconds)
atlas.hook.spool.filecache.filespool.archive.max.files = [Max files stored in 
archive directory]
atlas.hook.spool.filecache.filespool.app.type [application type such as hive, 
hbase etc]


Diffs
-

  notification/src/main/java/org/apache/atlas/hook/AtlasHook.java cc6546be8 
  notification/src/main/java/org/apache/atlas/hook/FailedMessagesLogger.java 
b319e81b8 
  
notification/src/main/java/org/apache/atlas/notification/NotificationException.java
 2dd9c9fa0 
  notification/src/main/java/org/apache/atlas/spool/AtlasFileCacheProvider.java 
PRE-CREATION 
  notification/src/main/java/org/apache/atlas/spool/AtlasFileSpool.java 
PRE-CREATION 
  
notification/src/main/java/org/apache/atlas/spool/AtlasKafkaMessageHandler.java 
PRE-CREATION 
  notification/src/main/java/org/apache/atlas/spool/MessageHandler.java 
PRE-CREATION 
  notification/src/main/java/org/apache/atlas/spool/MessageIndexRecord.java 
PRE-CREATION 
  notification/src/main/java/org/apache/atlas/spool/SpoolFileStatus.java 
PRE-CREATION 
  notification/src/test/java/org/apache/atlas/hook/AtlasHookTest.java 1ae7c278c 


Diff: https://reviews.apache.org/r/71631/diff/1/


Testing
---

Tested spooling on hive with atlas hive hook enabled in simple and kerberos env.

Existing testcase working fine.


Thanks,

Nixon Rodrigues

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated Oct. 17, 2019, 1:03 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

All the raised issues are been addresed in this patch.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description (updated)
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 357affd 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java f9d5fe4 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 9812356 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml ce7d6fb 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
be1698f 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/8/

Changes: https://reviews.apache.org/r/71482/diff/7-8/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71632: ATLAS-3471 : Regression : UI - Relationship attributes are not displayed in Audit Details Table

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71632/#review218263
---


Ship it!




Ship It!

- Sarath Subramanian


On Oct. 17, 2019, 5:23 a.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71632/
> ---
> 
> (Updated Oct. 17, 2019, 5:23 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3471
> https://issues.apache.org/jira/browse/ATLAS-3471
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Relationship attributes with legacyAttribute=True were displayed in the Audit 
> Details Table but it is not shown anymore. Example: columns, DB for an 
> hive_table entity.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/css/scss/common.scss 84afbc87a 
>   dashboardv2/public/js/templates/audit/CreateAuditTableLayoutView_tmpl.html 
> 88808d13f 
>   dashboardv2/public/js/views/audit/CreateAuditTableLayoutView.js a35eacbc1 
>   dashboardv3/public/css/scss/common.scss 00c67ea6e 
>   dashboardv3/public/css/scss/theme.scss 344e73f80 
>   dashboardv3/public/js/templates/audit/CreateAuditTableLayoutView_tmpl.html 
> 88808d13f 
>   dashboardv3/public/js/views/audit/CreateAuditTableLayoutView.js a35eacbc1 
> 
> 
> Diff: https://reviews.apache.org/r/71632/diff/1/
> 
> 
> Testing
> ---
> 
> Done one rounf of sanity test
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

[jira] [Commented] (ATLAS-3473) UI : Delay in loading Lineage icon because of sync img call

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953722#comment-16953722
 ] 

ASF subversion and git services commented on ATLAS-3473:


Commit 9ba321760e03148bcf59049888a42b69a3286541 in atlas's branch 
refs/heads/master from Keval Bhatt
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=9ba3217 ]

ATLAS-3473: UI : Delay in loading Lineage icon because of sync img call

Signed-off-by: Sarath Subramanian 


> UI : Delay in loading Lineage icon because of sync img call
> ---
>
> Key: ATLAS-3473
> URL: https://issues.apache.org/jira/browse/ATLAS-3473
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
>
> Lineage loads entity icon in synchronous mode and because of that rendering 
> is slow



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3473) UI : Delay in loading Lineage icon because of sync img call

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953723#comment-16953723
 ] 

ASF subversion and git services commented on ATLAS-3473:


Commit bf996ec99ac23566128b5b9cca2ce772c1a545d6 in atlas's branch 
refs/heads/branch-2.0 from Keval Bhatt
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=bf996ec ]

ATLAS-3473: UI : Delay in loading Lineage icon because of sync img call

Signed-off-by: Sarath Subramanian 
(cherry picked from commit 9ba321760e03148bcf59049888a42b69a3286541)


> UI : Delay in loading Lineage icon because of sync img call
> ---
>
> Key: ATLAS-3473
> URL: https://issues.apache.org/jira/browse/ATLAS-3473
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
>
> Lineage loads entity icon in synchronous mode and because of that rendering 
> is slow



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3471) Regression : UI - Relationship attributes are not displayed in Audit Details Table

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953720#comment-16953720
 ] 

ASF subversion and git services commented on ATLAS-3471:


Commit 8654188f233e5f3acef982c52b6221b5331f6370 in atlas's branch 
refs/heads/branch-2.0 from Keval Bhatt
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=8654188 ]

ATLAS-3471: Regression : UI - Relationship attributes are not displayed in 
Audit Details Table

Signed-off-by: Sarath Subramanian 
(cherry picked from commit e04acda4868a8db9b41f6b55d33b25c66bccebe3)


> Regression : UI - Relationship attributes are not displayed in Audit Details 
> Table
> --
>
> Key: ATLAS-3471
> URL: https://issues.apache.org/jira/browse/ATLAS-3471
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
> Attachments: ATLAS-3471.patch
>
>
> Relationship attributes with legacyAttribute=True were displayed in the Audit 
> Details Table but it is not shown anymore. Example: columns, DB for an 
> hive_table entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3471) Regression : UI - Relationship attributes are not displayed in Audit Details Table

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953717#comment-16953717
 ] 

ASF subversion and git services commented on ATLAS-3471:


Commit e04acda4868a8db9b41f6b55d33b25c66bccebe3 in atlas's branch 
refs/heads/master from Keval Bhatt
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=e04acda ]

ATLAS-3471: Regression : UI - Relationship attributes are not displayed in 
Audit Details Table

Signed-off-by: Sarath Subramanian 


> Regression : UI - Relationship attributes are not displayed in Audit Details 
> Table
> --
>
> Key: ATLAS-3471
> URL: https://issues.apache.org/jira/browse/ATLAS-3471
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
> Attachments: ATLAS-3471.patch
>
>
> Relationship attributes with legacyAttribute=True were displayed in the Audit 
> Details Table but it is not shown anymore. Example: columns, DB for an 
> hive_table entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71628: ATLAS-3473 : UI : Delay in loading Lineage icon because of sync img call

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71628/#review218264
---


Ship it!




Ship It!

- Sarath Subramanian


On Oct. 17, 2019, 2:10 a.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71628/
> ---
> 
> (Updated Oct. 17, 2019, 2:10 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3473
> https://issues.apache.org/jira/browse/ATLAS-3473
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Lineage loads entity icon in synchronous mode and because of that rendering 
> is slow
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/views/graph/LineageLayoutView.js d48125175 
>   dashboardv3/public/js/views/graph/LineageLayoutView.js a15047c92 
> 
> 
> Diff: https://reviews.apache.org/r/71628/diff/1/
> 
> 
> Testing
> ---
> 
> Done one round of Sanity test
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Review Request 71632: ATLAS-3471 : Regression : UI - Relationship attributes are not displayed in Audit Details Table

[keval bhatt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71632/
---

Review request for atlas, Ashutosh Mestry, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3471
https://issues.apache.org/jira/browse/ATLAS-3471


Repository: atlas


Description
---

Relationship attributes with legacyAttribute=True were displayed in the Audit 
Details Table but it is not shown anymore. Example: columns, DB for an 
hive_table entity.


Diffs
-

  dashboardv2/public/css/scss/common.scss 84afbc87a 
  dashboardv2/public/js/templates/audit/CreateAuditTableLayoutView_tmpl.html 
88808d13f 
  dashboardv2/public/js/views/audit/CreateAuditTableLayoutView.js a35eacbc1 
  dashboardv3/public/css/scss/common.scss 00c67ea6e 
  dashboardv3/public/css/scss/theme.scss 344e73f80 
  dashboardv3/public/js/templates/audit/CreateAuditTableLayoutView_tmpl.html 
88808d13f 
  dashboardv3/public/js/views/audit/CreateAuditTableLayoutView.js a35eacbc1 


Diff: https://reviews.apache.org/r/71632/diff/1/


Testing
---

Done one rounf of sanity test


Thanks,

keval bhatt

[jira] [Updated] (ATLAS-3471) Regression : UI - Relationship attributes are not displayed in Audit Details Table

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3471:
---
Attachment: ATLAS-3471.patch

> Regression : UI - Relationship attributes are not displayed in Audit Details 
> Table
> --
>
> Key: ATLAS-3471
> URL: https://issues.apache.org/jira/browse/ATLAS-3471
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
> Attachments: ATLAS-3471.patch
>
>
> Relationship attributes with legacyAttribute=True were displayed in the Audit 
> Details Table but it is not shown anymore. Example: columns, DB for an 
> hive_table entity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading

[Jiaqi Shan (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jiaqi Shan updated ATLAS-3474:
--
Attachment: (was: ATLAS-3474.patch)

> Atlas throws IOException when call api in multithreading
> 
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3474.patch
>
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the 
> requests timeout frequently. I found the following exception in 
> atlas.20191017-104155.out: 
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~ 
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632) 
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
> expired: 3/3 ms at 
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
>  at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at 
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
>  at 
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
>  at 
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
>  at 
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) 
> at 
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) 
> at 
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
>  at 
> com.sun.jersey.server.impl.applicatio

[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading

[Jiaqi Shan (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jiaqi Shan updated ATLAS-3474:
--
Attachment: ATLAS-3474.patch

> Atlas throws IOException when call api in multithreading
> 
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3474.patch
>
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the 
> requests timeout frequently. I found the following exception in 
> atlas.20191017-104155.out: 
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~ 
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632) 
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
> expired: 3/3 ms at 
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
>  at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at 
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
>  at 
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
>  at 
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
>  at 
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) 
> at 
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) 
> at 
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
>  a

Review Request 71628: ATLAS-3473 : UI : Delay in loading Lineage icon because of sync img call

[keval bhatt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71628/
---

Review request for atlas, Ashutosh Mestry, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3473
https://issues.apache.org/jira/browse/ATLAS-3473


Repository: atlas


Description
---

Lineage loads entity icon in synchronous mode and because of that rendering is 
slow


Diffs
-

  dashboardv2/public/js/views/graph/LineageLayoutView.js d48125175 
  dashboardv3/public/js/views/graph/LineageLayoutView.js a15047c92 


Diff: https://reviews.apache.org/r/71628/diff/1/


Testing
---

Done one round of Sanity test


Thanks,

keval bhatt

[jira] [Updated] (ATLAS-3473) UI : Delay in loading Lineage icon because of sync img call

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3473:
---
Description: Lineage loads entity icon in synchronous mode and because of 
that rendering is slow  (was: Lineage loads entity icons in synchronous mode 
and because of that rendering is slow)

> UI : Delay in loading Lineage icon because of sync img call
> ---
>
> Key: ATLAS-3473
> URL: https://issues.apache.org/jira/browse/ATLAS-3473
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
>
> Lineage loads entity icon in synchronous mode and because of that rendering 
> is slow



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3473) UI : Delay in loading Lineage icon because of sync img call

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3473:
---
Description: Lineage loads entity icons in synchronous mode and because of 
that rendering is slow  (was: Lineage Icon load in synchronous mode and because 
of that Lineage render icon after few second.)

> UI : Delay in loading Lineage icon because of sync img call
> ---
>
> Key: ATLAS-3473
> URL: https://issues.apache.org/jira/browse/ATLAS-3473
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
>
> Lineage loads entity icons in synchronous mode and because of that rendering 
> is slow



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3473) UI : Delay in loading Lineage icon because of sync img call

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3473:
---
Description: Lineage Icon load in synchronous mode and because of that 
Lineage render icon after few second.

> UI : Delay in loading Lineage icon because of sync img call
> ---
>
> Key: ATLAS-3473
> URL: https://issues.apache.org/jira/browse/ATLAS-3473
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
>
> Lineage Icon load in synchronous mode and because of that Lineage render icon 
> after few second.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading

[Jiaqi Shan (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jiaqi Shan updated ATLAS-3474:
--
Attachment: ATLAS-3474.patch

> Atlas throws IOException when call api in multithreading
> 
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3474.patch
>
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the 
> requests timeout frequently. I found the following exception in 
> atlas.20191017-104155.out: 
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~ 
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632) 
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
> expired: 3/3 ms at 
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
>  at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at 
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
>  at 
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
>  at 
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
>  at 
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) 
> at 
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) 
> at 
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
>  a

[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading

[Jiaqi Shan (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jiaqi Shan updated ATLAS-3474:
--
Attachment: (was: ATLAS-3474.patch)

> Atlas throws IOException when call api in multithreading
> 
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the 
> requests timeout frequently. I found the following exception in 
> atlas.20191017-104155.out: 
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~ 
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632) 
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
> expired: 3/3 ms at 
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
>  at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at 
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
>  at 
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
>  at 
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
>  at 
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) 
> at 
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) 
> at 
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
>  at 
> com.sun.jersey.server.im

[jira] [Updated] (ATLAS-3474) Atlas throws IOException when call api in multithreading

[Jiaqi Shan (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jiaqi Shan updated ATLAS-3474:
--
Attachment: ATLAS-3474.patch

> Atlas throws IOException when call api in multithreading
> 
>
> Key: ATLAS-3474
> URL: https://issues.apache.org/jira/browse/ATLAS-3474
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3474.patch
>
>
> When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the 
> requests timeout frequently. I found the following exception in 
> atlas.20191017-104155.out: 
> {code:java}
> 2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~ 
> /api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632) 
> java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
> expired: 3/3 ms at 
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
>  at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at 
> org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
>  at 
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
>  at 
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
>  at 
> com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
>  at 
> com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
>  at 
> com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
>  at 
> com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
>  at 
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>  at 
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
>  at 
> com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
>  at 
> com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) 
> at 
> com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) 
> at 
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
>  at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
>  a

[jira] [Created] (ATLAS-3474) Atlas throws IOException when call api in multithreading

[Jiaqi Shan (Jira)]

Jiaqi Shan created ATLAS-3474:
-

 Summary: Atlas throws IOException when call api in multithreading
 Key: ATLAS-3474
 URL: https://issues.apache.org/jira/browse/ATLAS-3474
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0
Reporter: Jiaqi Shan


When call /api/atlas/v2/entity/uniqueAttribute/type in multithreading, the 
requests timeout frequently. I found the following exception in 
atlas.20191017-104155.out: 
{code:java}
2019-10-17 10:43:44,763 WARN - [pool-1-thread-9:] ~ 
/api/atlas/v2/entity/uniqueAttribute/type/hive_table (ServletHandler:632) 
java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout 
expired: 3/3 ms at 
org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
 at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) at 
org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:470) at 
org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write(OnCommittedResponseWrapper.java:639)
 at 
com.sun.jersey.spi.container.servlet.WebComponent$Writer.write(WebComponent.java:300)
 at 
com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
 at 
com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085)
 at 
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment2(UTF8JsonGenerator.java:1400)
 at 
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegment(UTF8JsonGenerator.java:1347)
 at 
com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeStringSegments(UTF8JsonGenerator.java:1228)
 at 
com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:453)
 at 
com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
 at 
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
 at 
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
 at 
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
 at 
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
 at 
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
 at 
com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
 at 
com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
 at 
com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
 at 
com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
 at 
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
 at 
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
 at 
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
 at 
com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
 at 
com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
 at 
com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
 at 
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
 at 
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
 at 
com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
 at 
com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) 
at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) 
at 
com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
 at 
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1510)
 at 
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
 at 
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
 at 
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
 at 
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
 at 
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at 
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845) at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1712)
 at org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:80

[GitHub] [atlas] andrewluotechnologies commented on a change in pull request #61: ATLAS-3295: Remove unused setup code and documentation

[GitBox]

andrewluotechnologies commented on a change in pull request #61: ATLAS-3295: 
Remove unused setup code and documentation
URL: https://github.com/apache/atlas/pull/61#discussion_r335847879
 
 

 ##
 File path: distro/src/bin/atlas_start.py
 ##
 @@ -143,13 +135,10 @@ def main():
 web_app_path = os.path.join(web_app_dir, "atlas")
 if (mc.isCygwin()):
 web_app_path = mc.convertCygwinPath(web_app_path)
-if not is_setup:
-start_atlas_server(atlas_classpath, atlas_pid_file, jvm_logdir, 
jvm_opts_list, web_app_path)
-mc.wait_for_startup(confdir, 300)
-print "Apache Atlas Server started!!!\n"
-else:
-process = mc.java("org.apache.atlas.web.setup.AtlasSetup", [], 
atlas_classpath, jvm_opts_list, jvm_logdir)
 
 Review comment:
   I don't believe there is a AtlasSetupModule anymore: 
https://github.com/apache/atlas/tree/master/webapp/src/main/java/org/apache/atlas/web/setup
   
   The purpose of this PR is to entirely remove the -setup option as it doesn't 
seem to be needed anymore.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [atlas] andrewluotechnologies closed pull request #62: ATLAS-3296: Correct elasticsearch hostname property in atlas-application.properties

[GitBox]

andrewluotechnologies closed pull request #62: ATLAS-3296: Correct 
elasticsearch hostname property in atlas-application.properties
URL: https://github.com/apache/atlas/pull/62
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [atlas] andrewluotechnologies commented on issue #62: ATLAS-3296: Correct elasticsearch hostname property in atlas-application.properties

[GitBox]

andrewluotechnologies commented on issue #62: ATLAS-3296: Correct elasticsearch 
hostname property in atlas-application.properties
URL: https://github.com/apache/atlas/pull/62#issuecomment-543045593
 
 
   This has been committed in a876e697c3c0c936e7efc33cecbe1e1c32a6be20


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (ATLAS-3447) Basic Search API does not return child classifications

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3447:
---
Fix Version/s: 3.0.0

> Basic Search API does not return child classifications
> --
>
> Key: ATLAS-3447
> URL: https://issues.apache.org/jira/browse/ATLAS-3447
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 
> child_classification_does_not_appear_after_parent_classification_is_reapplied.webm,
>  
> childclassification_vanishes_after_parent_classification_association_is_empty.webm
>
>
> Detailed explanation of the issue along with an additional related issue(with 
> videos attached).
> Scenario I:  
> [^childclassification_vanishes_after_parent_classification_association_is_empty.webm]
> Entity E1 is associated with classification 'Parent' and entity E2 is 
> associated with classification 'Child' where 'Child' is subclassification of 
> 'Parent'.
>  * In the 'Parent' classification details page, E1 and E2 are visible.
>  * 'Parent' classification is removed from E1.
>  * When we view Parent classification details, E1 no longer appears, but even 
> E2 is removed, which is incorrect as the 'Child' is a subclassification of 
> 'Parent' and should be visible on the page as the option "Exclude 
> subclassifications" is not selected.
>  
> Scenario II: 
> [^child_classification_does_not_appear_after_parent_classification_is_reapplied.webm]
> After performing steps from scenario I, 
>  *  Reassign 'Parent' classification to E1
>  * Navigate to 'Parent' classification details page.
>  * E1 is still the only entity visible, even E2 should also be visible as per 
> step 1 in Scenario I.
>  * Now, remove 'Child' classification from E2 and then reapply 'Child' to E2.
>  * Navigate again to 'Parent' classification details page.
>  * You will see that now E1 and E2 appear as expected. This seems to be a 
> result of a stale connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3447) Basic Search API does not return child classifications

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3447:
---
Fix Version/s: (was: 3.0.0)

> Basic Search API does not return child classifications
> --
>
> Key: ATLAS-3447
> URL: https://issues.apache.org/jira/browse/ATLAS-3447
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Priority: Major
> Attachments: 
> child_classification_does_not_appear_after_parent_classification_is_reapplied.webm,
>  
> childclassification_vanishes_after_parent_classification_association_is_empty.webm
>
>
> Detailed explanation of the issue along with an additional related issue(with 
> videos attached).
> Scenario I:  
> [^childclassification_vanishes_after_parent_classification_association_is_empty.webm]
> Entity E1 is associated with classification 'Parent' and entity E2 is 
> associated with classification 'Child' where 'Child' is subclassification of 
> 'Parent'.
>  * In the 'Parent' classification details page, E1 and E2 are visible.
>  * 'Parent' classification is removed from E1.
>  * When we view Parent classification details, E1 no longer appears, but even 
> E2 is removed, which is incorrect as the 'Child' is a subclassification of 
> 'Parent' and should be visible on the page as the option "Exclude 
> subclassifications" is not selected.
>  
> Scenario II: 
> [^child_classification_does_not_appear_after_parent_classification_is_reapplied.webm]
> After performing steps from scenario I, 
>  *  Reassign 'Parent' classification to E1
>  * Navigate to 'Parent' classification details page.
>  * E1 is still the only entity visible, even E2 should also be visible as per 
> step 1 in Scenario I.
>  * Now, remove 'Child' classification from E2 and then reapply 'Child' to E2.
>  * Navigate again to 'Parent' classification details page.
>  * You will see that now E1 and E2 appear as expected. This seems to be a 
> result of a stale connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3447) Basic Search API does not return child classifications

[Rahul Kurup (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rahul Kurup updated ATLAS-3447:
---
Description: 
Detailed explanation of the issue along with an additional related issue(with 
videos attached).



Scenario I:  
[^childclassification_vanishes_after_parent_classification_association_is_empty.webm]

Entity E1 is associated with classification 'Parent' and entity E2 is 
associated with classification 'Child' where 'Child' is subclassification of 
'Parent'.
 * In the 'Parent' classification details page, E1 and E2 are visible.
 * 'Parent' classification is removed from E1.
 * When we view Parent classification details, E1 no longer appears, but even 
E2 is removed, which is incorrect as the 'Child' is a subclassification of 
'Parent' and should be visible on the page as the option "Exclude 
subclassifications" is not selected.

 

Scenario II: 
[^child_classification_does_not_appear_after_parent_classification_is_reapplied.webm]

After performing steps from scenario I, 
 *  Reassign 'Parent' classification to E1
 * Navigate to 'Parent' classification details page.
 * E1 is still the only entity visible, even E2 should also be visible as per 
step 1 in Scenario I.
 * Now, remove 'Child' classification from E2 and then reapply 'Child' to E2.
 * Navigate again to 'Parent' classification details page.
 * You will see that now E1 and E2 appear as expected. This seems to be a 
result of a stale connection.

  was:
Steps to reproduce.
 # Run QuickStart.py
 # Create sub-classification
 # Apply sub-classification to an entity which does not have parent 
classification.
 # Go to the classification tab and click on the parent classification OR go to 
the basic search tab and search for the parent classification.

Expected O/P
 * It should display the entity upon which the child classification has been 
applied.


*NOTE*: If the entity that has been associated with the child classification 
already has an association with the parent classification, then the outcome is 
as expected.


> Basic Search API does not return child classifications
> --
>
> Key: ATLAS-3447
> URL: https://issues.apache.org/jira/browse/ATLAS-3447
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Priority: Major
> Attachments: 
> child_classification_does_not_appear_after_parent_classification_is_reapplied.webm,
>  
> childclassification_vanishes_after_parent_classification_association_is_empty.webm
>
>
> Detailed explanation of the issue along with an additional related issue(with 
> videos attached).
> Scenario I:  
> [^childclassification_vanishes_after_parent_classification_association_is_empty.webm]
> Entity E1 is associated with classification 'Parent' and entity E2 is 
> associated with classification 'Child' where 'Child' is subclassification of 
> 'Parent'.
>  * In the 'Parent' classification details page, E1 and E2 are visible.
>  * 'Parent' classification is removed from E1.
>  * When we view Parent classification details, E1 no longer appears, but even 
> E2 is removed, which is incorrect as the 'Child' is a subclassification of 
> 'Parent' and should be visible on the page as the option "Exclude 
> subclassifications" is not selected.
>  
> Scenario II: 
> [^child_classification_does_not_appear_after_parent_classification_is_reapplied.webm]
> After performing steps from scenario I, 
>  *  Reassign 'Parent' classification to E1
>  * Navigate to 'Parent' classification details page.
>  * E1 is still the only entity visible, even E2 should also be visible as per 
> step 1 in Scenario I.
>  * Now, remove 'Child' classification from E2 and then reapply 'Child' to E2.
>  * Navigate again to 'Parent' classification details page.
>  * You will see that now E1 and E2 appear as expected. This seems to be a 
> result of a stale connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3447) Basic Search API does not return child classifications

[Rahul Kurup (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rahul Kurup updated ATLAS-3447:
---
Attachment: 
childclassification_vanishes_after_parent_classification_association_is_empty.webm

child_classification_does_not_appear_after_parent_classification_is_reapplied.webm

> Basic Search API does not return child classifications
> --
>
> Key: ATLAS-3447
> URL: https://issues.apache.org/jira/browse/ATLAS-3447
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Priority: Major
> Attachments: 
> child_classification_does_not_appear_after_parent_classification_is_reapplied.webm,
>  
> childclassification_vanishes_after_parent_classification_association_is_empty.webm
>
>
> Steps to reproduce.
>  # Run QuickStart.py
>  # Create sub-classification
>  # Apply sub-classification to an entity which does not have parent 
> classification.
>  # Go to the classification tab and click on the parent classification OR go 
> to the basic search tab and search for the parent classification.
> Expected O/P
>  * It should display the entity upon which the child classification has been 
> applied.
> *NOTE*: If the entity that has been associated with the child classification 
> already has an association with the parent classification, then the outcome 
> is as expected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)