[jira] [Created] (ATLAS-1910) Classpath issue in import-hive.sh script
Benjamin BONNET created ATLAS-1910:
--
Summary: Classpath issue in import-hive.sh script
Key: ATLAS-1910
URL: https://issues.apache.org/jira/browse/ATLAS-1910
Project: Atlas
Issue Type: Bug
Affects Versions: 0.8-incubating, trunk, 0.9-incubating
Environment: HDP2.6 sandbox
Reporter: Benjamin BONNET
Hi,
trying to run Hive bridge, I get the following error :
{{[root@sandbox hook-bin]# ./import-hive.sh
/usr/hdp/2.6.0.3-8/hadoop/conf:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop/.//*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/./:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/.//*:/usr/hdp/2.6.0.3-8/hadoop-yarn/lib/*:/usr/hdp/2.6.0.3-8/hadoop-yarn/.//*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/.//*::jdbc-mysql.jar:mysql-connector-java-5.1.17.jar:mysql-connector-java-5.1.37.jar:mysql-connector-java.jar:/usr/hdp/2.6.0.3-8/tez/*:/usr/hdp/2.6.0.3-8/tez/lib/*:/usr/hdp/2.6.0.3-8/tez/conf
Using Hive configuration directory [/etc/hive/conf]
Log file for import is /usr/hdp/current/atlas-client/logs/import-hive.log
CP=:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-client-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-common-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-notification-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-typesystem-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/gson-2.5.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/guice-multibindings-4.1.0.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hdfs-model-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hive-bridge-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-ast_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-core_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-native_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka_2.11-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka-clients-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-compiler-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-library-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scalap-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-reflect-2.11.8.jar:/etc/hive/conf:/usr/hdp/current/atlas-client/../hive/lib/accumulo-core-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-fate-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-start-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-trace-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/activation-1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-launcher-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-2.7.7.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-runtime-3.4.jar:/usr/hdp/current/atlas-client/../hive/lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-commons-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-tree-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-metrics-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avro-1.7.5.jar:/usr/hdp/current/atlas-client/../hive/lib/bonecp-0.8.0.RELEASE.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-core-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-linq4j-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-cli-1.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-codec-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-collections-3.2.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compiler-2.7.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compress-1.4.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-dbcp-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-httpclient-3.0.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-io-2.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-lang-2.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-logging-1.1.3.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-math-2.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-pool-1.5.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-vfs2-2.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-client-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-framework
[jira] [Updated] (ATLAS-1910) Classpath issue in import-hive.sh script
[
https://issues.apache.org/jira/browse/ATLAS-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin BONNET updated ATLAS-1910:
---
Description:
Hi,
trying to run Hive bridge, I get the following error :
{{
[root@sandbox hook-bin]# ./import-hive.sh
/usr/hdp/2.6.0.3-8/hadoop/conf:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop/.//*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/./:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/.//*:/usr/hdp/2.6.0.3-8/hadoop-yarn/lib/*:/usr/hdp/2.6.0.3-8/hadoop-yarn/.//*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/.//*::jdbc-mysql.jar:mysql-connector-java-5.1.17.jar:mysql-connector-java-5.1.37.jar:mysql-connector-java.jar:/usr/hdp/2.6.0.3-8/tez/*:/usr/hdp/2.6.0.3-8/tez/lib/*:/usr/hdp/2.6.0.3-8/tez/conf
Using Hive configuration directory [/etc/hive/conf]
Log file for import is /usr/hdp/current/atlas-client/logs/import-hive.log
CP=:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-client-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-common-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-notification-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-typesystem-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/gson-2.5.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/guice-multibindings-4.1.0.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hdfs-model-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hive-bridge-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-ast_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-core_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-native_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka_2.11-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka-clients-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-compiler-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-library-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scalap-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-reflect-2.11.8.jar:/etc/hive/conf:/usr/hdp/current/atlas-client/../hive/lib/accumulo-core-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-fate-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-start-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-trace-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/activation-1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-launcher-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-2.7.7.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-runtime-3.4.jar:/usr/hdp/current/atlas-client/../hive/lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-commons-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-tree-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-metrics-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avro-1.7.5.jar:/usr/hdp/current/atlas-client/../hive/lib/bonecp-0.8.0.RELEASE.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-core-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-linq4j-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-cli-1.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-codec-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-collections-3.2.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compiler-2.7.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compress-1.4.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-dbcp-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-httpclient-3.0.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-io-2.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-lang-2.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-logging-1.1.3.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-math-2.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-pool-1.5.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-vfs2-2.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-client-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-framework-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-recipes-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/datanucleus-api-jdo-4.2.1.jar:/usr/hdp/current/atlas-client/../hive/lib/datanucleus-core-4.1.6
[jira] [Commented] (ATLAS-1910) Classpath issue in import-hive.sh script
[
https://issues.apache.org/jira/browse/ATLAS-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072320#comment-16072320
]
Benjamin BONNET commented on ATLAS-1910:
Inverting the order of HADOOP_CLASSPATH and HIVE_CLASSPATH computation does
the trick.
Please see : https://github.com/apache/incubator-atlas/pull/37
> Classpath issue in import-hive.sh script
>
>
> Key: ATLAS-1910
> URL: https://issues.apache.org/jira/browse/ATLAS-1910
> Project: Atlas
> Issue Type: Bug
>Affects Versions: trunk, 0.8-incubating, 0.9-incubating
> Environment: HDP2.6 sandbox
>Reporter: Benjamin BONNET
>
> Hi,
> trying to run Hive bridge, I get the following error :
> {{
> [root@sandbox hook-bin]# ./import-hive.sh
> /usr/hdp/2.6.0.3-8/hadoop/conf:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop/.//*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/./:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/.//*:/usr/hdp/2.6.0.3-8/hadoop-yarn/lib/*:/usr/hdp/2.6.0.3-8/hadoop-yarn/.//*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/.//*::jdbc-mysql.jar:mysql-connector-java-5.1.17.jar:mysql-connector-java-5.1.37.jar:mysql-connector-java.jar:/usr/hdp/2.6.0.3-8/tez/*:/usr/hdp/2.6.0.3-8/tez/lib/*:/usr/hdp/2.6.0.3-8/tez/conf
> Using Hive configuration directory [/etc/hive/conf]
> Log file for import is /usr/hdp/current/atlas-client/logs/import-hive.log
> CP=:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-client-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-common-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-notification-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-typesystem-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/gson-2.5.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/guice-multibindings-4.1.0.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hdfs-model-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hive-bridge-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-ast_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-core_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-native_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka_2.11-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka-clients-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-compiler-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-library-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scalap-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-reflect-2.11.8.jar:/etc/hive/conf:/usr/hdp/current/atlas-client/../hive/lib/accumulo-core-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-fate-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-start-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-trace-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/activation-1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-launcher-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-2.7.7.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-runtime-3.4.jar:/usr/hdp/current/atlas-client/../hive/lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-commons-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-tree-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-metrics-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avro-1.7.5.jar:/usr/hdp/current/atlas-client/../hive/lib/bonecp-0.8.0.RELEASE.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-core-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-linq4j-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-cli-1.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-codec-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-collections-3.2.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compiler-2.7.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compress-1.4.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-dbcp-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-httpclient-3.0.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-io-2.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-lang-2.6.jar:/u
[jira] [Created] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
Benjamin BONNET created ATLAS-1990:
--
Summary: Regression :Traits Attribute that References Another
Entity
Key: ATLAS-1990
URL: https://issues.apache.org/jira/browse/ATLAS-1990
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.8-incubating
Environment: HDP2.6
Reporter: Benjamin BONNET
Hi,
I defined a trait that includes one attribute. That attribute has type
"hive_column", i.e. it references a hive_column entity.
Using Altas 0.5 on a HDP 2.4, I could attach that trait to an entity so that it
refers a column. But using Atlas 0.8 on a HDP 2.6 that does not work anymore (I
get a NullPointerException).
I tried both V1 and V2 REST API and got the same error.
Here is a full sample to reproduce that issue.
Trait definition :
{noformat}
{"enumTypes":[],
"structTypes":[],
"traitTypes":[{
"superTypes":[],
"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType",
"typeName":"Reference",
"typeDescription":"reference test",
"attributeDefinitions":[{"name":"ref",
"dataTypeName":"hive_column",
"multiplicity":"required",
"isComposite":false,
"isUnique":false,
"isIndexable":true,
"reverseAttributeName":null
}
]}],
"classTypes":[]}
{noformat}
Then I use curl to create that type (ref file contains the traits definition):
{noformat}
curl -v -u admin:admin -X POST -d "`cat ref`" -H 'Content-Type:
application/json' 'http://127.0.0.1:21000/api/atlas/types'
{noformat}
Using that command, trait type is created OK on both 0.5 and 0.8.
Now, I can tag an entity on Atlas 0.5 :
{noformat}
curl -v -u admin:admin -X POST -d
'{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
"typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"9e2b0059-8e75-42ce-a32c-fe9a6157caed","version":0,"typeName":"hive_column"}}}'
-H "Content-Type: application/json"
"http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits";
{noformat}
I get a HTTP201 :
< HTTP/1.1 201 Created
< Date: Tue, 25 Jul 2017 09:11:23 GMT
< Location:
http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits/23d1c3e7-7ad9-47b6-ac54-1d507119d00d
< Content-Type: application/json; charset=UTF-8
< Transfer-Encoding: chunked
< Server: Jetty(9.2.12.v20150709)
<
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
{"requestId":"qtp999445351-15 -
6d4b21dd-af88-46f3-91f4-5b3b20b6840c","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d"}
And if I check my entity traits, I can see that tag.
{noformat}
curl -v -u admin:admin -X GET
"http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits";
* About to connect() to 127.0.0.1 port 21000 (#0)
* Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 21000 (#0)
* Server auth using Basic with user 'admin'
> GET /api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits HTTP/1.1
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1
> Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: 127.0.0.1:21000
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 25 Jul 2017 09:13:27 GMT
< Content-Type: application/json; charset=UTF-8
< Transfer-Encoding: chunked
< Server: Jetty(9.2.12.v20150709)
<
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
{"requestId":"qtp999445351-13 -
15a0ce26-484a-4181-9f5a-742227be5d09","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d","results":["tag1","tag2","Reference","tag3"],"count":4}
{noformat}
So far, so good, but If I try to do the same thing Atlas 0.8, i.e. tagging an
entity with that trait :
{noformat}
curl -v -u admin:admin -X POST -d
'{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
"typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"d5a507e6-0752-4922-a524-cac6e8606f7d","version":0,"typeName":"hive_column"}}}'
-H "Content-Type: application/json"
"http://127.0.0.1:21000/api/atlas/entities/f79fa725-f628-4672-8b7e-1568ebdf764a/traits";
{noformat}
I get a HTTP500
{noformat}
< HTTP/1.1 500 Internal Server Error
< Date: Mon, 24 Jul 2017 17:28:15 GMT
< Set-Cookie: ATLASSESSIONID=hnk1soc7nw8v7e4me880rct0;Path=/;HttpOnly
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-Frame-Options: DENY
< Content-Type: application/json; charset=UTF-8
< Transfer-Encoding: chunked
< Server: Jetty(9.2.12.v20150709)
<
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
{"error":"Failed with java.lang.NullPointerException"}
{noforma
[jira] [Updated] (ATLAS-1910) Classpath issue in import-hive.sh script
[
https://issues.apache.org/jira/browse/ATLAS-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin BONNET updated ATLAS-1910:
---
Description:
Hi,
trying to run Hive bridge, I get the following error :
{noformat}
[root@sandbox hook-bin]# ./import-hive.sh
/usr/hdp/2.6.0.3-8/hadoop/conf:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop/.//*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/./:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/.//*:/usr/hdp/2.6.0.3-8/hadoop-yarn/lib/*:/usr/hdp/2.6.0.3-8/hadoop-yarn/.//*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/.//*::jdbc-mysql.jar:mysql-connector-java-5.1.17.jar:mysql-connector-java-5.1.37.jar:mysql-connector-java.jar:/usr/hdp/2.6.0.3-8/tez/*:/usr/hdp/2.6.0.3-8/tez/lib/*:/usr/hdp/2.6.0.3-8/tez/conf
Using Hive configuration directory [/etc/hive/conf]
Log file for import is /usr/hdp/current/atlas-client/logs/import-hive.log
CP=:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-client-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-common-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-notification-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-typesystem-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/gson-2.5.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/guice-multibindings-4.1.0.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hdfs-model-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hive-bridge-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-ast_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-core_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-native_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka_2.11-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka-clients-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-compiler-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-library-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scalap-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-reflect-2.11.8.jar:/etc/hive/conf:/usr/hdp/current/atlas-client/../hive/lib/accumulo-core-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-fate-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-start-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-trace-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/activation-1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-launcher-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-2.7.7.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-runtime-3.4.jar:/usr/hdp/current/atlas-client/../hive/lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-commons-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-tree-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-metrics-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avro-1.7.5.jar:/usr/hdp/current/atlas-client/../hive/lib/bonecp-0.8.0.RELEASE.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-core-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-linq4j-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-cli-1.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-codec-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-collections-3.2.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compiler-2.7.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compress-1.4.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-dbcp-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-httpclient-3.0.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-io-2.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-lang-2.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-logging-1.1.3.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-math-2.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-pool-1.5.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-vfs2-2.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-client-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-framework-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-recipes-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/datanucleus-api-jdo-4.2.1.jar:/usr/hdp/current/atlas-client/../hive/lib/datanucleus-co
[jira] [Commented] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
[
https://issues.apache.org/jira/browse/ATLAS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099937#comment-16099937
]
Benjamin BONNET commented on ATLAS-1990:
Hi David,
thank you for your answer.
This is an example of use case I am trying to implement : I have tables A and B
that have a one to many relationship based on a primary/foreign key. So I would
like to tag the foreign key column (hive_column in B) with a trait that holds a
reference to the correspunding primary key (hive_column in A).
I have already made this case work with Atlas 0.5 but upgrading to 0.8 breaks
it.
Regards.
> Regression :Traits Attribute that References Another Entity
> ---
>
> Key: ATLAS-1990
> URL: https://issues.apache.org/jira/browse/ATLAS-1990
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
> Environment: HDP2.6
>Reporter: Benjamin BONNET
>
> Hi,
> I defined a trait that includes one attribute. That attribute has type
> "hive_column", i.e. it references a hive_column entity.
> Using Altas 0.5 on a HDP 2.4, I could attach that trait to an entity so that
> it refers a column. But using Atlas 0.8 on a HDP 2.6 that does not work
> anymore (I get a NullPointerException).
> I tried both V1 and V2 REST API and got the same error.
> Here is a full sample to reproduce that issue.
> Trait definition :
> {noformat}
> {"enumTypes":[],
> "structTypes":[],
> "traitTypes":[{
>"superTypes":[],
>"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType",
>"typeName":"Reference",
>"typeDescription":"reference test",
>"attributeDefinitions":[{"name":"ref",
>"dataTypeName":"hive_column",
> "multiplicity":"required",
> "isComposite":false,
> "isUnique":false,
> "isIndexable":true,
> "reverseAttributeName":null
> }
> ]}],
> "classTypes":[]}
> {noformat}
> Then I use curl to create that type (ref file contains the traits definition):
> {noformat}
> curl -v -u admin:admin -X POST -d "`cat ref`" -H 'Content-Type:
> application/json' 'http://127.0.0.1:21000/api/atlas/types'
> {noformat}
> Using that command, trait type is created OK on both 0.5 and 0.8.
> Now, I can tag an entity on Atlas 0.5 :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
> "typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"9e2b0059-8e75-42ce-a32c-fe9a6157caed","version":0,"typeName":"hive_column"}}}'
> -H "Content-Type: application/json"
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits";
> {noformat}
> I get a HTTP201 :
> < HTTP/1.1 201 Created
> < Date: Tue, 25 Jul 2017 09:11:23 GMT
> < Location:
> http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits/23d1c3e7-7ad9-47b6-ac54-1d507119d00d
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-15 -
> 6d4b21dd-af88-46f3-91f4-5b3b20b6840c","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d"}
> And if I check my entity traits, I can see that tag.
> {noformat}
> curl -v -u admin:admin -X GET
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits";
> * About to connect() to 127.0.0.1 port 21000 (#0)
> * Trying 127.0.0.1... connected
> * Connected to 127.0.0.1 (127.0.0.1) port 21000 (#0)
> * Server auth using Basic with user 'admin'
> > GET /api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits HTTP/1.1
> > Authorization: Basic YWRtaW46YWRtaW4=
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1
> > Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: 127.0.0.1:21000
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Date: Tue, 25 Jul 2017 09:13:27 GMT
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-13 -
> 15a0ce26-484a-4181-9f5a-742227be5d09","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d","results":["tag1","tag2","Reference","tag3"],"count":4}
> {noformat}
> So far, so good, but If I try to do the same thing Atlas 0.8, i.e. tagging an
> entity with that trait :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSeriali
