[
https://issues.apache.org/jira/browse/ATLAS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070057#comment-15070057
]
Aaron Dossett commented on ATLAS-409:
-------------------------------------
It turns out that the TABLE columns are being populated, but the STORAGE
DESCRIPTOR columns are not. That make sense for a table with an external
schema definition? New patch make columns required on the HiveTable (was
optional previously) and makes them optional for the storage descriptor. All
tests pass and here is part of my import-hive.log showing that the columns are
being populated for the table:
{code}
"columns":[
{
"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id":{
"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id":"-1743844062456504",
"version":0,
"typeName":"hive_column"
},
"typeName":"hive_column",
"values":{
"comment":"",
"qualifiedName":"default.example.application@primary",
"type":"struct<foo:string,version:string,bar:string>",
"name":"application"
},
"traitNames":[
],
"traits":{
}
}
],
{code}
> Atlas will not import avro tables with schema read from a file
> --------------------------------------------------------------
>
> Key: ATLAS-409
> URL: https://issues.apache.org/jira/browse/ATLAS-409
> Project: Atlas
> Issue Type: Bug
> Affects Versions: 0.6-incubating
> Reporter: Aaron Dossett
> Assignee: Aaron Dossett
> Attachments: ATLAS-409.patch, example.ddl, foo.avsc
>
>
> Atlas won't import a Hive table with no columns (see below for an example of
> a valid hive table with no explicit columns). This is because the Atlas Hive
> Storage Descriptor class REQUIRES columns, but the Hive Table class allows
> them to be OPTIONAL.
> {code}
> CREATE TABLE example
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> TBLPROPERTIES (
> 'avro.schema.url'='file:///path/to/the/schema/test_serializer.avsc');
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
