[
https://issues.apache.org/jira/browse/ATLAS-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikhil Bonte updated ATLAS-3879:
--------------------------------
Description:
The support for Apache Ozone was added in ATLAS-3836. Currently, ozone_key
entity is getting created under ozone_bucket, but there is no support to create
a hierarchy of parent ozone_key and child ozone_key.
e.g.
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated
by ' ' stored as textfile location
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
This query will create only 1 ozone_key directly under bucket "bucket1".
Created entities :
* o3fs://volume1@cm (ozone_volume)
* o3fs://volume1.bucket1@cm (ozone_bucket)
* o3fs://bucket1.volume1.ozone1/sale1/q1/sales@cm (ozone_key)
This Jira will improve implementation to support creating the hierarchy of
ozone_keys.
With this, above query will result into following entites
* o3fs://volume1@cm (ozone_volume)
* o3fs://volume1.bucket1@cm (ozone_bucket)
* o3fs://bucket1.volume1.ozone1/sale1/@cm (ozone_key) *child of ozone_bucket*
* o3fs://bucket1.volume1.ozone1/sale1/q1/@cm (ozone_key) *child of ozone_key*
* o3fs://bucket1.volume1.ozone1/sale1/q1/sales/@cm (ozone_key) *child of
ozone_key*
This will bring Ozone model to be consistent with AWS S3 v2 and ADLS-Gen2
models.
was:
The support for Apache Ozone was added in ATLAS-3836. Currently, ozone_key
entity is getting created under ozone_bucket, but there is no support to create
a hierarchy of parent ozone_key and child ozone_key.
e.g.
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated
by ' ' stored as textfile location
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
This query will create only 1 ozone_key directly under bucket "bucket1".
Created entities :
* o3fs://volume1@cm (ozone_key)
* o3fs://volume1.bucket1@cm (ozone_bucket)
* o3fs://bucket1.volume1.ozone1/sale1/q1/sales@cm (ozone_key)
This Jira will improve implementation to support creating the hierarchy of
ozone_keys.
With this, above query will result into following entites
* o3fs://volume1@cm (ozone_key)
* o3fs://volume1.bucket1@cm (ozone_bucket)
* o3fs://bucket1.volume1.ozone1/sale1/@cm (ozone_key) *child of ozone_bucket*
* o3fs://bucket1.volume1.ozone1/sale1/q1/@cm (ozone_key) *child of ozone_key*
* o3fs://bucket1.volume1.ozone1/sale1/q1/sales/@cm (ozone_key) *child of
ozone_key*
This will bring Ozone model to be consistent with AWS S3 v2 and ADLS-Gen2
models.
> Ozone: ozone_key entity is directly created under ozone_bucket
> --------------------------------------------------------------
>
> Key: ATLAS-3879
> URL: https://issues.apache.org/jira/browse/ATLAS-3879
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Affects Versions: 2.0.0
> Reporter: Nikhil Bonte
> Assignee: Nikhil Bonte
> Priority: Major
> Fix For: 2.1.0, 3.0.0
>
>
> The support for Apache Ozone was added in ATLAS-3836. Currently, ozone_key
> entity is getting created under ozone_bucket, but there is no support to
> create a hierarchy of parent ozone_key and child ozone_key.
>
> e.g.
> {code:java}
> -> CREATE EXTERNAL TABLE sales (id int) row format delimited fields
> terminated by ' ' stored as textfile location
> 'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
> This query will create only 1 ozone_key directly under bucket "bucket1".
> Created entities :
> * o3fs://volume1@cm (ozone_volume)
> * o3fs://volume1.bucket1@cm (ozone_bucket)
> * o3fs://bucket1.volume1.ozone1/sale1/q1/sales@cm (ozone_key)
>
> This Jira will improve implementation to support creating the hierarchy of
> ozone_keys.
> With this, above query will result into following entites
> * o3fs://volume1@cm (ozone_volume)
> * o3fs://volume1.bucket1@cm (ozone_bucket)
> * o3fs://bucket1.volume1.ozone1/sale1/@cm (ozone_key) *child of ozone_bucket*
> * o3fs://bucket1.volume1.ozone1/sale1/q1/@cm (ozone_key) *child of ozone_key*
> * o3fs://bucket1.volume1.ozone1/sale1/q1/sales/@cm (ozone_key) *child of
> ozone_key*
>
> This will bring Ozone model to be consistent with AWS S3 v2 and ADLS-Gen2
> models.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
