[
https://issues.apache.org/jira/browse/HDFS-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhan Sundararajan Devaki updated HDFS-8012:
---------------------------------------------
Description:
Improvement: Updatable HAR Filesystem.
The following operations may be supported additionally.
+ Add new files [ -a filename-uri1 filename-uri2 ... / -a dirname-uri1
dirname-uri2 ...]
+ Remove existing files [ -d filename-uri1 filename-uri2 ... / -d dirname-uri1
dirname-uri2 ...]
+ Update/Replace existing files (Optional) [ -u old-filename-uri
new-filename-uri]
This is required in cases where data is stored in AVRO format in HDFS and the
corresponding .avsc files are used to create Hive external tables.
This will lead to the small files (.avsc files in this case) problem when there
are a large number of tables that need to be loaded into Hive as external
tables as is the typical case during a Datawarehouse migration.
was:
Is there a plan to support updatable HAR Filesystem? If so, by when is this
expected please?
The following operations may be supported.
+ Add new files [ -a filename-uri1 filename-uri2 ... / -a dirname-uri1
dirname-uri2 ...]
+ Remove existing files [ -d filename-uri1 filename-uri2 ... / -d dirname-uri1
dirname-uri2 ...]
+ Update/Replace existing files (Optional) [ -u old-filename-uri
new-filename-uri]
This is required in cases where data is stored in AVRO format in HDFS and the
corresponding .avsc files are used to create Hive external tables.
This will lead to the small files (.avsc files in this case) problem when there
are a large number of tables that need to be loaded into Hive as external
tables.
> Updatable HAR Filesystem
> ------------------------
>
> Key: HDFS-8012
> URL: https://issues.apache.org/jira/browse/HDFS-8012
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, hdfs-client
> Reporter: Madhan Sundararajan Devaki
> Priority: Critical
>
> Improvement: Updatable HAR Filesystem.
> The following operations may be supported additionally.
> + Add new files [ -a filename-uri1 filename-uri2 ... / -a dirname-uri1
> dirname-uri2 ...]
> + Remove existing files [ -d filename-uri1 filename-uri2 ... / -d
> dirname-uri1 dirname-uri2 ...]
> + Update/Replace existing files (Optional) [ -u old-filename-uri
> new-filename-uri]
> This is required in cases where data is stored in AVRO format in HDFS and the
> corresponding .avsc files are used to create Hive external tables.
> This will lead to the small files (.avsc files in this case) problem when
> there are a large number of tables that need to be loaded into Hive as
> external tables as is the typical case during a Datawarehouse migration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
