[
https://issues.apache.org/jira/browse/FALCON-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Kawa updated FALCON-1096:
------------------------------
Summary: Scan Hive Metastore to automatically create Falcon feeds for
existing Hive tables (was: Scanning Hive Metastore to automatically create
Falcone feed for existing Hive tables)
> Scan Hive Metastore to automatically create Falcon feeds for existing Hive
> tables
> ---------------------------------------------------------------------------------
>
> Key: FALCON-1096
> URL: https://issues.apache.org/jira/browse/FALCON-1096
> Project: Falcon
> Issue Type: New Feature
> Reporter: Adam Kawa
>
> In my organisation we create a Hive table for each production dataset in
> HDFS. When creating a Hive table, you supply a lot of information about your
> dataset: its name, fields and their types and comments, the location, the
> data format, properties in form of the key-value pairs and meaningful
> description of the dataset. We think of Hive as a central and nicely
> documented repository of our datasets.
> When using Falcon, we again need to create Falcon feed for each dataset (that
> corresponds to a Hive table) and even specify multiple redundant properties
> (e.g. description).
> To make it simpler, Falcon could scan the Hive Metastore and automatically
> create feeds for each Hive table and inherit its properties.
> The properties of Hive tables could be also used when searching for a dataset
> using new Falcon Web UI e.g. field name, field comment, file format (some
> other statistics like total size, the last modification or access time could
> be also used).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
