[ 
https://issues.apache.org/jira/browse/PIG-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716722#action_12716722
 ] 

Jeff Hammerbacher commented on PIG-823:
---------------------------------------

Hey Alan,

Thanks for the additional detail. I suppose I can wait for the document to be 
released to the public, but it sounds as if you're creating a separate 
"extended attributes" service to host non-core file and directory metadata 
separately from the NN. It's not clear to me that this is a positive 
development for Hadoop. Perhaps we should spend the engineering effort on a 
single, partitioned, available metadata service for all file and directory 
attributes? The project has larger scope and requires but is potentially a 
cleaner solution for the long term.

Later,
Jeff

> Hadoop Metadata Service
> -----------------------
>
>                 Key: PIG-823
>                 URL: https://issues.apache.org/jira/browse/PIG-823
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Olga Natkovich
>
> This JIRA is created to track development of a metadata system for  Hadoop. 
> The goal of the system is to allow users and applications to register data 
> stored on HDFS, search for the data available on HDFS, and associate metadata 
> such as schema, statistics, etc. with a particular data unit or a data set 
> stored on HDFS. The initial goal is to provide a fairly generic, low level 
> abstraction that any user or application on HDFS can use to store an retrieve 
> metadata. Over time a higher level abstractions closely tied to particular 
> applications or tools can be developed.
> Over time, it would make sense for the metadata service to become a 
> subproject within Hadoop. For now, the proposal is to make it a contrib to 
> Pig since Pig SQL is likely to be the first user of the system.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to