[ https://issues.apache.org/jira/browse/PIG-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716722#action_12716722 ]
Jeff Hammerbacher commented on PIG-823: --------------------------------------- Hey Alan, Thanks for the additional detail. I suppose I can wait for the document to be released to the public, but it sounds as if you're creating a separate "extended attributes" service to host non-core file and directory metadata separately from the NN. It's not clear to me that this is a positive development for Hadoop. Perhaps we should spend the engineering effort on a single, partitioned, available metadata service for all file and directory attributes? The project has larger scope and requires but is potentially a cleaner solution for the long term. Later, Jeff > Hadoop Metadata Service > ----------------------- > > Key: PIG-823 > URL: https://issues.apache.org/jira/browse/PIG-823 > Project: Pig > Issue Type: New Feature > Reporter: Olga Natkovich > > This JIRA is created to track development of a metadata system for Hadoop. > The goal of the system is to allow users and applications to register data > stored on HDFS, search for the data available on HDFS, and associate metadata > such as schema, statistics, etc. with a particular data unit or a data set > stored on HDFS. The initial goal is to provide a fairly generic, low level > abstraction that any user or application on HDFS can use to store an retrieve > metadata. Over time a higher level abstractions closely tied to particular > applications or tools can be developed. > Over time, it would make sense for the metadata service to become a > subproject within Hadoop. For now, the proposal is to make it a contrib to > Pig since Pig SQL is likely to be the first user of the system. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.