Dear Wiki user, You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The "LoadStoreRedesignProposal" page has been changed by PradeepKamath. http://wiki.apache.org/pig/LoadStoreRedesignProposal?action=diff&rev1=13&rev2=14 -------------------------------------------------- Nov 2 2009, Pradeep Kamath - In LoadFunc: + In !LoadFunc: - * Added relativeToAbsolutePath() method in LoadFunc per http://issues.apache.org/jira/browse/PIG-879?focusedCommentId=12768818&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12768818 + * Added relativeToAbsolutePath() method in !LoadFunc per http://issues.apache.org/jira/browse/PIG-879?focusedCommentId=12768818&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12768818 * Changed comments in setLocation regarding the location passed - the location will now be the return value of relativeToAbsolutePath() - * setLocation() now also takes a Job argument since the main purpose of this call is to an opportunity to the LoadFunc implementation to communicate the input location to underlying InputFormat. InputFormat implementations inturn seem to be storing this information inthe Job. For example, FileInputFormat has the following static method to set the input location: setInputPaths(JobConf conf, String commaSeparatedPaths) ; + * setLocation() now also takes a Job argument since the main purpose of this call is to an opportunity to the !LoadFunc implementation to communicate the input location to underlying !InputFormat. !InputFormat implementations inturn seem to be storing this information inthe Job. For example, !FileInputFormat has the following static method to set the input location: setInputPaths(JobConf conf, String commaSeparatedPaths) ; * Removed doneReading() method since there is already a RecordReader.close() method which will be called by Hadoop wherein all the functionality that needs to be done on completion of reading can be done. * All methods now can throw IOException - this keeps the interface more flexible for exception cases - In LoadMetadata: + In !LoadMetadata: * getSchema(), getStatistics() and getPartitionKeys() methods now take a location and Configuration argument so that the implementation can use that information in returning the information requested. - In StoreFunc: + In !StoreFunc: - * Added relativeToAbsolutePath() method per http://issues.apache.org/jira/browse/PIG-879?focusedCommentId=12768818&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12768818 + * Added relToAbsPathForStoreLocation() method per http://issues.apache.org/jira/browse/PIG-879?focusedCommentId=12768818&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12768818 * Methods which did not throw IOException now do so to enable exceptions in implementations - * Removed doneWriting() - same functionality already present in RecordWriter.close() and OutputCommitter.commitTask() + * Removed doneWriting() - same functionality already present in !RecordWriter.close() and !OutputCommitter.commitTask() * Changed setSchema() to checkSchema since this method is called only to allow StoreFunc to check - * Removed allFinished() - same functionality already present in OutputCommitter.cleanupJob() + * Removed allFinished() - same functionality already present in !OutputCommitter.cleanupJob()
