Dear Wiki user, You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The "Pig070IncompatibleChanges" page has been changed by RichardDing. http://wiki.apache.org/pig/Pig070IncompatibleChanges?action=diff&rev1=37&rev2=38 -------------------------------------------------- || Removing Split by file feature || Split by File || Low to None || Input format of the loader would need to support this || We don't know that this feature was widely/ever used || || Local files no longer accessible from cluster || Access to Local Files from Map-Reduce Mode || low to none || copy the file to the cluster using copyToLocal command prior to the load || This feature was not documented || || Removing Custom Comparators || Removing Custom Comparators || Low to None || None || This feature has been deprecated since Pig 0.5.0 release. We don't have a single known use case || - || Using PigFileInputFormat and PigTextInputFormat || Changes to custom Load Functions || Low to medium || Custom loaders using a text-based input format and like to support recursive file listing need to use these classes || This is to get around the problem of MAPREDUCE-1577. || + || Using !PigFileInputFormat and !PigTextInputFormat || Changes to custom Load Functions || Low to medium || Custom loaders using a text-based input format and like to support recursive file listing need to use these classes || This is to get around the problem of MAPREDUCE-1577. || == Changes to the Load and Store Functions == @@ -90, +90 @@ == Merge Join == In Pig.0.6.0 there was a pre-condition for merge join: "The loadfunc for the right input of the join should implement the !SamplableLoader interface" - instead the !LoadFunc should now implement !OrderedLoadFunc interface in Pig 0.7.0. All other pre-condtions still hold. + == PigFileInputFormat and PigTextInputFormat == + + Given a load location, Pig 0.6.0 loaders load recursively all files under the location (can be multi-level). To get around MAPREDUCE-1577, Pig 0.7.0 adds !PigFileInputFormat and !PigTextInputFormat classes. They are subclasses of the Hadoop !FileInputFormat and !TextInputFormat overriding the listStatus method to support multi-level/recursive directory/file listing. Any custom loader that uses !FileInputFormat or !TextInputFormat, and wants to support recursive file listing should use the corresponding Pig version of the !InputFormat. +
