Dear Wiki user, You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The "Pig070IncompatibleChanges" page has been changed by OlgaN. http://wiki.apache.org/pig/Pig070IncompatibleChanges?action=diff&rev1=4&rev2=5 -------------------------------------------------- == Changes to the Load and Store Functions == - TBW + TBW [Need to take a load (with and withoutcustom slicer) and a store function and create new versions as examples. Can use PigStorage for (1) and (3) but need some loader for (2).] + == Handling Compressed Data == @@ -21, +22 @@ If you have a custom load/store function that needs to support compression, you would need to make sure that the underlying Input/OutputFormat supports this type of compression. == Local Mode == + + The main change here is that we switched from Pig's native local mode to Hadoop's local mode. This change should be transparent for most applications. Possible differnces you will see: + + 1. Hadoop local mode is about order of magnitude slower than Pig's local mode. Something that Hadoop team promised to address. + 2. For algebraic functions, no the entire Algebraic interface will be used which is likely a good think if you are using local mode for testing your production applications. + == Streaming == There are two things that are changing in streaming. First, in the initial (0.7.0) release, '''we will not support for optimization''' where if streaming follows load of compatible format or is followed by format compatible store the data is not parsed but passed in chunks from the loader or to the store. The main reason we are not porting the optimization is that the work is not trivial and that the optimization was never documented and so unlekly to be used. - Second, '''you can no longer use load/store functions for (de)serialization.''' + Second, '''you can no longer use load/store functions for (de)serialization.''' A new interface has been defined that needed to be implemented for custom (de)serializations. The defaul (PigStorage) format will continue to work. Details of the new interface are describe in http://wiki.apache.org/pig/LoadStoreRedesignProposal. == Split by File ==