Dear Wiki user, You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The "Pig070IncompatibleChanges" page has been changed by OlgaN. http://wiki.apache.org/pig/Pig070IncompatibleChanges?action=diff&rev1=6&rev2=7 -------------------------------------------------- == Local Mode == - The main change here is that we switched from Pig's native local mode to Hadoop's local mode. This change should be transparent for most applications. Possible differnces you will see: + The main change here is that we switched from Pig's native local mode to Hadoop's local mode. This change should be transparent for most applications. Possible differnces you will see are: 1. Hadoop local mode is about order of magnitude slower than Pig's local mode. Something that Hadoop team promised to address. - 2. For algebraic functions, no the entire Algebraic interface will be used which is likely a good think if you are using local mode for testing your production applications. + 2. For algebraic functions, now the entire Algebraic interface will be used which is likely a good thing if you are using local mode for testing your production applications. == Streaming == There are two things that are changing in streaming. - First, in the initial (0.7.0) release, '''we will not support for optimization''' where if streaming follows load of compatible format or is followed by format compatible store the data is not parsed but passed in chunks from the loader or to the store. The main reason we are not porting the optimization is that the work is not trivial and that the optimization was never documented and so unlikely to be used. + First, in the initial (0.7.0) release, '''we will not support optimization''' where if streaming follows load of compatible format or is followed by format compatible store the data is not parsed but passed in chunks from the loader or to the store. The main reason we are not porting the optimization is that the work is not trivial and the optimization was never documented and so unlikely to be used. Second, '''you can no longer use load/store functions for (de)serialization.''' A new interface has been defined that needed to be implemented for custom (de)serializations. The default (PigStorage) format will continue to work. Details of the new interface are describe in http://wiki.apache.org/pig/LoadStoreRedesignProposal. @@ -46, +46 @@ == Open Questions == + Q: Should String->Text conversion be part of this release. + A: Pros: 20-30% improved memory utilization; cons: more compatibility is broken. +
