Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change 

The "Pig070IncompatibleChanges" page has been changed by OlgaN.


  == Changes to the Load and Store Functions ==
+ TBW [Need to take a load (with and withoutcustom slicer) and a store function 
and create new versions as examples. Can use PigStorage for (1) and (3) but 
need some loader for (2).]
  == Handling Compressed Data ==
@@ -21, +22 @@

  If you have a custom load/store function that needs to support compression, 
you would need to make sure that the underlying Input/OutputFormat supports 
this type of compression.
  == Local Mode ==
+ The main change here is that we switched from Pig's native local mode to 
Hadoop's local mode. This change should be transparent for most applications. 
Possible differnces you will see:
+  1. Hadoop local mode is about order of magnitude slower than Pig's local 
mode. Something that Hadoop team promised to address.
+  2. For algebraic functions, no the entire Algebraic interface will be used 
which is likely a good think if you are using local mode for testing your 
production applications.
  == Streaming ==
  There are two things that are changing in streaming.
  First, in the initial (0.7.0) release, '''we will not support for 
optimization''' where if streaming follows load of compatible format or is 
followed by format compatible store the data is not parsed but passed in chunks 
from the loader or to the store. The main reason we are not porting the 
optimization is that the work is not trivial and that the optimization was 
never documented and so unlekly to be used.
- Second, '''you can no longer use load/store functions for 
+ Second, '''you can no longer use load/store functions for 
(de)serialization.''' A new interface has been defined that needed to be 
implemented for custom (de)serializations. The defaul (PigStorage) format will 
continue to work. Details of the new interface are describe in
  == Split by File ==

Reply via email to