Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change 
notification.

The following page has been changed by OlgaN:
http://wiki.apache.org/pig/PigStreamingFunctionalSpec

------------------------------------------------------------------------------
  define X `stream.pl foo.txt` ship('stream.pl', 'foo.txt')
  }}}
  
+ Note that Pig would not automatically ship any dependecies. It is the 
responsibility of the user to specify all the dependencies explicitely and also 
make sure that the software that the processing relies on such as, for 
instance, `perl` or `python` is installed on the cluster.
+ 
  If `ship` and `cache` options are not specified, pig will attempt to ship the 
binary in the following way:
  
     * If the first word on the streaming command is `perl` or `python`, pig 
would assume that the binary is the first string it encounters that does not 
start with dash.
-    * Otherwise, pig will attempt to ship the first string from the command 
line as long as it does not come from `/bin, /user/bin, /user/local/bin, 
/home/y/bin`. It will determine that by scanning the path if an absolute path 
is provided or by executing `which`.
+    * Otherwise, pig will attempt to ship the first string from the command 
line as long as it does not come from `/bin, /user/bin, /user/local/bin`. It 
will determine that by scanning the path if an absolute path is provided or by 
executing `which`. The paths can be made configurable via `set stream.skippath 
<paths>` option.
  
- To prevent a command from being shipped, an empty list can be passed to 
`clause`.
+ To prevent a command from being shipped, an empty list can be passed to 
`ship` clause.
  
  Note that we need to make sure that executables retain their permissions and 
can be executed on the compute nodes.
  

Reply via email to