Dear Wiki user, You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The following page has been changed by Arun C Murthy: http://wiki.apache.org/pig/PigStreamingFunctionalSpec ------------------------------------------------------------------------------ If `ship` and `cache` options are not specified, pig will attempt to ship the binary in the following way: - * If the first word on the streaming command is `perl` or `python`, pig would assume that the binary is the first string it encounters that does not start with dash. + * If the first word on the streaming command is `perl` or `python`, pig would assume that the binary is the first non-quoted string it encounters that does not start with dash. - * Otherwise, pig will attempt to ship the first string from the command line as long as it does not come from `/bin, /usr/bin, /usr/local/bin`. It will determine that by scanning the path if an absolute path is provided or by executing `which`. The paths can be made configurable via `set stream.skippath <paths>` option. + * Otherwise, pig will attempt to ship the first string from the command line as long as it does not come from `/bin, /usr/bin, /usr/local/bin`. It will determine that by scanning the path if an absolute path is provided or by executing `which`. The paths can be made configurable via `set stream.skippath <path>` option. (Users can use multiple `set` commands for specifying more than one path to skip.) To prevent a command from being shipped, an empty list can be passed to `ship` clause. - Note that we need to make sure that executables retain their permissions and can be executed on the compute nodes. + Note that we need to make sure that executables retain their permissions and can be executed on the compute nodes. + + If the user does supply a DEFINE for a given streaming command, then the above 'auto-shipping' is turned off. ==== 2.2 Ability to cache data ====
