[
https://issues.apache.org/jira/browse/PIG-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565977#action_12565977
]
Olga Natkovich commented on PIG-58:
-----------------------------------
This is in response to comment
https://issues.apache.org/jira/browse/PIG-58?focusedCommentId=12565959#action_12565959
>> 1. The parameter usage notation seems to arcane
Hopefully, the new proposal to use #define to declare all variables addresses
this issue
>> 2. "-param" key seems redundant and will cause confusion
I usually prefer explicit rather than implicit specifications. Also, the -param
approach separates pigs command line namespace from user parameter one. But I
could go either way on this.
>> 3. We need an ability to specify "DFS working directory" for the whole Pig
>> job.
There is already way in pig to do that by issuing CD command
>> 4. Need to specify the precedence rules
I thought I already specified this in the proposal. Please, let me know which
parts are unclear.
>> 5. Can parameters be used in RHS of declare?
My plan was to avoid it for the initial release to keep things simple since
most of the use cases I saw did not need that. However, I don't think it would
be difficult to implement so we can put that in.
> parameterized Pig scripts
> -------------------------
>
> Key: PIG-58
> URL: https://issues.apache.org/jira/browse/PIG-58
> Project: Pig
> Issue Type: New Feature
> Reporter: Olga Natkovich
>
> This feature has been requested by several users and would be very useful in
> conjunction with streaming. The feature would allow pig script to include
> parameters that are replaced at run time. For instance, if your script needs
> to run on a daily basis over the data of the previous day, you would be able
> to use the script and providing a date as a run-time parameter to it.
> Example:
> =======
> Pig script myscript.pig:
> A = load '/data/mydata/%date%';
> B = filter A by $0>'5';
> .....
> Pig command line:
> pig -param date='20080110' myscript.pig
> Proposed interface and implementation:
> Interface:
> =======
> (0) Substitution will be only supported with pig script files.
> (1) Parameters are specified on the command line via -param <param>=<val>
> construct. Multiple parameters can be specified. They are applied to the
> script in the order they are specified on the command line
> (2) Default values for the parameters can be specified within the script via
> decare statement:
> decare <param>=<value>
> (3) Withint the script the parameter will be enclosed in %%. \% can be used
> te escape.
> Implementation:
> ============
> Use preprocessor to do the substitution. The preprocessor would be invoced by
> Main before grunt is instanciated and do the following:
> - create a new file in temp location
> - build a hash of parameters from command line and declare statement
> - for each line in the original script
> if this is a declare line, skip it
> else for each unescaped pattern %<identifie>% look for a match in the hash.
> Replace, if found. Write the line to the temp file.
> - pass the temp file to grunt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
