Nick White created PIG-3095:
-------------------------------
Summary: "which" is called many, many times for each Pig STREAM
statement
Key: PIG-3095
URL: https://issues.apache.org/jira/browse/PIG-3095
Project: Pig
Issue Type: Bug
Components: grunt, impl
Affects Versions: 0.12
Reporter: Nick White
Assignee: Nick White
Fix For: 0.12
Attachments: PIG-3095.patch
STREAM statements are checked by the LogicalPlanBuilder as it comes across them
- and these checks include running the system utility "which". However, due to
the backtracking parsing mechanism "which" is called repeatedly with the same
arguments (I noticed this while profiling a script with 4 STREAM statements -
"which" was run over 230 times!). The attached patch just caches the return
value of "which", reducing the overhead of running a system process to a Map
lookup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
