liyunzhang_intel created PIG-4239:
-------------------------------------
Summary: "pig.output.lazy" not works in spark mode
Key: PIG-4239
URL: https://issues.apache.org/jira/browse/PIG-4239
Project: Pig
Issue Type: Bug
Components: spark
Reporter: liyunzhang_intel
Assignee: liyunzhang_intel
If "pig.output.lazy" is "true", empty part files from the output will be
omitted.
steps to reproduce
1. set "pig.output.lazy" as “true" in $PIG_HOME/conf/pig.properties.
2. run following lazy.pig script in spark mode:
cat lazy.pig
a = load '/user/pig/lazy' using PigStorage();
b = filter a by $0 == 'hey';
c = store b into '/tmp/lazy.out';
lazy.pig and lazy are attached
3. empty file "/tmp/lazy.out/part-rxxxx” still is generated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
