[
https://issues.apache.org/jira/browse/PIG-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876225#comment-14876225
]
Rohini Palaniswamy commented on PIG-4554:
-----------------------------------------
bq. Misunderstood "Submit Patch" as the workflow to submit the patch.
You were right the first time. Once you upload patch. You click on "Submit
Patch" to make it "Patch Available".
Have few comments based on the new feature we added to show the uncompressed
Pig script in Tez UI.
1) ScriptState.java
- Can you also retain the old truncated script in a new variable
- Can you change current getScript() to getSerializedScript(); and
TezScriptState and MRScriptState refer to that.
- Can you change getScript() to now return the truncated orginal script.
2) TezJobCompiler.java
Change
String script = new
String(Base64.decodeBase64(TezScriptState.get().getScript()));
tezDag.setDAGInfo(createDagInfo(script));
to
tezDag.setDAGInfo(createDagInfo(TezScriptState.get().getScript())); // The
truncated uncompressed script is shown in the Tez DAG UI. I have seen a lot of
huge scripts. So better to show the truncated one here and have folks go
decompress pig.script if full script is needed.
> Compress pig.script before encoding
> -----------------------------------
>
> Key: PIG-4554
> URL: https://issues.apache.org/jira/browse/PIG-4554
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.14.0
> Reporter: Rohini Palaniswamy
> Labels: newbie
> Fix For: 0.16.0
>
> Attachments: PIG-4554.patch
>
>
> Currently we truncate the pig script (maxScriptSize = 10240) and base64
> encode it and store in config. We should remove the truncation and store the
> full script by compressing and then doing base64 encoding. We already do that
> for udfcontext, etc. It will save space as it will compress really well and
> will also give the full pig script while debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
