[
https://issues.apache.org/jira/browse/TEZ-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15685220#comment-15685220
]
Hitesh Shah commented on TEZ-3545:
----------------------------------
The config knob does indicate that it will generate debug artifacts. The dag
plan is published in its entirety to aid debugging. Why not just set the config
knob to false?
> Don't log tez user payload at any level
> ---------------------------------------
>
> Key: TEZ-3545
> URL: https://issues.apache.org/jira/browse/TEZ-3545
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.9.0, 0.8.4
> Reporter: Prasanth Jayachandran
> Priority: Blocker
>
> Even at WARN log level LLAP log files end up at 24MB after very few runs.
> Looks like tez_user_payload logs the binary formatted payload information
> which I don't think is useful.
> Excerpt from application log:
> {code}
> End of LogType:dag_1477340478603_0832_3.dot
> LogType:dag_1477340478603_0832_4-tez-dag.pb.txt
> Log Upload Time:Mon Nov 21 18:23:27 -0500 2016
> LogLength:6099817
> Log Contents:
> name: "select count(*) \nfrom (select distinc...null(Stage-1)"
> vertex {
> name: "Map 7"
> type: NORMAL
> processor_descriptor {
> class_name: "org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor"
> tez_user_payload {
> user_payload:
> "x\001\314\275KscW\266&\026)\245^\347V\326\275\005\225\252JRI\312\2222KR2\361~\021Y\245\222@<\b\222
>
> @\002|\337\354K\037\000\a\017\022/\236\003\020$\253\025\216p\207\303\341\b\207#<\350n\317\034\366\017\260\303\203\236\370/x\352\221\247\0368\302c\207\307\376\326Z{\237\a@*U\367\206\243\255\356[I\000\347\271\367\332\353\361\255o\255m<7~?2\247\266\325\231\267\255\310\305\2445\263\315\366\245eG\314N\307\266\034\'\364\316p\3226\207\306\277}\313\370\344\326\264\307\021s:\035\016\332\346l0\031G\332C\323q\246\346\254\037\372^?\036=\253\344\213\365\372\336y\241^+\237\027\267\032/\243s\307\216\366;\323h{n\333\326x\026\355\233\235\311d\032n\017\a\364\351\305\233\016\030\016Z?qP\277\323u\336|)\377Q?}=z\2677_\317^?\024_\317\370\306\370\245\fKg4\300\340\264\207\241\337\230\355\331\340z0\273=7\307\346\360\366\316\262_\322\021\306s\343\327\375\301\265\025\261n\254v\304\231\231\275\301\270\327\031\330\241\'\021\372:\254\2761\376d$\344\202\323i\304\233\030sDs\023iOF\243\301l\206\351i\233\343\2665\f\317\006#k2\237\205\336\311\304\360\237\361G\343s\357\036\023\273\035\351X]s>\234\321\211\230cL\347\343\263\352\326\206\2214\376\300\307u\354\371\240\023i\331\023\337\214\353sB\277\344\231\357O\234\331\253\365\330z\302\370\334\370H\036\210\236k\212g\030[\213\2609\035\204\336\351\232C\307ro>\034\232\323\310`\022\241\373\323\363\341\346\3631\236\0317\2
> ....
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
