[
https://issues.apache.org/jira/browse/TEZ-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15685317#comment-15685317
]
Hitesh Shah edited comment on TEZ-3545 at 11/22/16 1:16 AM:
------------------------------------------------------------
There is already a limited json based dag plan that is published to the logs in
debug log level. Is this what you are really looking for or are you saying that
you want to keep tez.generate.debug.artifacts set to true but at the same time
not publish the full proto in text mode?
was (Author: hitesh):
There is already a json based dag plan that is published to the logs in debug
log level. Is this what you are really looking for or are you saying that you
want to keep tez.generate.debug.artifacts set to true but at the same time not
publish the full proto in text mode?
> Don't log tez user payload at any level
> ---------------------------------------
>
> Key: TEZ-3545
> URL: https://issues.apache.org/jira/browse/TEZ-3545
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.9.0, 0.8.4
> Reporter: Prasanth Jayachandran
> Priority: Critical
>
> Even at WARN log level LLAP log files end up at 24MB after very few runs.
> Looks like tez_user_payload logs the binary formatted payload information
> which I don't think is useful.
> Excerpt from application log:
> {code}
> End of LogType:dag_1477340478603_0832_3.dot
> LogType:dag_1477340478603_0832_4-tez-dag.pb.txt
> Log Upload Time:Mon Nov 21 18:23:27 -0500 2016
> LogLength:6099817
> Log Contents:
> name: "select count(*) \nfrom (select distinc...null(Stage-1)"
> vertex {
> name: "Map 7"
> type: NORMAL
> processor_descriptor {
> class_name: "org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor"
> tez_user_payload {
> user_payload:
> "x\001\314\275KscW\266&\026)\245^\347V\326\275\005\225\252JRI\312\2222KR2\361~\021Y\245\222@<\b\222
>
> @\002|\337\354K\037\000\a\017\022/\236\003\020$\253\025\216p\207\303\341\b\207#<\350n\317\034\366\017\260\303\203\236\370/x\352\221\247\0368\302c\207\307\376\326Z{\237\a@*U\367\206\243\255\356[I\000\347\271\367\332\353\361\255o\255m<7~?2\247\266\325\231\267\255\310\305\2445\263\315\366\245eG\314N\307\266\034\'\364\316p\3226\207\306\277}\313\370\344\326\264\307\021s:\035\016\332\346l0\031G\332C\323q\246\346\254\037\372^?\036=\253\344\213\365\372\336y\241^+\237\027\267\032/\243s\307\216\366;\323h{n\333\326x\026\355\233\235\311d\032n\017\a\364\351\305\233\016\030\016Z?qP\277\323u\336|)\377Q?}=z\2677_\317^?\024_\317\370\306\370\245\fKg4\300\340\264\207\241\337\230\355\331\340z0\273=7\307\346\360\366\316\262_\322\021\306s\343\327\375\301\265\025\261n\254v\304\231\231\275\301\270\327\031\330\241\'\021\372:\254\2761\376d$\344\202\323i\304\233\030sDs\023iOF\243\301l\206\351i\233\343\2665\f\317\006#k2\237\205\336\311\304\360\237\361G\343s\357\036\023\273\035\351X]s>\234\321\211\230cL\347\343\263\352\326\206\2214\376\300\307u\354\371\240\023i\331\023\337\214\353sB\277\344\231\357O\234\331\253\365\330z\302\370\334\370H\036\210\236k\212g\030[\213\2609\035\204\336\351\232C\307ro>\034\232\323\310`\022\241\373\323\363\341\346\3631\236\0317\2
> ....
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
