If you set pretty verbose level in log4j then you can see output in log files. I've seen it there. Then you can use regexps to strip out paragraph outputs from rest of debugging messages. May work as a one off effort. Might be a good idea to file an enhancement request - this can be also useful for scheduled notebook runs - would be great to go back and review each scheduled note executions etc.
On Tue, Jun 19, 2018 at 2:56 AM Alessandro Liparoti < alessandro.l...@gmail.com> wrote: > I am comparing performances between different implementations of a spark > job and I am testing a chunk of code which prints partial results and info > to sdtout. I can surely replace all the prints with logger calls and > collect them. I just wanted to know if there was a way to avoid this or if > this functionality was of easier implementation. > > *Alessandro Liparoti* > > 2018-06-19 10:52 GMT+02:00 Jeff Zhang <zjf...@gmail.com>: > >> >> Not sure what kind of analysis you want to do, is the logging info in the >> interpreter log file enough for you ? (You can update the log level in >> log4j.properties to get more logs) >> >> Alessandro Liparoti <alessandro.l...@gmail.com>于2018年6月19日周二 下午4:47写道： >> >>> I would like to post-analyze the output of verbose jobs in the notebook >>> and save them, avoiding to relaunch the jobs again. It would be also good >>> to have the stderr logged to file. >>> >>> Thanks >>> >>> *Alessandro Liparoti* >>> >>> 2018-06-19 10:43 GMT+02:00 Jeff Zhang <zjf...@gmail.com>: >>> >>>> >>>> I am not afraid it is not possible now. The stdout of notebooks is not >>>> based on log4j. If you want it output to file as well, you might need to >>>> change the code of the interpreter itself. >>>> Usually it is not necessary to log it to log file as well, could you >>>> tell why you want that ? Thanks >>>> >>>> >>>> >>>> alessandro.l...@gmail.com <alessandro.l...@gmail.com>于2018年6月19日周二 >>>> 下午3:52写道： >>>> >>>>> Good morning, >>>>> I would like to have stdout of notebooks both printed out to console >>>>> and file. How can I achieve that? I tried to play around with log4j but >>>>> without any success; it seems it requires a custom appender >>>>> implementation. >>>>> Any other simpler idea? >>>>> >>>> >>> >
