Hi, I can help too. Lukas Dne pá 24. 1. 2020 15:29 uživatel David Mollitor <[email protected]> napsal:
> Hello Team, > > I am happy to do the work of reviewing all Parquet logging, but I need help > getting the work committed. > > Fokko Driesprong has been a wonderfully ally in helping me get incremental > improvements into Parquet, but I wonder if there's anyone else that can > share in the load. > > Thanks, > David > > On Thu, Jan 23, 2020 at 11:55 AM Michael Heuer <[email protected]> wrote: > > > Hello David, > > > > As I mentioned on PARQUET-1758, we have been frustrated by overly verbose > > logging in Parquet for a long time. Various workarounds have been more > or > > less successful, e.g. > > > > https://github.com/bigdatagenomics/adam/issues/851 < > > https://github.com/bigdatagenomics/adam/issues/851> > > > > I would support a move making Parquet a silent partner. :) > > > > michael > > > > > > > On Jan 23, 2020, at 10:25 AM, David Mollitor <[email protected]> > wrote: > > > > > > Hello Team, > > > > > > I have been a consumer of Apache Parquet through Apache Hive for > several > > > years now. For a long time, logging in Parquet has been pretty > painful. > > > Some of the logging was going to STDOUT and some was going to Log4J. > > > Overall, though the framework has been too verbose, spewing many log > > lines > > > about internal details of Parquet I don't understand. > > > > > > The logging has gotten a lot better with recent releases moving solidly > > > into SLF4J. That is awesome and very welcomed. However, (opinion > > alert) I > > > think the logging is still too verbose. I think Parquet should be a > > silent > > > partner in data processing. If everything is going well, it should be > > > silent (DEBUG level logging). If things are going wrong, it should > throw > > > an Exception. > > > > > > If an operator suspects Parquet is the issue (and that's rarely the > first > > > thing to check), they can set the logging for all of the Loggers in the > > > entire Parquet package (org.apache.parquet) to DEBUG to get the > required > > > information. Not to mention, the less logging it does, the faster it > > will > > > be. > > > > > > I've opened this discussion because I've got two PRs related to this > > topic > > > ready to go: > > > > > > PARQUET-1758 > > > PARQUET-1761 > > > > > > Thanks, > > > David > > > > >
