anuengineer commented on pull request #921: URL: https://github.com/apache/hadoop-ozone/pull/921#issuecomment-648375979
> The longest line is 7434 characters long. Even if you remove the `docker-compose`-specific prefix, which does not appear in real clusters, we still have 7420 chars. What you are pointing out it is the there are lines which are really long, and generally when you see a line of 7434, you need a something that warps that info for you to read effectively. I was agree to that. This is an example of what I meant by no carriage return, not literally, but in the practical sense of machine generated stuff that gets consumed -- it is hard to even know what the length is. So we treat these insanely long lines as if they are without carriage return since whatever wrapping you need the consumer puts them up. > Each line starting with datanode_1 | follows a carriage return at the end of the previous line. As you correctly pointed out, each of these line can be really long. -- **7434** chars is extremely long and has absolutely no bearing on the discussion of lines length which is human produced and neatly formatted like source code. That is why we are discussing 80/120 chars. There is also another point, it is trivial to attach all kinds of collectors to the log pipeline -- log4j itself ships with some many. So Transformation on data is designed to be trivial and if you need to make it different, it is simply possible. So to be precise: I was trying to stress the point that while each log is separated by a carriage return, the general expectation is that lines can be really long because each line in a log is self contained -- and each source line that produces that information is producing it without reference to other parts of the code. This property makes it easy to write parsers and indexers over log files. In fact, there is a very large industry which produces tools to solve this issue. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
