To follow up, I did some research yesterday on removing --info and my findings are: - Gradle Test tasks generate HTML and Junit XML reports. Both contain a stacktrace, STDOUT, and STDERR of the failed test (example <https://builds.apache.org/job/beam_PreCommit_Java_Phrase/515/testReport/junit/org.apache.beam.runners.samza.runtime/SamzaStoreStateInternalsTest/testSetStateIterator/> ). So even though --info wasn't specified the output are not lost. - Python SDK tests don't use Test tasks (they exec tox), and thus are not affected by --info. Python tests aren't excessively verbose however. - Go tests should also generate reports (via gogradle), but I haven't found any and I can't seem to run ./gradlew :beam-sdks-go:test on my workstation.
Suggestion: - Remove --info (https://github.com/apache/beam/pull/7409) - If we find Gradle tasks that aren't somehow reporting or logging to console on failure, that's a bug and the task should be fixed. On Thu, Dec 20, 2018 at 6:09 AM Kenneth Knowles <[email protected]> wrote: > I support lowering the log level. The default is `lifecycle`. > > For reference, here's where it was increased to `info`: > https://github.com/apache/beam/pull/4644 > > The reason was to see more details about Gradle's dependency management. > We were seeing dependency download flakes on things that should not require > re-downloading. No longer an issue. > > To easily tweak it on a one-off basis without having to change a Jenkins > job, you can edit gradle.properties in a commit on your PR: > > org.gradle.logging.level=(quiet,warn,lifecycle,info,debug) > org.gradle.warning.mode=(all,none,summary) > org.gradle.console=(auto,plain,rich,verbose) > org.gradle.caching.debug=(true,false) > > Kenn > > On Thu, Dec 20, 2018 at 6:49 AM Robert Bradshaw <[email protected]> > wrote: > >> Interestingly, I was thinking exactly the same thing the other day. >> >> If we could drop the info logs for passing tests, that would be ideal. >> Regardless, tests should fail (when possible) with actionable >> messages. I think the rare case of not being able to reproduce the >> error locally if info logs are needed makes it OK to go and add >> logging to jenkins as a one-off. (If it's about jenkins build errors, >> perhaps we could build with higher verbosity before testing with a >> lower one.) >> On Thu, Dec 20, 2018 at 11:24 AM Maximilian Michels <[email protected]> >> wrote: >> > >> > Thanks Udi for bringing this up. I'm also for dropping INFO. It's just >> > incredible verbose. More importantly, from my experience the INFO level >> doesn't >> > help debugging problems, but it makes finding errors messages or >> warnings harder. >> > >> > That said, here's what I do to search through the log: >> > >> > 1) curl <build_url>/consoleText | less >> > >> > This is when I just want to quickly look for something. >> > >> > 2) curl <build_url>/consoleText > log.txt >> > less log.txt >> > >> > Here we store the log to a file first, then use 'less' or 'grep' to >> search it. >> > >> > When in 'less', I use '/' to grep through the lines. Pressing 'n' or >> 'N' gets >> > you forward and back in the search results. >> > >> > That works pretty well, but I think we would do us a favor by dropping >> the log >> > level. Shall we try it out? >> > >> > -Max >> > >> > On 19.12.18 23:27, Udi Meiri wrote: >> > > The gradle scan doesn't pinpoint the error message, and it doesn't >> contain all >> > > the lines: https://scans.gradle.com/s/ckhjrjdexpuzm/console-log >> > > >> > > The logs might be useful, but usually not from passing tests. Doesn't >> gradle log >> > > output from failed tests by default? >> > > >> > > On Wed, Dec 19, 2018 at 1:22 PM Thomas Weise <[email protected] >> > > <mailto:[email protected]>> wrote: >> > > >> > > I usually follow the download procedure outlined by Scott to look >> at the logs. >> > > >> > > These logs are big, but when there is a problem it is sometimes >> essential to >> > > have the extra output, especially for less frequent flakes. >> > > >> > > Reducing logs would then require the author to add extra logging >> to the PR >> > > (and attempt to reproduce), which is also not nice. >> > > >> > > Thomas >> > > >> > > >> > > On Wed, Dec 19, 2018 at 11:47 AM Scott Wegner <[email protected] >> > > <mailto:[email protected]>> wrote: >> > > >> > > I'm not sure what we lose by dropping the --info flag, but I >> generally >> > > worry about reducing log output since logs are the main >> resource for >> > > diagnosing Jenkins build errors. >> > > >> > > It seems the issue is that Chrome doesn't scale well to large >> log files. >> > > A few alternative solutions: >> > > >> > > 1. Use the produced Build Scan (example: [1]) instead of the >> raw console >> > > log. The build scan is quite useful at pointing to what >> actually failed, >> > > and filtering log output for only that task. >> > > 2. Instead of consoleFull, use consoleText ("View as plain >> text" link in >> > > Jenkins), which seems to be much easier on Chrome >> > > 3. Download the consoleText output locally and use your >> favorite log >> > > viewer that can scale to large files. >> > > >> > > [1] https://gradle.com/s/ckhjrjdexpuzm >> > > >> > > On Wed, Dec 19, 2018 at 10:42 AM Udi Meiri <[email protected] >> > > <mailto:[email protected]>> wrote: >> > > >> > > Hi all, >> > > I'd like to reduce precommit log sizes on Jenkins. For >> example: >> > > >> https://builds.apache.org/job/beam_PreCommit_Java_Commit/3181/consoleFull >> > > is 79M, which makes Chrome sluggish to use on it (tab is >> constantly >> > > using a whole cpu core). >> > > >> > > I know this might be controversial, but I'd like to >> propose to >> > > remove the --info flag from the gradlew command line. >> > > >> > > >> > > >> > > -- >> > > >> > > >> > > >> > > >> > > Got feedback? tinyurl.com/swegner-feedback >> > > <https://tinyurl.com/swegner-feedback> >> > > >> >
smime.p7s
Description: S/MIME Cryptographic Signature
