I also see this almost every other run. On Sun, Feb 17, 2019 at 8:24 AM Ryan Williams <r...@runsascoded.com> wrote:
> I still see this about 20% of the time I run `./gradlew compileTestJava > spotlessApply checkstyleMain checkstyleTest` (with 8-way parallelism), > which I've taken to running as a kind of "pre-PreCommit" (for Java). > > In total I'd estimate I've seen this error 20 times (across multiple > `./gradlew clean`s and spanning several weeks), but never twice in a row: I > always immediately re-run, and the error goes away. > > That seems consistent with the idea that gradle is racing the codegen task > against the compile task that requires the codegen's outputs. > > However, `./gradlew -p sdks/java/extensions/sql -m compileJava | grep > compileJavacc` implies that Gradle knows that compilation depends on the > codegen having run. > > Here are three successive invocations of `./gradlew compileTestJava > spotlessApply checkstyleMain checkstyleTest` > <https://gist.github.com/ryan-williams/6d2735b2484e9459be656abf379533c5>; > > - The first one succeeds (and includes output showing that the codegen > task, `:beam-sdks-java-extensions-sql:compileJavacc`, was run). > - Then, I rebased on one new upstream commit and ran again, and saw > the failure. > - Immediately re-running saw the failure go away. > > I guess in that output, tasks that are run but produce no stdout/stderr > are elided from the output, so we can't completely tell when the codegen > and compile tasks in question are being run. I'll start running with > --console=verbose > <https://stackoverflow.com/questions/45883963/gradle-4-0-does-not-display-executed-tasks-in-command-line#comment92914258_45889966>, > which I think will give a better sense of how this is happening. > > > On Fri, Feb 8, 2019 at 12:41 AM Ryan Williams <r...@runsascoded.com> > wrote: > >> After your last message, Alex, I saw the issue on a branch with minimal >> unrelated changes on top of b4b5495307 >> <https://github.com/apache/beam/commit/b4b549530730d9e0283ad0cdb203d384a107dfbf> >> (January >> 28). >> >> I ran `./gradlew clean` for the first time in a while at that time, and >> worked happily for 11 days, but just saw the issue again on a branch >> rebased on top of 381ab55b59 >> <https://github.com/apache/beam/commit/381ab55b59> (today): >> >> > Task :beam-sdks-java-extensions-sql:compileJava FAILED >> …/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29: >> error: cannot find symbol >> import >> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl; >> ^ >> symbol: class BeamSqlParserImpl >> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl >> 1 error >> >> Every time I've seen it, I immediately re-run the compile, and it >> succeeds. >> >> Perhaps something is still wrong with my environment, but otherwise it >> would seem that something is still flaky here. >> >> I'm compiling on an 8-core macOS machine, fwiw, and usually running >> `./gradlew compileTestJava` which compiles many projects concurrently. >> >> On Mon, Jan 28, 2019 at 4:12 PM Alex Amato <ajam...@google.com> wrote: >> >>> If it continues to occur, maybe it is an environmental issue, be sure to >>> try to clean as well. >>> ./gradlew clean >>> >>> On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <r...@runsascoded.com> >>> wrote: >>> >>>> Yea I was rebased on top of a more recent master than your previous >>>> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I >>>> see it again, thanks. >>>> >>>> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <ajam...@google.com> wrote: >>>> >>>>> After I did a rebase, it went away for me. So I think that this should >>>>> work. Are you saying that you did rebase ontop of master and it still >>>>> occurred? Strange. >>>>> >>>>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <r...@runsascoded.com> >>>>> wrote: >>>>> >>>>>> Hm, I just encountered this again on a branch that based on >>>>>> 5b46b02b49 (top of trunk from this afternoon). Is it definitely supposed >>>>>> to >>>>>> be fixed? >>>>>> >>>>>> >>>>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <ajam...@google.com> >>>>>> wrote: >>>>>> >>>>>>> Please try rebasing from master, I believe this issue has been >>>>>>> resolved. >>>>>>> >>>>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <r...@runsascoded.com> >>>>>>> wrote: >>>>>>> >>>>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due to >>>>>>>> this; re-running the commit has always succeeded, so far. >>>>>>>> >>>>>>>> Sounds like there is not an immediate fix in the works / no one >>>>>>>> assigned on the JIRA? >>>>>>>> >>>>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <k...@apache.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> This might connect to vendoring Calcite. It will be easiest, and >>>>>>>>> have the best incremental build, if we separate the generated code >>>>>>>>> into its >>>>>>>>> own module that has relocation to match the vendored Calcite. >>>>>>>>> >>>>>>>>> Kenn >>>>>>>>> >>>>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> We don't pre-generate the code as a separate step. Code gen from >>>>>>>>>> the SQL parser syntax spec and its compilation happens both during >>>>>>>>>> the Beam >>>>>>>>>> SQL build task. Splitting the code generation and compilation might >>>>>>>>>> not be >>>>>>>>>> trivial. We definitely should look into fixing this though. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Anton >>>>>>>>>> >>>>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <ajam...@google.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it >>>>>>>>>>> fails to generate the dep, rather than when compiling the java code >>>>>>>>>>> later on >>>>>>>>>>> >>>>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems >>>>>>>>>>>> that there's a race condition there and it sometimes fails. >>>>>>>>>>>> Rerunning the >>>>>>>>>>>> build works for me. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Anton >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <ajam...@google.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2 >>>>>>>>>>>>> >>>>>>>>>>>>> Any ideas, how this got through the precommit? >>>>>>>>>>>>> >>>>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED >>>>>>>>>>>>> >>>>>>>>>>>>> /usr/local/google/home/ajamato/go/src/ >>>>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29: >>>>>>>>>>>>> error: cannot find symbol >>>>>>>>>>>>> >>>>>>>>>>>>> import >>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl; >>>>>>>>>>>>> >>>>>>>>>>>>> ^ >>>>>>>>>>>>> >>>>>>>>>>>>> symbol: class BeamSqlParserImpl >>>>>>>>>>>>> >>>>>>>>>>>>> location: package >>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl >>>>>>>>>>>>> >>>>>>>>>>>>> 1 error >>>>>>>>>>>>> >>>>>>>>>>>>>
