I believe I got to the bottom of this and fixed it. The Crave server initialization script, which I wrote (at least the project/Solr specific parts) was doing a "gradle assemble" at the end, and so there's lots of compiled code & various stuff in place. For "old PRs" or some scenarios, we need a clean build. I removed this and a PR #2084 for example finally passed. My objective was to make builds faster but I think this should be done with standard Gradle techniques like its build cache (which are retained on the build server because that's in the home dir).
~ David On Fri, Oct 6, 2023 at 8:05 PM Jason Gerlowski <gerlowsk...@gmail.com> wrote: > > branch_9x not being cleaned before the tests are run? > > That'd be my guess as well. 'CoresApi.java' is a SolrJ class that's > being generated from our v2 "OpenAPI Spec" (OAS). I merged a PR a few > days ago that adds the install-core-data API to our OAS (which causes > us to generate code for it in CoresApi.java)....but only in 'main'. > The code hasn't made it to branch_9x yet. > > I can see both a git-clean and gradle-clean happening at various > points in the Jenkins logs, which looks sufficient at a glance. But > evidently it's not somehow - I'm a bit stumped. > > I'm traveling at Community Over Code this week and won't have tons of > time to dig in. If no one else has any theories I can revert the > commit until I have time to dig in. Or alternatively, we can roll > forward and backport the 'main' commit to branch_9x and fix the > compilation issue that way (though that'll leave the underlying build > weirdness unsolved). > > Jason > > On Fri, Oct 6, 2023 at 4:45 PM Kevin Risden <kris...@apache.org> wrote: > > > > At least the last few are failing with: > > > > https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/5616/console > > > > ``` > > > > > Task :solr:solrj:compileJava > > > /tmp/src/solr/solr/solrj/build/generated/src/main/java/org/apache/solr/client/solrj/request/CoresApi.java:20: > > error: cannot find symbol > > import org.apache.solr.client.api.model.InstallCoreDataRequestBody; > > ^ > > symbol: class InstallCoreDataRequestBody > > location: package org.apache.solr.client.api.model > > > /tmp/src/solr/solr/solrj/build/generated/src/main/java/org/apache/solr/client/solrj/request/CoresApi.java:43: > > error: cannot find symbol > > private final InstallCoreDataRequestBody requestBody; > > ^ > > symbol: class InstallCoreDataRequestBody > > location: class InstallCoreData > > > /tmp/src/solr/solr/solrj/build/generated/src/main/java/org/apache/solr/client/solrj/request/CoresApi.java:57: > > error: cannot find symbol > > this.requestBody = new InstallCoreDataRequestBody(); > > ^ > > symbol: class InstallCoreDataRequestBody > > location: class InstallCoreData > > Note: Some input files use or override a deprecated API. > > Note: Recompile with -Xlint:deprecation for details. > > 3 errors > > > > ``` > > > > which I think is due to branch_9x not being cleaned before the tests are > > run? not 100% sure > > > > Kevin Risden > > > > > > On Fri, Oct 6, 2023 at 3:43 PM Kevin Risden <kris...@apache.org> wrote: > > > > > For PRs crave might be doing ok, but branch_9x check builds are all > > > failing for a while now? > > > > > > > > > > https://lists.apache.org/list?bui...@solr.apache.org:lte=1y:%22rsync%20error:%20some%20files/attrs%20were%20not%20transferred%22 > > > > > > Kevin Risden > > > > > > > > > On Fri, Oct 6, 2023 at 3:38 PM Eric Pugh < > ep...@opensourceconnections.com> > > > wrote: > > > > > >> Agreed on the branch merging. It’s been great to have it running the > > >> full set of tests! > > >> > > >> > > >> > On Oct 5, 2023, at 10:58 PM, David Smiley <dsmi...@apache.org> > wrote: > > >> > > > >> > I believe the Crave issues with branch merging seem to have been > > >> fixed. If someone sees otherwise, please let me know. > > >> > > > >> > And boy Crave is fast! The whole GHA action takes 8m but Crave > side is > > >> 6m of which 4m of it is tests running. It's faster than "precommit" > will > > >> is still running in a standard GHA. Isn't that crazy! Yes, there's > room > > >> for improvement. > > >> > > > >> > There are opportunities for Crave to come up with a GHA self hosted > > >> runner to substantially eat away at that 2m, like a needless checkout > of > > >> all the code on the GHA side that basically isn't used. > > >> > > > >> > There are opportunities for our project to try to optimize the > Gradle > > >> build so that it can start running tests (or whatever task) as soon as > > >> possible no matter where it runs. There's a whole section to the > Gradle > > >> docs on build optimization. Maybe someone would like to explore > that, like > > >> trying the "configuration cache" > > >> https://docs.gradle.org/current/userguide/configuration_cache.html > > >> > > > >> > I have access to build analytics in Crave that give some insights: > The > > >> first 48 seconds is not very concurrent and not downloading > anything. The > > >> next 36 seconds it downloads 100MB of something (don't know what). > Then > > >> CPUs go full tilt with tests. It's very apparent that Gradle testing > has > > >> no "work stealing" algorithm amongst the runners. > > >> > > > >> > > > >> > > > >> > I'm a bit perplexed at the downloading of 100MB because the image > for > > >> the build machine has commands I added to pre-download stuff. That > looks > > >> like the following: > > >> > > > >> > # Pre-download what we can through Gradle > > >> > ./gradlew --write-verification-metadata sha256 --dry-run > > >> > rm gradle/verification-metadata.dryrun.xml > > >> > ./gradlew -p solr/solr-ref-guide downloadAntora > > >> > ./gradlew -p solr/packaging downloadBats > > >> > # May need more memory > > >> > sed -i 's/-Xmx1g/-Xmx2g/g' gradle.properties > > >> > # Use lots of CPUs > > >> > sed -i 's/org.gradle.workers.max=.*/org.gradle.workers.max=96/' > > >> gradle.properties > > >> > sed -i 's/tests.jvms=.*/tests.jvms=96/' gradle.properties > > >> > > > >> > ./gradlew assemble || true > > >> > > > >> > ~ David Smiley > > >> > Apache Lucene/Solr Search Developer > > >> > http://www.linkedin.com/in/davidwsmiley > > >> > > >> _______________________ > > >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | > 434.466.1467 | > > >> http://www.opensourceconnections.com < > > >> http://www.opensourceconnections.com/> | My Free/Busy < > > >> http://tinyurl.com/eric-cal> > > >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > > >> > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > > > > >> > > >> This e-mail and all contents, including attachments, is considered to > be > > >> Company Confidential unless explicitly stated otherwise, regardless of > > >> whether attachments are marked as such. > > >> > > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > For additional commands, e-mail: dev-h...@solr.apache.org > >
