For PRs crave might be doing ok, but branch_9x check builds are all failing for a while now?
https://lists.apache.org/list?bui...@solr.apache.org:lte=1y:%22rsync%20error:%20some%20files/attrs%20were%20not%20transferred%22 Kevin Risden On Fri, Oct 6, 2023 at 3:38 PM Eric Pugh <ep...@opensourceconnections.com> wrote: > Agreed on the branch merging. It’s been great to have it running the full > set of tests! > > > > On Oct 5, 2023, at 10:58 PM, David Smiley <dsmi...@apache.org> wrote: > > > > I believe the Crave issues with branch merging seem to have been fixed. > If someone sees otherwise, please let me know. > > > > And boy Crave is fast! The whole GHA action takes 8m but Crave side is > 6m of which 4m of it is tests running. It's faster than "precommit" will > is still running in a standard GHA. Isn't that crazy! Yes, there's room > for improvement. > > > > There are opportunities for Crave to come up with a GHA self hosted > runner to substantially eat away at that 2m, like a needless checkout of > all the code on the GHA side that basically isn't used. > > > > There are opportunities for our project to try to optimize the Gradle > build so that it can start running tests (or whatever task) as soon as > possible no matter where it runs. There's a whole section to the Gradle > docs on build optimization. Maybe someone would like to explore that, like > trying the "configuration cache" > https://docs.gradle.org/current/userguide/configuration_cache.html > > > > I have access to build analytics in Crave that give some insights: The > first 48 seconds is not very concurrent and not downloading anything. The > next 36 seconds it downloads 100MB of something (don't know what). Then > CPUs go full tilt with tests. It's very apparent that Gradle testing has > no "work stealing" algorithm amongst the runners. > > > > > > > > I'm a bit perplexed at the downloading of 100MB because the image for > the build machine has commands I added to pre-download stuff. That looks > like the following: > > > > # Pre-download what we can through Gradle > > ./gradlew --write-verification-metadata sha256 --dry-run > > rm gradle/verification-metadata.dryrun.xml > > ./gradlew -p solr/solr-ref-guide downloadAntora > > ./gradlew -p solr/packaging downloadBats > > # May need more memory > > sed -i 's/-Xmx1g/-Xmx2g/g' gradle.properties > > # Use lots of CPUs > > sed -i 's/org.gradle.workers.max=.*/org.gradle.workers.max=96/' > gradle.properties > > sed -i 's/tests.jvms=.*/tests.jvms=96/' gradle.properties > > > > ./gradlew assemble || true > > > > ~ David Smiley > > Apache Lucene/Solr Search Developer > > http://www.linkedin.com/in/davidwsmiley > > _______________________ > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> | My Free/Busy < > http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. > >
