nevzheng opened a new issue, #13642: URL: https://github.com/apache/gravitino/issues/13642
### What would you like to be improved? Gravitino CI rebuilds and retests everything that its cache misses, and today every build-file change makes the cache start cold. The ASF runs a Develocity instance ([develocity.apache.org](https://develocity.apache.org)) that offers build scans and a remote build cache. That cache would let CI write task outputs once and let everyone read them. This is related to some experiments I'm doing on modernizing the Gradle build (Gradle 9). I'll share those more broadly when they're ready. Knowing what ASF Develocity offers would help shape that work and the proposal. I'd like to explore using it, but I'm blocked on information. Infra's policy for the remote cache is visible only to committers. From public sources: - Build scans are available to ASF projects. Kafka, Lucene, Solr, Beam and Polaris publish them. - The remote cache needs per-project onboarding. Beam has its own node. Grails asked in INFRA-27642 and still gets 403 on writes. - An anonymous probe today: reads return 404 (allowed), writes return 403. ### How should we improve? First, get answers from Infra, with help from a committer or PMC member: 1. Shared instance with a scoped CI user, or a dedicated node? 2. Are anonymous reads allowed for contributors and forks? 3. Is there a quota or retention limit per project? 4. How is the write key given to GitHub Actions (a repository secret, used only on push and schedule events)? 5. Who files the INFRA ticket? Then, if the PMC agrees: - Publish build scans from CI, only when authenticated. - Add `gradle/actions/setup-gradle` (on the ASF approved list) for CI caching in the meantime. - Turn on the remote cache after onboarding: CI on `main` writes; pull requests, forks and laptops read. ### Appendix: background **Build scans.** A build scan is a shareable web report of one Gradle build. It records task timings, test results, failures, dependencies and the environment. Scans help us: - debug CI failures and flaky tests without re-running them locally; - see which tasks are slow and how CI time changes over time; - find out why a cache entry was missed (scans compare task inputs between two builds). **Remote build cache.** Gradle caches each task's output under a key computed from its inputs (sources, classpath, settings). A local cache already lets one machine reuse its own outputs. A remote cache lets machines share them: CI on `main` stores outputs, and any other build with the same inputs downloads them instead of recompiling or retesting. This helps by: - making CI faster on pull requests that touch only a few modules; - letting contributors skip rebuilding what CI already built; - lowering CI cost and queue time. **Why ASF Develocity.** The ASF provides it to its projects at no cost and runs it. Gravitino would need no new infrastructure. Only trusted CI writes, so cached outputs stay trustworthy. **Caveat.** A cache only helps if task inputs are stable across machines (no absolute paths, timestamps or unpinned downloads). Some cleanup in the build may be needed first to get good hit rates. cc @jerryshao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
