A small contribution in this direction. We were generating the TPC-DS dataset for every benchmark run. Avoiding this when it isn’t needed would save about 10 minutes per run. https://github.com/apache/spark/pull/5647
On Fri, Jun 12, 2026 at 9:54 PM Tian Gao via dev <[email protected]> wrote: > Hi, > > According to the infra report, Spark used 202,846 minutes in the past 7 > days, which is already below Apache's 250k limit. We reduced usage from > 350k/week. Does the current state comply with Apache's restriction now? > > Tian > > On Fri, Jun 12, 2026 at 7:29 AM Allen Geer <[email protected]> wrote: > >> The speed of adoption of CI pipelines with new agentic tools will >> literally increase demand on your servers exponentially in the next month. >> Asking folks to tone it down, wont be sufficient. I Suggest a different >> strategy. >> >> -A G. >> >> > On Jun 12, 2026, at 7:57 AM, Robert Thomson <[email protected]> >> wrote: >> > >> > Hi, >> > >> > In 2024, the ASF introduced the policy for GitHub Actions usage across >> > the foundation[1]. As you have likely experienced, the ASF Github >> > shared pool of Github-hosted runners has been at, or very close to the >> > limit of 900 jobs most of the time in the past few weeks, causing jobs >> > to queue for lengthy periods before running. >> > >> > Your project has again been identified as being among the top consumers >> of >> > build time over the past 7 days and we request that you make efforts >> > to bring your usage down by stream-lining long-running builds and >> > minimising the number of concurrent builds. >> > >> > You can use the infra reporting tool [2] to monitor your GHA usage as >> you >> > work on stream-lining and to locate any bottlenecks in the workflows. >> > >> > There is now on-going community discussion on a Slack channel, >> > “project-workflow-optimisations” about the topic of optimising GitHub >> > Actions workflows so as to minimise both the number of concurrent jobs >> > running on our limited allocation, and the number of long-running >> > jobs. >> > >> > To share best practices between projects on this area we encourage >> > projects to contribute to this cwiki page: >> > >> > >> https://cwiki.apache.org/confluence/display/INFRA/GitHub+Actions+Recommended+Practices >> > >> > It is also possible for a project to maintain their own self-hosted >> > runners if they can provide a suitable VM/on-prem hardware to execute >> > the runner: >> > >> > >> https://cwiki.apache.org/confluence/display/INFRA/GitHub+self-hosted+runners >> > >> > Kind regards, >> > Bob Thomson, on behalf of ASF Infrastructure. >> > >> > >> > [1] https://infra.apache.org/github-actions-policy.html >> > [2] https://infra-reports.apache.org/#ghactions >> > >> > --------------------------------------------------------------------- >> > To unsubscribe e-mail: [email protected] >> > >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: [email protected] >> >>
