As you may have noticed, the CI is slow again.
There are more than 140 workflows pending:
https://github.com/apache/pulsar/actions?query=is%3Aqueued
There are only 2-3 workflows in progress:
https://github.com/apache/pulsar/actions?query=is%3Ain_progress
Lari and I believe that we're still penaliz
Fantastic, thank you Lari and Nicolò!
- Michael
On Thu, Sep 8, 2022 at 9:03 PM Haiting Jiang wrote:
>
> Great work. Thank you, Lari and Nicolò.
>
> BR,
> Haiting
>
> On Fri, Sep 9, 2022 at 9:36 AM tison wrote:
> >
> > Thank you, Lari and Nicolò!
> > Best,
> > tison.
> >
> >
> > Nicolò Boschi 于
Great work. Thank you, Lari and Nicolò.
BR,
Haiting
On Fri, Sep 9, 2022 at 9:36 AM tison wrote:
>
> Thank you, Lari and Nicolò!
> Best,
> tison.
>
>
> Nicolò Boschi 于2022年9月9日周五 02:41写道：
>
> > Dear community,
> >
> > The plan has been executed.
> > The summary of our actions is:
> > 1. We cance
Thank you, Lari and Nicolò!
Best,
tison.
Nicolò Boschi 于2022年9月9日周五 02:41写道：
> Dear community,
>
> The plan has been executed.
> The summary of our actions is:
> 1. We cancelled all pending jobs (queue and in-progress)
> 2. We removed the required checks to be able to merge improvements on the
Dear community,
The plan has been executed.
The summary of our actions is:
1. We cancelled all pending jobs (queue and in-progress)
2. We removed the required checks to be able to merge improvements on the
CI workflow
3. We merged a couple of improvements:
1. workarounded the possible bug trigg
Thank you Nicolo.
There's lazy consensus, let's go forward with the action plan.
-Lari
On 2022/09/08 08:16:05 Nicolò Boschi wrote:
> This is the pull for step 2. https://github.com/apache/pulsar/pull/17539
>
> This is the script I'm going to use to cancel pending workflows.
> https://github.com/
This is the pull for step 2. https://github.com/apache/pulsar/pull/17539
This is the script I'm going to use to cancel pending workflows.
https://github.com/nicoloboschi/pulsar-validation-tool/blob/master/pulsar-scripts/pulsar-gha/cancel-workflows.js
I'm going to run the script in minutes.
I adv
Thanks Lari for the detailed explanation. This is kind of an emergency
situation and I believe your plan is the way to go now.
I already prepared a pull for moving the flaky suite out of the Pulsar CI
workflow: https://github.com/nicoloboschi/pulsar/pull/8
I can take care of the execution of the p
If my assumption of the GitHub usage metrics bug in the GitHub Actions build
job queue fairness algorithm is correct, what would help is running the flaky
unit test group outside of Pulsar CI workflow. In that case, the impact of the
usage metrics would be limited.
The example of https://github
My current assumption of the CI slowness problem is that the usage metrics for
Apache Pulsar builds on GitHub side is done incorrectly and that is resulting
in apache/pulsar builds getting throttled. This assumption might be wrong, but
it's the best guess at the moment.
The facts that support t
On 2022/09/07 17:27:45 tison wrote:
> Today Pulsar repo runs almost up to one worflow run at the same time. It's
> a new situation I didn't notice before.
>
> > drop the "required checks"
>
> This can be dangerous to the repo status. I think the essential problem we
> meet here is about prioritiz
Here is another patch that can reduce unnecessary workload:
https://github.com/apache/pulsar/pull/17529
We don't create flaky-test issues/PRs frequently; it's about tens in one
month. The project owner should be able to handle it manually in minutes
per month (since candidates are already labeled)
Today Pulsar repo runs almost up to one worflow run at the same time. It's
a new situation I didn't notice before.
> drop the "required checks"
This can be dangerous to the repo status. I think the essential problem we
meet here is about prioritizing specific PR, instead of releasing the guard
to
On 2022/09/07 16:59:33 tison wrote:
> > selecting which jobs to process
>
> Do you have a patch to implement this? IIRC it requires interacting with
> outside service or at least we may add an ok-to-test label.
Very good idea, I didn't think that far ahead. It seems that Apache Spark has
some so
One possible way forward:
1. Cancel all existing builds in_progress or queued
2. Edit .asf.yaml and drop the "required checks" requirement for merging PRs.
3. Wait for build to run for .asf.yaml change, merge it
4. Disable all workflows
5. Process specific PRs manually to improve the situation.
> selecting which jobs to process
Do you have a patch to implement this? IIRC it requires interacting with
outside service or at least we may add an ok-to-test label.
Besides, it increases committers/PMC members' workload - be aware of it, or
most of contributions will stall.
Best,
tison.
Lari
The problem with CI is becoming worse. The build queue is 235 jobs now and the
queue time is over 7 hours.
We will need to start shedding load in the build queue and get some fixes in.
https://issues.apache.org/jira/browse/INFRA-23633 continues to contain details
about some activities. I have cr
Thanks Lari!
Does this issue cause the tests for PRs like
https://github.com/apache/pulsar/pull/17198 to be hang?
On 2022/09/06 14:41:07 Dave Fisher wrote:
> We are going to need to take actions to fix our problems. See
> https://issues.apache.org/jira/browse/INFRA-23633?focusedCommentId=176007
We are going to need to take actions to fix our problems. See
https://issues.apache.org/jira/browse/INFRA-23633?focusedCommentId=17600749&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17600749
Jarek has done a large amount of GitHub Action work with Apache Airflo
The Apache Infra ticket is https://issues.apache.org/jira/browse/INFRA-23633 .
-Lari
On 2022/09/06 11:36:46 Lari Hotari wrote:
> I asked for an update on the Apache org GitHub Actions usage stats from Gavin
> McDonald on the-asf slack in this thread:
> https://the-asf.slack.com/archives/CBX4TS
I asked for an update on the Apache org GitHub Actions usage stats from Gavin
McDonald on the-asf slack in this thread:
https://the-asf.slack.com/archives/CBX4TSBQ8/p1662464113873539?thread_ts=1661512133.913279&cid=CBX4TSBQ8
.
I hope we get this issue resolved since it delays PR processing a lo
Pulsar CI continues to be congested, and the build queue [1] is very long at
the moment. There are 147 build jobs in the queue and 16 jobs in progress at
the moment.
I would strongly advice everyone to use "personal CI" to mitigate the issue of
the long delay of CI feedback. You can simply open
Lari,
Il giorno mar 30 ago 2022 alle ore 14:39 Lari Hotari
ha scritto:
>
> Pulsar CI continues to be congested, and the build queue is long.
>
> I would strongly advice everyone to use "personal CI" to mitigate the issue
> of the long delay of CI feedback. You can simply open a PR to your own
>
Pulsar CI continues to be congested, and the build queue is long.
I would strongly advice everyone to use "personal CI" to mitigate the issue of
the long delay of CI feedback. You can simply open a PR to your own personal
fork of apache/pulsar to run the builds in your "personal CI". There's mor
master branch is broken once again. Here's the fix:
https://github.com/apache/pulsar/pull/17339
Please review and merge
-Lari
On 2022/08/26 12:00:20 Lari Hotari wrote:
> Hi,
>
> GitHub Actions builds have been piling up in the build queue in the last few
> days.
> I posted on bui...@apache.org
And +1 vote for "Provide information about GitHub Actions usage for apache
organization", which would be greatly helpful by making these information
transparency.
Best,
Max Xu
On Fri, Aug 26, 2022 at 8:00 PM Lari Hotari wrote:
> Hi,
>
> GitHub Actions builds have been piling up in the build qu
Hi, Lari
Thanks for bringing this to our attention!
I was wondering if we could consider using the self-hosted runner? As there
are currently more than 2k projects in one apache org.
Best,
Max Xu
On Fri, Aug 26, 2022 at 8:00 PM Lari Hotari wrote:
> Hi,
>
> GitHub Actions builds have been pi
Hi,
GitHub Actions builds have been piling up in the build queue in the last few
days.
I posted on bui...@apache.org
https://lists.apache.org/thread/6lbqr0f6mqt9s8ggollp5kj2nv7rlo9s and created
INFRA ticket https://issues.apache.org/jira/browse/INFRA-23633 about this issue.
There's also a threa
28 matches
Mail list logo
