Hi Wes, I fielded ARROW-11140 <https://issues.apache.org/jira/browse/ARROW-11140> with what I would consider a successful POC from Rust implementation's point of view. Let's move the discussion there. I was already able to run Rust's first steps there (using my own mac as a runner), and, so far, looks pretty good I must say. Would it be possible to add a linux agent (amd64) on it? I would expect https://github.com/apache/arrow/pull/9111 to be triggered.
Best, Jorge On Tue, Jan 5, 2021 at 11:33 PM Wes McKinney <wesmck...@gmail.com> wrote: > hi Jorge -- as far as I know, all that's necessary to use BK is the > pipeline configuration in the repo and to attach workers to Arrow's > (free) account (which I'm happy to add you to). We can likely get a > lot of free cloud credits from Google, Microsoft, or Amazon if we > wanted to go that route to keep costs down (I could help pay for the > cloud bill, too). > > The one thing to be aware of with BK is the possible security risks > from builds running on pull requests. With reasonable security > practices (not putting sensitive data on the build hosts) this should > be too much of a problem. > > On Tue, Jan 5, 2021 at 11:46 AM Jorge Cardoso Leitão > <jorgecarlei...@gmail.com> wrote: > > > > Krisztián, > > > > I agree with you that there is an ongoing problem with the queue. Thanks > > for the tips wrt to WIP. > > > > Wes, I would be up for moving Rust workflows to Buildkite. Is the > > integration in place wrt to reporting to triggers and reporting back to > > gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is there > > any PMC-specific activity that is blocking us from working on this? > > > > Best, > > Jorge > > > > > > > > > > On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > At the risk of sounding like a broken record -- we are almost > > > certainly going to have to move our builds to dedicated infrastructure > > > that this community has complete agency over sometime between now and > > > 2025. Maybe it will be this year, maybe next year, but to me it is an > > > inevitability. I spent quite a bit of time getting Apache Infra to > > > enable Buildkite for apache/* repositories precisely so that we have a > > > coherent path. I hope to be able to have folks on my team focus on > > > this at some point and contribute it to the community once we have the > > > bandwidth. > > > > > > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs > > > <szucs.kriszt...@gmail.com> wrote: > > > > > > > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <jeroeno...@gmail.com> > wrote: > > > > > > > > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs > > > > > <szucs.kriszt...@gmail.com> wrote: > > > > > > > > > > > > Hi, > > > > > > > > > > > > I'm concerned about the overall feedback time we have on pull > > > requests. > > > > > > I have a simple PR to make the comment bot working again, but no > > > > > > builds are running even after 30 minutes. > > > > > > > > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the > > > > > same problem sometimes with pull requests taking long to build, but > > > > > this usually happens when there are large jobs running under my > name > > > > > in completely unrelated projects. Also note that if you use e.g. > > > > > scheduled (cron) actions, then the jobs are executed in the queue > of > > > > > the github user who last touched the yml file for that workflow. > > > > Based on the "Actions" tab the queueing seems chronological to me, > > > > although I can imagine the behaviour you described. > > > > > > > > Has anyone experienced similar clogging on apache/arrow's gha queue > > > lately? > > > >
