Note that 20 builds are running on master only, including windows, macos, and documentation builds.
On Mon, Nov 11, 2019, 12:33 PM Antoine Pitrou <anto...@python.org> wrote: > > Ah, no, sorry, I read wrongly. It's 20 builds total. That's still a > lot, IMO. > > Regards > > Antoine. > > > Le 11/11/2019 à 12:32, Antoine Pitrou a écrit : > > > > A C++ change will end up triggering 38 builds (C++, Python, Ruby, R)... > > > > Regards > > > > Antoine. > > > > > > Le 11/11/2019 à 12:25, Wes McKinney a écrit : > >> That's too many for C++ and Python changes in my opinion. Let's focus > >> on getting the PR merged and then we can address this problem > >> > >> On Mon, Nov 11, 2019 at 3:21 AM Krisztián Szűcs > >> <szucs.kriszt...@gmail.com> wrote: > >>> > >>> It depends on the change: > >>> > >>> - [C++]: 20 > >>> > https://github.com/kszucs/arrow/commit/5654da07c21a7cf6a0e08894f4bf4e7aaa70ed26/checks?check_suite_id=304713473 > >>> - [Python]: 8 > >>> > https://github.com/kszucs/arrow/commit/5084c23f4d8b0fe05f1056808e7066aadbf0033e/checks?check_suite_id=304716138 > >>> - [Ruby]: 5 > >>> > https://github.com/kszucs/arrow/commit/e90e92514f69b0734e8126a949858476bfb70c1a/checks?check_suite_id=304717369 > >>> - [JS]: 6 > >>> > https://github.com/kszucs/arrow/commit/f27d6500f628c152c732f2372beae15a34f088e9/checks?check_suite_id=304718837 > >>> - [R]: 5 > >>> > https://github.com/kszucs/arrow/commit/eaec66cbde923687beebe31114dea6d0410043c6/checks?check_suite_id=304719993 > >>> - [Rust]: 6 > >>> > https://github.com/kszucs/arrow/commit/0c52ad15a54e78fdd1fe797668ebf0cdfbf6f4ab/checks?check_suite_id=304720884 > >>> - [Go]: 7 > >>> > https://github.com/kszucs/arrow/commit/30ab7f5e8cf6373774ce6ea75b0473da5ec808a3/checks?check_suite_id=304722182 > >>> > >>> > >>> On Sat, Nov 9, 2019 at 2:19 AM Wes McKinney <wesmck...@gmail.com> > wrote: > >>>> > >>>> Just to be sure, if this PR is merged, how many GHA tasks will be run > >>>> on each commit to master? > >>>> > >>>> On Fri, Nov 8, 2019 at 12:07 PM Krisztián Szűcs > >>>> <szucs.kriszt...@gmail.com> wrote: > >>>>> > >>>>> I've trimmed down the number of triggered builds on pull requests by > >>>>> converting them to run on master only or cron builds. Alsoe added > >>>>> the action filters including the changed path patterns. I've also > >>>>> collected ~30 follow up JIRAs aggregating the problems I came across > >>>>> during the refactor and possible further improvements and > optimizations > >>>>> like caching. > >>>>> > >>>>> The PR is ready to be merged [1]. I expect multiple issues after > merging > >>>>> the PR, despite that I was trying to port everything thoroughly. > >>>>> The changeset is simply too big, but we need to start somewhere, and > >>>>> sooner is better. > >>>>> > >>>>> [1]: https://github.com/apache/arrow/pull/5589 > >>>>> > >>>>> On Thu, Oct 31, 2019 at 10:21 PM Wes McKinney <wesmck...@gmail.com> > wrote: > >>>>>> > >>>>>> hi Krisz -- I just left comments on the PR. This definitely looks > like > >>>>>> a good step forward. My main comment is that I think there are too > >>>>>> many C++/Python tasks to run on an each-commit basis. Ideally many > of > >>>>>> these would be run nightly. There is also a certain amount of > >>>>>> redundancy in rebuilding the C++ library multiple times before > running > >>>>>> each dependent set of tests, whereas in Travis we build the C++ > >>>>>> library once then test both C++ and Python. If there is sufficient > >>>>>> number of builders then perhaps it doesn't matter so much > >>>>>> > >>>>>> It seems there are a few things, like action filtering (similar to > >>>>>> "detect-changes.py") based on what was changed that would need to > get > >>>>>> done before this can be merged. > >>>>>> > >>>>>> - Wes > >>>>>> > >>>>>> On Fri, Oct 25, 2019 at 7:25 PM Krisztián Szűcs > >>>>>> <szucs.kriszt...@gmail.com> wrote: > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> During the release of 0.15.1-RC0 I literally had to wait days > >>>>>>> to ensure that the Travis, Appveyor and Crossbow builds > >>>>>>> were all passing for the release branch. Additionally each > >>>>>>> newly added patch was delaying the process by 8 hrs or so > >>>>>>> (actually felt like 16). > >>>>>>> > >>>>>>> Recently I've been working on to incorporate the advantages > >>>>>>> of the Buildbot setup into our current docker-compose > >>>>>>> configuration, including support for multiple architectures > >>>>>>> and platforms, reusing docker images and caching dependency > >>>>>>> installation steps. It tries to follow the semantics of ursabot, > >>>>>>> but using only docker-compose and tiny shell scripts. > >>>>>>> > >>>>>>> This refactoring also includes GitHub Actions workflows for > >>>>>>> Windows and macOS as well, reusing the same (bash) builds > >>>>>>> scripts. The docker configuration and the scripts are CI agnostic. > >>>>>>> Last but not least, I've managed to clean up a lot of things > >>>>>>> including every travis builds, and three Appveyor builds. > >>>>>>> As an example the ci [3] and dev [4] folders got much cleaner. > >>>>>>> > >>>>>>> The majority of the builds are passing [2], but due to the size > >>>>>>> of the pull request [1] reviews for relevant workflows like the > >>>>>>> JavaScript, C#, Rust, JNI, etc. would be much appreciated. > >>>>>>> I'll be on vacation until Wednesday, but will try to respond on > >>>>>>> both GH and the ML. > >>>>>>> > >>>>>>> Thanks, Krisztian > >>>>>>> > >>>>>>> [1]: https://github.com/apache/arrow/pull/5589 > >>>>>>> [2]: https://github.com/apache/arrow/runs/275685241 > >>>>>>> [3]: > https://github.com/apache/arrow/tree/9c7e7289b9c9486c13a02e7cb5682a0f9f274ec6/ci > >>>>>>> [4]: > https://github.com/apache/arrow/tree/9c7e7289b9c9486c13a02e7cb5682a0f9f274ec6/dev >
