Upgrading dependencies is an important topic thanks for starting the discussion Hongyu.
In terms of tooling, my experience with Dependabot in Apache Hive is rather negative. Out of 52 PRs [1] raised by the bot 34 [2] are failing the build/tests. In most cases committers do not follow-up to resolve the failures and currently there are only 3 [3] that are merged to master. In Hive, the presence of the bot adds noise and wastes resources. Calcite is a simpler project so maybe Dependabot does a better job but unless there are people who are willing to track and follow up on those PRs the result may be similar. Best, Stamatis [1] https://github.com/apache/hive/pulls?q=is%3Apr+author%3Aapp%2Fdependabot+ [2] https://github.com/apache/hive/pulls?q=is%3Apr+author%3Aapp%2Fdependabot+label%3A%22tests+unstable%22%2C%22tests+failed%22 [3] https://github.com/apache/hive/commits?author=dependabot%5Bbot%5D On Sun, Nov 5, 2023 at 10:03 PM Francis Chuang <[email protected]> wrote: > > Perhaps refreshVersions [1] can be used. > > [1] https://splitties.github.io/refreshVersions/ > > On 6/11/2023 7:54 am, Julian Hyde wrote: > > I agree that we should be trying to stay on the most recent version of > > our dependencies (with a few exceptions, such as JavaCC). Most of our > > dependencies are mature libraries, and the latest version is more > > likely to fix security problems than to introduce bugs. > > > > However, I'm not sure that Dependabot is the best way to do it. One, > > dependabot generates quite a lot of noise (frequent upgrades). Two, we > > would have to restructure our build files. The best process is > > probably to do manual upgrades, like > > https://github.com/apache/calcite/pull/3504, just before each release. > > Is there a straightforward way to script those upgrades? > > > > Julian > > > > > > On Sun, Nov 5, 2023 at 1:09 AM Jiajun Xie <[email protected]> wrote: > >> > >> Hi, Hongyu. > >> > >> Your idea is great and you also introduced the steps to use it. > >> > >> We need more feedback about benefits and risks from calcite users. > >> # What are the benefits? > >> - Quickly fix dependency vulnerabilities. > >> - Balancing the workload of each upgrade(Not 4.0 to 7.x). > >> - ... > >> > >> # What are the risks? > >> - The latest version may be unstable. > >> - The burden of upgrading Calcite for users has increased. > >> - ... > >> > >> For me, the risks are acceptable. > >> I am willing to help you complete this work. > >> > >> On Sat, 4 Nov 2023 at 21:04, Hongyu Guo <[email protected]> wrote: > >> > >>> Hi all, > >>> > >>> Recently, I opened 2 PRs about removing an unused library[1] and bumping > >>> various libraries[2]. I noticed that many dependencies of calcite are > >>> outdated. To address this issue, I suggest enabling dependabot[3] to > >>> automatically open "bump dependency" PRs and make calcite healthier. > >>> > >>> If we enable dependabot, what should we do? > >>> > >>> - Add `dependabot.yml` to `.github/`. It is straightforward, just follow > >>> the instructions in the documentation[4]. > >>> - Refactor gradle project files: Dependabot's support for gradle is not > >>> sufficient as it only reads the text of `build.gradle.kts`, > >>> `build.gradle`, > >>> and `settings.gradle.kts` instead of running gradle. Additionally, > >>> dependabot can NOT read `gradle.properties`, so we need to refactor the > >>> gradle project files. > >>> - Ignore some dependencies: Some dependencies cannot be upgraded. For > >>> example, I attempted to bump javacc from 4.0 to 7.x, but due to > >>> incompatibility caused by the large version span, I had to give up. Also, > >>> we cannot upgrade elasticsearch due to licensing restrictions. > >>> > >>> What is your opinion on dependabot? > >>> > >>> [1]https://github.com/apache/calcite/pull/3502 > >>> [2]https://github.com/apache/calcite/pull/3504 > >>> [3] > >>> > >>> https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/about-dependabot-version-updates > >>> [4] > >>> > >>> https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#gradle > >>> > >>> Best, > >>> Hongyu > >>>
