Re: [DISCUSSION] Cassandra's code style and source code analysis
Hello everyone, Some updates. There are issues that we have put on hold, waiting for the CEPs to be finalized. The java imports are one of these issues, let's do not forget them ^^ I've created a label to track it: https://issues.apache.org/jira/issues/?jql=labels%20%3D%20code-polishing On Tue, 1 Aug 2023 at 10:46, Miklosovic, Stefan wrote: > > I think we might wait for Accord and transactional metadata as the last big > contributions in 5.0 (if I have not forgotten something) and then we can just > polish it all just before the release. There will be still some room to do > the housekeeping like this after these patches lend. It is not like Accord > will be in trunk on Monday and we release Tuesday ... > > > From: Maxim Muzafarov > Sent: Monday, July 31, 2023 23:05 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > > > Hello everyone, > > > It's been a long time since the last discussion about the import order > code style, so I want to give these changes a chance as all the major > JIRA issues have already landed on the release branch so we won't > affect anyone. I'd be happy to find any reviewers who are interested > in helping with the next steps :-) I've updated the changes to reflect > the latest checkstyle work, so here they are: > > https://issues.apache.org/jira/browse/CASSANDRA-17925 > https://github.com/apache/cassandra/pull/2108 > > > The changes look scary at first glance, but they're actually quite > simple and in line with what we've discussed above. In short, we can > divide all the affected files into two parts: the update of the code > style configuration files (checkstyle + IDE configs), and the update > of all the sources to match the code style. > > In short: > > - "import order" hotkey will work regardless of which IDE you are using; > - updated checkstyle configuration, and IDEA, Eclipse, NetBeans > configurations have been updated; > - AvoidStarImport checkstyle rule applied as well; > > The import order we've agreed upon: > > java.* > [blank line] > javax.* > [blank line] > com.* > [blank line] > net.* > [blank line] > org.* > [blank line] > org.apache.cassandra.* > [blank line] > all other imports > [blank line] > static all other imports > > On Mon, 27 Feb 2023 at 13:26, Maxim Muzafarov wrote: > > > > > I suppose it can be easy for the existing feature branches if they have a > > > single commit. Don't we need to adjust each commit for multi-commit > > > feature branches? > > > > It depends on how feature branches are maintained and developed, I > > guess. My thoughts here are that the IDE's hotkeys should just work to > > resolve any code-style issues that arise during rebase/maintenance. > > I'm not talking about enforcing all our code-style rules but giving > > developers good flexibility. The classes import order rule might be a > > good example here. > > > > On Wed, 22 Feb 2023 at 21:27, Jacek Lewandowski > > wrote: > > > > > > I suppose it can be easy for the existing feature branches if they have a > > > single commit. Don't we need to adjust each commit for multi-commit > > > feature branches? > > > > > > śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov > > > napisał: > > >> > > >> Hello everyone, > > >> > > >> I have created an issue CASSANDRA-18277 that may help us move forward > > >> with code style changes. It only affects the way we store the IntelliJ > > >> code style configuration and has no effect on any current (or any) > > >> releases, so it should be safe to merge. So, once the issue is > > >> resolved, every developer that checkouts a release branch will use the > > >> same code style stored in that branch. This in turn makes rebasing a > > >> big change like the import order [1] a really straightforward matter > > >> (by pressing Crtl + Opt + O in their local branch to organize > > >> imports). > > >> > > >> See: > > >> > > >> Move the IntelliJ Idea code style and inspections configuration to the > > >> project's root .idea directory > > >> https://issues.apache.org/jira/browse/CASSANDRA-18277 > > >> > > >> > > >> > > >> [1] https://issues.apache.org/jira/
Re: [DISCUSSION] Cassandra's code style and source code analysis
I think we might wait for Accord and transactional metadata as the last big contributions in 5.0 (if I have not forgotten something) and then we can just polish it all just before the release. There will be still some room to do the housekeeping like this after these patches lend. It is not like Accord will be in trunk on Monday and we release Tuesday ... From: Maxim Muzafarov Sent: Monday, July 31, 2023 23:05 To: dev@cassandra.apache.org Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hello everyone, It's been a long time since the last discussion about the import order code style, so I want to give these changes a chance as all the major JIRA issues have already landed on the release branch so we won't affect anyone. I'd be happy to find any reviewers who are interested in helping with the next steps :-) I've updated the changes to reflect the latest checkstyle work, so here they are: https://issues.apache.org/jira/browse/CASSANDRA-17925 https://github.com/apache/cassandra/pull/2108 The changes look scary at first glance, but they're actually quite simple and in line with what we've discussed above. In short, we can divide all the affected files into two parts: the update of the code style configuration files (checkstyle + IDE configs), and the update of all the sources to match the code style. In short: - "import order" hotkey will work regardless of which IDE you are using; - updated checkstyle configuration, and IDEA, Eclipse, NetBeans configurations have been updated; - AvoidStarImport checkstyle rule applied as well; The import order we've agreed upon: java.* [blank line] javax.* [blank line] com.* [blank line] net.* [blank line] org.* [blank line] org.apache.cassandra.* [blank line] all other imports [blank line] static all other imports On Mon, 27 Feb 2023 at 13:26, Maxim Muzafarov wrote: > > > I suppose it can be easy for the existing feature branches if they have a > > single commit. Don't we need to adjust each commit for multi-commit feature > > branches? > > It depends on how feature branches are maintained and developed, I > guess. My thoughts here are that the IDE's hotkeys should just work to > resolve any code-style issues that arise during rebase/maintenance. > I'm not talking about enforcing all our code-style rules but giving > developers good flexibility. The classes import order rule might be a > good example here. > > On Wed, 22 Feb 2023 at 21:27, Jacek Lewandowski > wrote: > > > > I suppose it can be easy for the existing feature branches if they have a > > single commit. Don't we need to adjust each commit for multi-commit feature > > branches? > > > > śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov > > napisał: > >> > >> Hello everyone, > >> > >> I have created an issue CASSANDRA-18277 that may help us move forward > >> with code style changes. It only affects the way we store the IntelliJ > >> code style configuration and has no effect on any current (or any) > >> releases, so it should be safe to merge. So, once the issue is > >> resolved, every developer that checkouts a release branch will use the > >> same code style stored in that branch. This in turn makes rebasing a > >> big change like the import order [1] a really straightforward matter > >> (by pressing Crtl + Opt + O in their local branch to organize > >> imports). > >> > >> See: > >> > >> Move the IntelliJ Idea code style and inspections configuration to the > >> project's root .idea directory > >> https://issues.apache.org/jira/browse/CASSANDRA-18277 > >> > >> > >> > >> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925 > >> > >> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan > >> wrote: > >> > > >> > Thank you Maxim for doing this. > >> > > >> > It is nice to see this effort materialized in a PR. > >> > > >> > I would wait until bigger chunks of work are committed to trunk (like > >> > CEP-15) to not collide too much. I would say we can postpone doing this > >> > until the actual 5.0 release, last weeks before it so we would not clash > >> > with any work people would like to include in 5.0. This can go in > >> > anytime, basically. > >> > > >> > Are people on the same page? > >> > > >> > Regards > >> > > >> >
Re: [DISCUSSION] Cassandra's code style and source code analysis
Hello everyone, It's been a long time since the last discussion about the import order code style, so I want to give these changes a chance as all the major JIRA issues have already landed on the release branch so we won't affect anyone. I'd be happy to find any reviewers who are interested in helping with the next steps :-) I've updated the changes to reflect the latest checkstyle work, so here they are: https://issues.apache.org/jira/browse/CASSANDRA-17925 https://github.com/apache/cassandra/pull/2108 The changes look scary at first glance, but they're actually quite simple and in line with what we've discussed above. In short, we can divide all the affected files into two parts: the update of the code style configuration files (checkstyle + IDE configs), and the update of all the sources to match the code style. In short: - "import order" hotkey will work regardless of which IDE you are using; - updated checkstyle configuration, and IDEA, Eclipse, NetBeans configurations have been updated; - AvoidStarImport checkstyle rule applied as well; The import order we've agreed upon: java.* [blank line] javax.* [blank line] com.* [blank line] net.* [blank line] org.* [blank line] org.apache.cassandra.* [blank line] all other imports [blank line] static all other imports On Mon, 27 Feb 2023 at 13:26, Maxim Muzafarov wrote: > > > I suppose it can be easy for the existing feature branches if they have a > > single commit. Don't we need to adjust each commit for multi-commit feature > > branches? > > It depends on how feature branches are maintained and developed, I > guess. My thoughts here are that the IDE's hotkeys should just work to > resolve any code-style issues that arise during rebase/maintenance. > I'm not talking about enforcing all our code-style rules but giving > developers good flexibility. The classes import order rule might be a > good example here. > > On Wed, 22 Feb 2023 at 21:27, Jacek Lewandowski > wrote: > > > > I suppose it can be easy for the existing feature branches if they have a > > single commit. Don't we need to adjust each commit for multi-commit feature > > branches? > > > > śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov > > napisał: > >> > >> Hello everyone, > >> > >> I have created an issue CASSANDRA-18277 that may help us move forward > >> with code style changes. It only affects the way we store the IntelliJ > >> code style configuration and has no effect on any current (or any) > >> releases, so it should be safe to merge. So, once the issue is > >> resolved, every developer that checkouts a release branch will use the > >> same code style stored in that branch. This in turn makes rebasing a > >> big change like the import order [1] a really straightforward matter > >> (by pressing Crtl + Opt + O in their local branch to organize > >> imports). > >> > >> See: > >> > >> Move the IntelliJ Idea code style and inspections configuration to the > >> project's root .idea directory > >> https://issues.apache.org/jira/browse/CASSANDRA-18277 > >> > >> > >> > >> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925 > >> > >> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan > >> wrote: > >> > > >> > Thank you Maxim for doing this. > >> > > >> > It is nice to see this effort materialized in a PR. > >> > > >> > I would wait until bigger chunks of work are committed to trunk (like > >> > CEP-15) to not collide too much. I would say we can postpone doing this > >> > until the actual 5.0 release, last weeks before it so we would not clash > >> > with any work people would like to include in 5.0. This can go in > >> > anytime, basically. > >> > > >> > Are people on the same page? > >> > > >> > Regards > >> > > >> > > >> > From: Maxim Muzafarov > >> > Sent: Monday, January 23, 2023 19:46 > >> > To: dev@cassandra.apache.org > >> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > >> > > >> > NetApp Security WARNING: This is an external email. Do not click links > >> > or open attachments unless you recognize the sender and know the content > >> > is safe. > >> > > >> > > >> > > >> > > >> > Hello everyone, > >> > > >> > You can find the changes here: > >> > https://issues.apache.org/jira/browse/CASSAND
Re: [DISCUSSION] Cassandra's code style and source code analysis
> I suppose it can be easy for the existing feature branches if they have a > single commit. Don't we need to adjust each commit for multi-commit feature > branches? It depends on how feature branches are maintained and developed, I guess. My thoughts here are that the IDE's hotkeys should just work to resolve any code-style issues that arise during rebase/maintenance. I'm not talking about enforcing all our code-style rules but giving developers good flexibility. The classes import order rule might be a good example here. On Wed, 22 Feb 2023 at 21:27, Jacek Lewandowski wrote: > > I suppose it can be easy for the existing feature branches if they have a > single commit. Don't we need to adjust each commit for multi-commit feature > branches? > > śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov > napisał: >> >> Hello everyone, >> >> I have created an issue CASSANDRA-18277 that may help us move forward >> with code style changes. It only affects the way we store the IntelliJ >> code style configuration and has no effect on any current (or any) >> releases, so it should be safe to merge. So, once the issue is >> resolved, every developer that checkouts a release branch will use the >> same code style stored in that branch. This in turn makes rebasing a >> big change like the import order [1] a really straightforward matter >> (by pressing Crtl + Opt + O in their local branch to organize >> imports). >> >> See: >> >> Move the IntelliJ Idea code style and inspections configuration to the >> project's root .idea directory >> https://issues.apache.org/jira/browse/CASSANDRA-18277 >> >> >> >> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925 >> >> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan >> wrote: >> > >> > Thank you Maxim for doing this. >> > >> > It is nice to see this effort materialized in a PR. >> > >> > I would wait until bigger chunks of work are committed to trunk (like >> > CEP-15) to not collide too much. I would say we can postpone doing this >> > until the actual 5.0 release, last weeks before it so we would not clash >> > with any work people would like to include in 5.0. This can go in anytime, >> > basically. >> > >> > Are people on the same page? >> > >> > Regards >> > >> > >> > From: Maxim Muzafarov >> > Sent: Monday, January 23, 2023 19:46 >> > To: dev@cassandra.apache.org >> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >> > >> > NetApp Security WARNING: This is an external email. Do not click links or >> > open attachments unless you recognize the sender and know the content is >> > safe. >> > >> > >> > >> > >> > Hello everyone, >> > >> > You can find the changes here: >> > https://issues.apache.org/jira/browse/CASSANDRA-17925 >> > >> > While preparing the code style configuration for the Eclipse IDE, I >> > discovered that there was no easy way to have complex grouping options >> > for the set of packages. So we need to add extra blank lines between >> > each group of packages so that all the configurations for Eclipse, >> > NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have >> > checked this earlier for sure, but I only did it for static imports >> > and some groups, my bad. The resultant configuration looks like this: >> > >> > java.* >> > [blank line] >> > javax.* >> > [blank line] >> > com.* >> > [blank line] >> > net.* >> > [blank line] >> > org.* >> > [blank line] >> > org.apache.cassandra.* >> > [blank line] >> > all other imports >> > [blank line] >> > static all other imports >> > >> > The pull request is here: >> > https://github.com/apache/cassandra/pull/2108 >> > >> > The configuration-related changes are placed in a dedicated commit, so >> > it should be easy to make a review: >> > https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a >> > >> > >> > >> > Another important thing to mention is that the total amount of changes >> > for organising imports is really big (more than 2000 files!), so we >> > need to decide the right time to merge this PR. Although rebasing or >> > merging changes
Re: [DISCUSSION] Cassandra's code style and source code analysis
I suppose it can be easy for the existing feature branches if they have a
single commit. Don't we need to adjust each commit for multi-commit feature
branches?
śr., 22 lut 2023, 19:48 użytkownik Maxim Muzafarov
napisał:
> Hello everyone,
>
> I have created an issue CASSANDRA-18277 that may help us move forward
> with code style changes. It only affects the way we store the IntelliJ
> code style configuration and has no effect on any current (or any)
> releases, so it should be safe to merge. So, once the issue is
> resolved, every developer that checkouts a release branch will use the
> same code style stored in that branch. This in turn makes rebasing a
> big change like the import order [1] a really straightforward matter
> (by pressing Crtl + Opt + O in their local branch to organize
> imports).
>
> See:
>
> Move the IntelliJ Idea code style and inspections configuration to the
> project's root .idea directory
> https://issues.apache.org/jira/browse/CASSANDRA-18277
>
>
>
> [1] https://issues.apache.org/jira/browse/CASSANDRA-17925
>
> On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan
> wrote:
> >
> > Thank you Maxim for doing this.
> >
> > It is nice to see this effort materialized in a PR.
> >
> > I would wait until bigger chunks of work are committed to trunk (like
> CEP-15) to not collide too much. I would say we can postpone doing this
> until the actual 5.0 release, last weeks before it so we would not clash
> with any work people would like to include in 5.0. This can go in anytime,
> basically.
> >
> > Are people on the same page?
> >
> > Regards
> >
> > ____________
> > From: Maxim Muzafarov
> > Sent: Monday, January 23, 2023 19:46
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >
> > NetApp Security WARNING: This is an external email. Do not click links
> or open attachments unless you recognize the sender and know the content is
> safe.
> >
> >
> >
> >
> > Hello everyone,
> >
> > You can find the changes here:
> > https://issues.apache.org/jira/browse/CASSANDRA-17925
> >
> > While preparing the code style configuration for the Eclipse IDE, I
> > discovered that there was no easy way to have complex grouping options
> > for the set of packages. So we need to add extra blank lines between
> > each group of packages so that all the configurations for Eclipse,
> > NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> > checked this earlier for sure, but I only did it for static imports
> > and some groups, my bad. The resultant configuration looks like this:
> >
> > java.*
> > [blank line]
> > javax.*
> > [blank line]
> > com.*
> > [blank line]
> > net.*
> > [blank line]
> > org.*
> > [blank line]
> > org.apache.cassandra.*
> > [blank line]
> > all other imports
> > [blank line]
> > static all other imports
> >
> > The pull request is here:
> > https://github.com/apache/cassandra/pull/2108
> >
> > The configuration-related changes are placed in a dedicated commit, so
> > it should be easy to make a review:
> >
> https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
> >
> >
> >
> > Another important thing to mention is that the total amount of changes
> > for organising imports is really big (more than 2000 files!), so we
> > need to decide the right time to merge this PR. Although rebasing or
> > merging changes to development branches should become much easier
> > ("Accept local" + "Organize imports"), we still need to pay extra
> > attention here to minimise the impact on major patches for the next
> > release.
> >
> > On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov wrote:
> > >
> > > Stefan,
> > >
> > > Thank you for bringing this topic up. I'll prepare the PR shortly with
> > > option 4, so everyone can take a look at the amount of changes. This
> > > does not force us to go exactly this path, but it may shed light on
> > > changes in general.
> > >
> > > What exactly we're planning to do in the PR:
> > >
> > > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> > > 2. Checkstyle ImportOrder rule, for controlling the order.
> > > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> > > Eclipse (it doesn't exist for Eclipse yet).
>
Re: [DISCUSSION] Cassandra's code style and source code analysis
Hello everyone,
I have created an issue CASSANDRA-18277 that may help us move forward
with code style changes. It only affects the way we store the IntelliJ
code style configuration and has no effect on any current (or any)
releases, so it should be safe to merge. So, once the issue is
resolved, every developer that checkouts a release branch will use the
same code style stored in that branch. This in turn makes rebasing a
big change like the import order [1] a really straightforward matter
(by pressing Crtl + Opt + O in their local branch to organize
imports).
See:
Move the IntelliJ Idea code style and inspections configuration to the
project's root .idea directory
https://issues.apache.org/jira/browse/CASSANDRA-18277
[1] https://issues.apache.org/jira/browse/CASSANDRA-17925
On Wed, 25 Jan 2023 at 13:05, Miklosovic, Stefan
wrote:
>
> Thank you Maxim for doing this.
>
> It is nice to see this effort materialized in a PR.
>
> I would wait until bigger chunks of work are committed to trunk (like CEP-15)
> to not collide too much. I would say we can postpone doing this until the
> actual 5.0 release, last weeks before it so we would not clash with any work
> people would like to include in 5.0. This can go in anytime, basically.
>
> Are people on the same page?
>
> Regards
>
>
> From: Maxim Muzafarov
> Sent: Monday, January 23, 2023 19:46
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> Hello everyone,
>
> You can find the changes here:
> https://issues.apache.org/jira/browse/CASSANDRA-17925
>
> While preparing the code style configuration for the Eclipse IDE, I
> discovered that there was no easy way to have complex grouping options
> for the set of packages. So we need to add extra blank lines between
> each group of packages so that all the configurations for Eclipse,
> NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> checked this earlier for sure, but I only did it for static imports
> and some groups, my bad. The resultant configuration looks like this:
>
> java.*
> [blank line]
> javax.*
> [blank line]
> com.*
> [blank line]
> net.*
> [blank line]
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
>
> The pull request is here:
> https://github.com/apache/cassandra/pull/2108
>
> The configuration-related changes are placed in a dedicated commit, so
> it should be easy to make a review:
> https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
>
>
>
> Another important thing to mention is that the total amount of changes
> for organising imports is really big (more than 2000 files!), so we
> need to decide the right time to merge this PR. Although rebasing or
> merging changes to development branches should become much easier
> ("Accept local" + "Organize imports"), we still need to pay extra
> attention here to minimise the impact on major patches for the next
> release.
>
> On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov wrote:
> >
> > Stefan,
> >
> > Thank you for bringing this topic up. I'll prepare the PR shortly with
> > option 4, so everyone can take a look at the amount of changes. This
> > does not force us to go exactly this path, but it may shed light on
> > changes in general.
> >
> > What exactly we're planning to do in the PR:
> >
> > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> > 2. Checkstyle ImportOrder rule, for controlling the order.
> > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> > Eclipse (it doesn't exist for Eclipse yet).
> > 4. The import order according to option 4:
> >
> > ```
> > java.*
> > javax.*
> > [blank line]
> > com.*
> > net.*
> > org.*
> > [blank line]
> > org.apache.cassandra.*
> > [blank line]
> > all other imports
> > [blank line]
> > static all other imports
> > ```
> >
> >
> >
> > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> > wrote:
> > >
> > > Based on the voting we should go with option 4?
> > >
> > > Two weeks passed without anybody joining so I guess folks are all happy
> > > with that or this just went unnoticed?
> > >
> > > Let's give it time until the
Re: [DISCUSSION] Cassandra's code style and source code analysis
Turn it on at warning (or lower) level now, so people have some idea of the
size of change to their current code.
On Wed, Jan 25, 2023 at 12:05 PM Miklosovic, Stefan <
stefan.mikloso...@netapp.com> wrote:
> Thank you Maxim for doing this.
>
> It is nice to see this effort materialized in a PR.
>
> I would wait until bigger chunks of work are committed to trunk (like
> CEP-15) to not collide too much. I would say we can postpone doing this
> until the actual 5.0 release, last weeks before it so we would not clash
> with any work people would like to include in 5.0. This can go in anytime,
> basically.
>
> Are people on the same page?
>
> Regards
>
>
> From: Maxim Muzafarov
> Sent: Monday, January 23, 2023 19:46
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
>
>
>
> Hello everyone,
>
> You can find the changes here:
> https://issues.apache.org/jira/browse/CASSANDRA-17925
>
> While preparing the code style configuration for the Eclipse IDE, I
> discovered that there was no easy way to have complex grouping options
> for the set of packages. So we need to add extra blank lines between
> each group of packages so that all the configurations for Eclipse,
> NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
> checked this earlier for sure, but I only did it for static imports
> and some groups, my bad. The resultant configuration looks like this:
>
> java.*
> [blank line]
> javax.*
> [blank line]
> com.*
> [blank line]
> net.*
> [blank line]
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
>
> The pull request is here:
> https://github.com/apache/cassandra/pull/2108
>
> The configuration-related changes are placed in a dedicated commit, so
> it should be easy to make a review:
>
> https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
>
>
>
> Another important thing to mention is that the total amount of changes
> for organising imports is really big (more than 2000 files!), so we
> need to decide the right time to merge this PR. Although rebasing or
> merging changes to development branches should become much easier
> ("Accept local" + "Organize imports"), we still need to pay extra
> attention here to minimise the impact on major patches for the next
> release.
>
> On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov wrote:
> >
> > Stefan,
> >
> > Thank you for bringing this topic up. I'll prepare the PR shortly with
> > option 4, so everyone can take a look at the amount of changes. This
> > does not force us to go exactly this path, but it may shed light on
> > changes in general.
> >
> > What exactly we're planning to do in the PR:
> >
> > 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> > 2. Checkstyle ImportOrder rule, for controlling the order.
> > 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> > Eclipse (it doesn't exist for Eclipse yet).
> > 4. The import order according to option 4:
> >
> > ```
> > java.*
> > javax.*
> > [blank line]
> > com.*
> > net.*
> > org.*
> > [blank line]
> > org.apache.cassandra.*
> > [blank line]
> > all other imports
> > [blank line]
> > static all other imports
> > ```
> >
> >
> >
> > On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> > wrote:
> > >
> > > Based on the voting we should go with option 4?
> > >
> > > Two weeks passed without anybody joining so I guess folks are all
> happy with that or this just went unnoticed?
> > >
> > > Let's give it time until the end of this week (Friday 12:00 UTC).
> > >
> > > Regards
> > >
> > >
> > > From: Maxim Muzafarov
> > > Sent: Tuesday, January 3, 2023 14:31
> > > To: dev@cassandra.apache.org
> > > Subject: Re: [DISCUSSION] Cassandra's code style and source code
> analysis
> > >
> > > NetApp Security WARNING: This is an external email. Do not click links
> or open attachments unless you recognize the sender and know the content is
> safe.
> > >
> > >
> > >
> > >
> > &
Re: [DISCUSSION] Cassandra's code style and source code analysis
Thank you Maxim for doing this.
It is nice to see this effort materialized in a PR.
I would wait until bigger chunks of work are committed to trunk (like CEP-15)
to not collide too much. I would say we can postpone doing this until the
actual 5.0 release, last weeks before it so we would not clash with any work
people would like to include in 5.0. This can go in anytime, basically.
Are people on the same page?
Regards
From: Maxim Muzafarov
Sent: Monday, January 23, 2023 19:46
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
NetApp Security WARNING: This is an external email. Do not click links or open
attachments unless you recognize the sender and know the content is safe.
Hello everyone,
You can find the changes here:
https://issues.apache.org/jira/browse/CASSANDRA-17925
While preparing the code style configuration for the Eclipse IDE, I
discovered that there was no easy way to have complex grouping options
for the set of packages. So we need to add extra blank lines between
each group of packages so that all the configurations for Eclipse,
NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
checked this earlier for sure, but I only did it for static imports
and some groups, my bad. The resultant configuration looks like this:
java.*
[blank line]
javax.*
[blank line]
com.*
[blank line]
net.*
[blank line]
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports
The pull request is here:
https://github.com/apache/cassandra/pull/2108
The configuration-related changes are placed in a dedicated commit, so
it should be easy to make a review:
https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
Another important thing to mention is that the total amount of changes
for organising imports is really big (more than 2000 files!), so we
need to decide the right time to merge this PR. Although rebasing or
merging changes to development branches should become much easier
("Accept local" + "Organize imports"), we still need to pay extra
attention here to minimise the impact on major patches for the next
release.
On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov wrote:
>
> Stefan,
>
> Thank you for bringing this topic up. I'll prepare the PR shortly with
> option 4, so everyone can take a look at the amount of changes. This
> does not force us to go exactly this path, but it may shed light on
> changes in general.
>
> What exactly we're planning to do in the PR:
>
> 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> 2. Checkstyle ImportOrder rule, for controlling the order.
> 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> Eclipse (it doesn't exist for Eclipse yet).
> 4. The import order according to option 4:
>
> ```
> java.*
> javax.*
> [blank line]
> com.*
> net.*
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
>
>
> On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> wrote:
> >
> > Based on the voting we should go with option 4?
> >
> > Two weeks passed without anybody joining so I guess folks are all happy
> > with that or this just went unnoticed?
> >
> > Let's give it time until the end of this week (Friday 12:00 UTC).
> >
> > Regards
> >
> > ________
> > From: Maxim Muzafarov
> > Sent: Tuesday, January 3, 2023 14:31
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >
> > NetApp Security WARNING: This is an external email. Do not click links or
> > open attachments unless you recognize the sender and know the content is
> > safe.
> >
> >
> >
> >
> > Folks,
> >
> > Let me update the voting status and put together everything we have so
> > far. We definitely need more votes to have a solid foundation for this
> > change, so I encourage everyone to consider the options above and
> > share them in this thread.
> >
> >
> > Total for each applicable option:
> >
> > 4-th option -- 4 votes
> > 3-rd option -- 3 votes
> > 5-th option -- 1 vote
> > 1-st option -- 0 votes
> > 2-nd option -- 0 votes
> >
> > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever wrote:
> > >>
> > >>
> > >> 3. Total 5 groups, 2968 files to change
> > >>
> > >> ```
> > >> org.apache.cassandra.*
> > >> [blank line]
> > >&
Re: [DISCUSSION] Cassandra's code style and source code analysis
Hello everyone,
You can find the changes here:
https://issues.apache.org/jira/browse/CASSANDRA-17925
While preparing the code style configuration for the Eclipse IDE, I
discovered that there was no easy way to have complex grouping options
for the set of packages. So we need to add extra blank lines between
each group of packages so that all the configurations for Eclipse,
NetBeans, IntelliJ IDEA and checkstyle are aligned. I should have
checked this earlier for sure, but I only did it for static imports
and some groups, my bad. The resultant configuration looks like this:
java.*
[blank line]
javax.*
[blank line]
com.*
[blank line]
net.*
[blank line]
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports
The pull request is here:
https://github.com/apache/cassandra/pull/2108
The configuration-related changes are placed in a dedicated commit, so
it should be easy to make a review:
https://github.com/apache/cassandra/pull/2108/commits/84e292ddc9671a0be76ceb9304b2b9a051c2d52a
Another important thing to mention is that the total amount of changes
for organising imports is really big (more than 2000 files!), so we
need to decide the right time to merge this PR. Although rebasing or
merging changes to development branches should become much easier
("Accept local" + "Organize imports"), we still need to pay extra
attention here to minimise the impact on major patches for the next
release.
On Mon, 16 Jan 2023 at 13:16, Maxim Muzafarov wrote:
>
> Stefan,
>
> Thank you for bringing this topic up. I'll prepare the PR shortly with
> option 4, so everyone can take a look at the amount of changes. This
> does not force us to go exactly this path, but it may shed light on
> changes in general.
>
> What exactly we're planning to do in the PR:
>
> 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed.
> 2. Checkstyle ImportOrder rule, for controlling the order.
> 3. The IDE code style configuration for Intellij IDEA, NetBeans, and
> Eclipse (it doesn't exist for Eclipse yet).
> 4. The import order according to option 4:
>
> ```
> java.*
> javax.*
> [blank line]
> com.*
> net.*
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
>
>
> On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan
> wrote:
> >
> > Based on the voting we should go with option 4?
> >
> > Two weeks passed without anybody joining so I guess folks are all happy
> > with that or this just went unnoticed?
> >
> > Let's give it time until the end of this week (Friday 12:00 UTC).
> >
> > Regards
> >
> > ________
> > From: Maxim Muzafarov
> > Sent: Tuesday, January 3, 2023 14:31
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
> >
> > NetApp Security WARNING: This is an external email. Do not click links or
> > open attachments unless you recognize the sender and know the content is
> > safe.
> >
> >
> >
> >
> > Folks,
> >
> > Let me update the voting status and put together everything we have so
> > far. We definitely need more votes to have a solid foundation for this
> > change, so I encourage everyone to consider the options above and
> > share them in this thread.
> >
> >
> > Total for each applicable option:
> >
> > 4-th option -- 4 votes
> > 3-rd option -- 3 votes
> > 5-th option -- 1 vote
> > 1-st option -- 0 votes
> > 2-nd option -- 0 votes
> >
> > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever wrote:
> > >>
> > >>
> > >> 3. Total 5 groups, 2968 files to change
> > >>
> > >> ```
> > >> org.apache.cassandra.*
> > >> [blank line]
> > >> java.*
> > >> [blank line]
> > >> javax.*
> > >> [blank line]
> > >> all other imports
> > >> [blank line]
> > >> static all other imports
> > >> ```
> > >
> > >
> > >
> > > 3, then 5.
> > > There's lots under com.*, net.*, org.* that is essentially the same as
> > > "all other imports", what's the reason to separate those?
> > >
> > > My preference for 3 is simply that imports are by default collapsed, and
> > > if I expand them it's the dependencies on other cassandra stuff I'm first
> > > grokking. It's also our only imports that lead to cyclic dependencies
> > > (which we're not good at).
Re: [DISCUSSION] Cassandra's code style and source code analysis
Stefan, Thank you for bringing this topic up. I'll prepare the PR shortly with option 4, so everyone can take a look at the amount of changes. This does not force us to go exactly this path, but it may shed light on changes in general. What exactly we're planning to do in the PR: 1. Checkstyle AvoidStarImport rule, so no star imports will be allowed. 2. Checkstyle ImportOrder rule, for controlling the order. 3. The IDE code style configuration for Intellij IDEA, NetBeans, and Eclipse (it doesn't exist for Eclipse yet). 4. The import order according to option 4: ``` java.* javax.* [blank line] com.* net.* org.* [blank line] org.apache.cassandra.* [blank line] all other imports [blank line] static all other imports ``` On Mon, 16 Jan 2023 at 12:39, Miklosovic, Stefan wrote: > > Based on the voting we should go with option 4? > > Two weeks passed without anybody joining so I guess folks are all happy with > that or this just went unnoticed? > > Let's give it time until the end of this week (Friday 12:00 UTC). > > Regards > > > From: Maxim Muzafarov > Sent: Tuesday, January 3, 2023 14:31 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > > > Folks, > > Let me update the voting status and put together everything we have so > far. We definitely need more votes to have a solid foundation for this > change, so I encourage everyone to consider the options above and > share them in this thread. > > > Total for each applicable option: > > 4-th option -- 4 votes > 3-rd option -- 3 votes > 5-th option -- 1 vote > 1-st option -- 0 votes > 2-nd option -- 0 votes > > On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever wrote: > >> > >> > >> 3. Total 5 groups, 2968 files to change > >> > >> ``` > >> org.apache.cassandra.* > >> [blank line] > >> java.* > >> [blank line] > >> javax.* > >> [blank line] > >> all other imports > >> [blank line] > >> static all other imports > >> ``` > > > > > > > > 3, then 5. > > There's lots under com.*, net.*, org.* that is essentially the same as "all > > other imports", what's the reason to separate those? > > > > My preference for 3 is simply that imports are by default collapsed, and if > > I expand them it's the dependencies on other cassandra stuff I'm first > > grokking. It's also our only imports that lead to cyclic dependencies > > (which we're not good at).
Re: [DISCUSSION] Cassandra's code style and source code analysis
Based on the voting we should go with option 4? Two weeks passed without anybody joining so I guess folks are all happy with that or this just went unnoticed? Let's give it time until the end of this week (Friday 12:00 UTC). Regards From: Maxim Muzafarov Sent: Tuesday, January 3, 2023 14:31 To: dev@cassandra.apache.org Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. Folks, Let me update the voting status and put together everything we have so far. We definitely need more votes to have a solid foundation for this change, so I encourage everyone to consider the options above and share them in this thread. Total for each applicable option: 4-th option -- 4 votes 3-rd option -- 3 votes 5-th option -- 1 vote 1-st option -- 0 votes 2-nd option -- 0 votes On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever wrote: >> >> >> 3. Total 5 groups, 2968 files to change >> >> ``` >> org.apache.cassandra.* >> [blank line] >> java.* >> [blank line] >> javax.* >> [blank line] >> all other imports >> [blank line] >> static all other imports >> ``` > > > > 3, then 5. > There's lots under com.*, net.*, org.* that is essentially the same as "all > other imports", what's the reason to separate those? > > My preference for 3 is simply that imports are by default collapsed, and if I > expand them it's the dependencies on other cassandra stuff I'm first > grokking. It's also our only imports that lead to cyclic dependencies (which > we're not good at).
Re: [DISCUSSION] Cassandra's code style and source code analysis
Folks, Let me update the voting status and put together everything we have so far. We definitely need more votes to have a solid foundation for this change, so I encourage everyone to consider the options above and share them in this thread. Total for each applicable option: 4-th option -- 4 votes 3-rd option -- 3 votes 5-th option -- 1 vote 1-st option -- 0 votes 2-nd option -- 0 votes On Thu, 22 Dec 2022 at 22:06, Mick Semb Wever wrote: >> >> >> 3. Total 5 groups, 2968 files to change >> >> ``` >> org.apache.cassandra.* >> [blank line] >> java.* >> [blank line] >> javax.* >> [blank line] >> all other imports >> [blank line] >> static all other imports >> ``` > > > > 3, then 5. > There's lots under com.*, net.*, org.* that is essentially the same as "all > other imports", what's the reason to separate those? > > My preference for 3 is simply that imports are by default collapsed, and if I > expand them it's the dependencies on other cassandra stuff I'm first > grokking. It's also our only imports that lead to cyclic dependencies (which > we're not good at).
Re: [DISCUSSION] Cassandra's code style and source code analysis
> > > 3. Total 5 groups, 2968 files to change > > ``` > org.apache.cassandra.* > [blank line] > java.* > [blank line] > javax.* > [blank line] > all other imports > [blank line] > static all other imports > ``` > 3, then 5. There's lots under com.*, net.*, org.* that is essentially the same as "all other imports", what's the reason to separate those? My preference for 3 is simply that imports are by default collapsed, and if I expand them it's the dependencies on other cassandra stuff I'm first grokking. It's also our only imports that lead to cyclic dependencies (which we're not good at).
Re: [DISCUSSION] Cassandra's code style and source code analysis
Im good with 3 and 4.
> On Dec 22, 2022, at 10:41 AM, Derek Chen-Becker wrote:
>
> I vote for #4. I've always used the convention of having stdlib stuff
> first, external stuff second, and same-project imports last. I guess
> increasing order of specificity?
>
> Happy Holidays!
>
> Derek
>
> On Thu, Dec 22, 2022 at 7:52 AM Maxim Muzafarov wrote:
>>
>> Hello everyone, have a great vacation and happy holidays to all!
>>
>>
>> I've completed a small research about how the classe's import order
>> rule are spread in the Apache projects. Some of the projects don't
>> have any restrictions over the imports even if they are using the
>> checkstyle configuration. The other ones may have only the consensus
>> over the imports, but they are not reflected in the checkstyle yet
>> (e.g. Kafka). The conclusion here can only be that there is a very
>> large variability in the classe's import order, so we have to agree on
>> the order on our own.
>>
>> You can find the projects, IDEs and frameworks and their corresponding
>> classe's import order below:
>> https://mmuzaf.github.io/blog/Java_Import_Orders.html
>>
>>
>> Most of the time during development in an IDE the classe's imports
>> remains collapsed, so from my point of view the following things
>> related to the classe's import comes into the first place to consider:
>>
>> - a PR review: newly imports must be clearly visible;
>> - try to minimize the total amount of affected files;
>> - the import order rule must be implemented in a simple way and well
>> supported by IDEs and its plugins;
>>
>> In addition to the last mentioned option, the checkstyle itself has
>> some limitations also. For instance, the ImportOrder has a limitation
>> by design to enforce an empty line between groups ("java", "javax"),
>> or CustomImportOrder may have only up to 4 custom groups separated by
>> a blank line.
>>
>>
>>
>> Based on all of the above I can propose the following classe's order.
>> All of them are tested on the latest changes from the trunk branch
>> (commit hash: b171b4ba294126e985d0ee629744516f19c8644e)
>>
>>
>> 1. Total 2 groups, 3072 files to change
>>
>> ```
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>>
>> 2. Total 3 groups, 2345 files to change
>>
>> ```
>> java.*
>> javax.*
>> [blank line]
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>>
>> 3. Total 5 groups, 2968 files to change
>>
>> ```
>> org.apache.cassandra.*
>> [blank line]
>> java.*
>> [blank line]
>> javax.*
>> [blank line]
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>>
>> 4. Total 5 groups, 1792 files to change
>>
>> ```
>> java.*
>> javax.*
>> [blank line]
>> com.*
>> net.*
>> org.*
>> [blank line]
>> org.apache.cassandra.*
>> [blank line]
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>>
>> 5. Total 2 groups, 3114 files to change
>>
>> ```
>> java.*
>> javax.*
>> org.apache.cassandra.*
>> all other imports
>> [blank line]
>> static all other imports
>> ```
>>
>>
>> Of course, any suggestions are really appreciated.
>> Please, share your thoughts.
>>
>> On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever wrote:
Another angle I forgot to mention is that this is quite a big patch and
there are quite big pieces of work coming, being it CEP-15, for example.
So I am trying to figure out if we are ok to just merge this work first
and devs doing CEP-15 will need to rework their imports or we merge this
after them so we will fix their stuff. I do not know what is more
preferable.
>>>
>>>
>>>
>>> Thank you for bringing this point up Stefan.
>>>
>>> I would be actively reaching out to all those engaged with current CEPs,
>>> asking them the rebase impact this would cause and if they are ok with it.
>>> The CEPs are our priority, and we have a significant amount of them in
>>> progress compared to anything we've had for many years.
>>>
>>>
>>>
>
>
>
> --
> +---+
> | Derek Chen-Becker |
> | GPG Key available at https://keybase.io/dchenbecker and |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC |
> +---+
Re: [DISCUSSION] Cassandra's code style and source code analysis
I vote for #4. I've always used the convention of having stdlib stuff
first, external stuff second, and same-project imports last. I guess
increasing order of specificity?
Happy Holidays!
Derek
On Thu, Dec 22, 2022 at 7:52 AM Maxim Muzafarov wrote:
>
> Hello everyone, have a great vacation and happy holidays to all!
>
>
> I've completed a small research about how the classe's import order
> rule are spread in the Apache projects. Some of the projects don't
> have any restrictions over the imports even if they are using the
> checkstyle configuration. The other ones may have only the consensus
> over the imports, but they are not reflected in the checkstyle yet
> (e.g. Kafka). The conclusion here can only be that there is a very
> large variability in the classe's import order, so we have to agree on
> the order on our own.
>
> You can find the projects, IDEs and frameworks and their corresponding
> classe's import order below:
> https://mmuzaf.github.io/blog/Java_Import_Orders.html
>
>
> Most of the time during development in an IDE the classe's imports
> remains collapsed, so from my point of view the following things
> related to the classe's import comes into the first place to consider:
>
> - a PR review: newly imports must be clearly visible;
> - try to minimize the total amount of affected files;
> - the import order rule must be implemented in a simple way and well
> supported by IDEs and its plugins;
>
> In addition to the last mentioned option, the checkstyle itself has
> some limitations also. For instance, the ImportOrder has a limitation
> by design to enforce an empty line between groups ("java", "javax"),
> or CustomImportOrder may have only up to 4 custom groups separated by
> a blank line.
>
>
>
> Based on all of the above I can propose the following classe's order.
> All of them are tested on the latest changes from the trunk branch
> (commit hash: b171b4ba294126e985d0ee629744516f19c8644e)
>
>
> 1. Total 2 groups, 3072 files to change
>
> ```
> all other imports
> [blank line]
> static all other imports
> ```
>
> 2. Total 3 groups, 2345 files to change
>
> ```
> java.*
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 3. Total 5 groups, 2968 files to change
>
> ```
> org.apache.cassandra.*
> [blank line]
> java.*
> [blank line]
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 4. Total 5 groups, 1792 files to change
>
> ```
> java.*
> javax.*
> [blank line]
> com.*
> net.*
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 5. Total 2 groups, 3114 files to change
>
> ```
> java.*
> javax.*
> org.apache.cassandra.*
> all other imports
> [blank line]
> static all other imports
> ```
>
>
> Of course, any suggestions are really appreciated.
> Please, share your thoughts.
>
> On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever wrote:
> >>
> >> Another angle I forgot to mention is that this is quite a big patch and
> >> there are quite big pieces of work coming, being it CEP-15, for example.
> >> So I am trying to figure out if we are ok to just merge this work first
> >> and devs doing CEP-15 will need to rework their imports or we merge this
> >> after them so we will fix their stuff. I do not know what is more
> >> preferable.
> >
> >
> >
> > Thank you for bringing this point up Stefan.
> >
> > I would be actively reaching out to all those engaged with current CEPs,
> > asking them the rebase impact this would cause and if they are ok with it.
> > The CEPs are our priority, and we have a significant amount of them in
> > progress compared to anything we've had for many years.
> >
> >
> >
--
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC |
+---+
Re: [DISCUSSION] Cassandra's code style and source code analysis
4
I would like to avoid star imports in general. Having them explicitly
enumerated seems to be better, it can not happen we use some import wrongly (by
accident). There is rational behind this here (1).
This check can be configured in such a way that star imports could be possible
on some packages. For example "org.apache.cassandra.tools.NodeTool" has star
import "import org.apache.cassandra.tools.nodetool.*;" where all nodetool
commands are. This can be configured so it would remain to be star import on
that package so we do not need to explicitly import 100+ classes. There might
be more cases like this but they would need to be reviewed more closely.
(1) https://checkstyle.org/config_imports.html#AvoidStarImport
From: Maxim Muzafarov
Sent: Thursday, December 22, 2022 15:52
To: dev@cassandra.apache.org
Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis
NetApp Security WARNING: This is an external email. Do not click links or open
attachments unless you recognize the sender and know the content is safe.
Hello everyone, have a great vacation and happy holidays to all!
I've completed a small research about how the classe's import order
rule are spread in the Apache projects. Some of the projects don't
have any restrictions over the imports even if they are using the
checkstyle configuration. The other ones may have only the consensus
over the imports, but they are not reflected in the checkstyle yet
(e.g. Kafka). The conclusion here can only be that there is a very
large variability in the classe's import order, so we have to agree on
the order on our own.
You can find the projects, IDEs and frameworks and their corresponding
classe's import order below:
https://mmuzaf.github.io/blog/Java_Import_Orders.html
Most of the time during development in an IDE the classe's imports
remains collapsed, so from my point of view the following things
related to the classe's import comes into the first place to consider:
- a PR review: newly imports must be clearly visible;
- try to minimize the total amount of affected files;
- the import order rule must be implemented in a simple way and well
supported by IDEs and its plugins;
In addition to the last mentioned option, the checkstyle itself has
some limitations also. For instance, the ImportOrder has a limitation
by design to enforce an empty line between groups ("java", "javax"),
or CustomImportOrder may have only up to 4 custom groups separated by
a blank line.
Based on all of the above I can propose the following classe's order.
All of them are tested on the latest changes from the trunk branch
(commit hash: b171b4ba294126e985d0ee629744516f19c8644e)
1. Total 2 groups, 3072 files to change
```
all other imports
[blank line]
static all other imports
```
2. Total 3 groups, 2345 files to change
```
java.*
javax.*
[blank line]
all other imports
[blank line]
static all other imports
```
3. Total 5 groups, 2968 files to change
```
org.apache.cassandra.*
[blank line]
java.*
[blank line]
javax.*
[blank line]
all other imports
[blank line]
static all other imports
```
4. Total 5 groups, 1792 files to change
```
java.*
javax.*
[blank line]
com.*
net.*
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports
```
5. Total 2 groups, 3114 files to change
```
java.*
javax.*
org.apache.cassandra.*
all other imports
[blank line]
static all other imports
```
Of course, any suggestions are really appreciated.
Please, share your thoughts.
On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever wrote:
>>
>> Another angle I forgot to mention is that this is quite a big patch and
>> there are quite big pieces of work coming, being it CEP-15, for example. So
>> I am trying to figure out if we are ok to just merge this work first and
>> devs doing CEP-15 will need to rework their imports or we merge this after
>> them so we will fix their stuff. I do not know what is more preferable.
>
>
>
> Thank you for bringing this point up Stefan.
>
> I would be actively reaching out to all those engaged with current CEPs,
> asking them the rebase impact this would cause and if they are ok with it.
> The CEPs are our priority, and we have a significant amount of them in
> progress compared to anything we've had for many years.
>
>
>
Re: [DISCUSSION] Cassandra's code style and source code analysis
I like 3 or 4.
We need to be sure we have a way of deactivating the check with code comments
tho, as Java 8 has some bug with import order that can rarely break
compilation, so we need to have some mechanism for permitting a different
import order.
Did we decide any changes to star imports?
> On 22 Dec 2022, at 14:53, Maxim Muzafarov wrote:
>
> Hello everyone, have a great vacation and happy holidays to all!
>
>
> I've completed a small research about how the classe's import order
> rule are spread in the Apache projects. Some of the projects don't
> have any restrictions over the imports even if they are using the
> checkstyle configuration. The other ones may have only the consensus
> over the imports, but they are not reflected in the checkstyle yet
> (e.g. Kafka). The conclusion here can only be that there is a very
> large variability in the classe's import order, so we have to agree on
> the order on our own.
>
> You can find the projects, IDEs and frameworks and their corresponding
> classe's import order below:
> https://mmuzaf.github.io/blog/Java_Import_Orders.html
>
>
> Most of the time during development in an IDE the classe's imports
> remains collapsed, so from my point of view the following things
> related to the classe's import comes into the first place to consider:
>
> - a PR review: newly imports must be clearly visible;
> - try to minimize the total amount of affected files;
> - the import order rule must be implemented in a simple way and well
> supported by IDEs and its plugins;
>
> In addition to the last mentioned option, the checkstyle itself has
> some limitations also. For instance, the ImportOrder has a limitation
> by design to enforce an empty line between groups ("java", "javax"),
> or CustomImportOrder may have only up to 4 custom groups separated by
> a blank line.
>
>
>
> Based on all of the above I can propose the following classe's order.
> All of them are tested on the latest changes from the trunk branch
> (commit hash: b171b4ba294126e985d0ee629744516f19c8644e)
>
>
> 1. Total 2 groups, 3072 files to change
>
> ```
> all other imports
> [blank line]
> static all other imports
> ```
>
> 2. Total 3 groups, 2345 files to change
>
> ```
> java.*
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 3. Total 5 groups, 2968 files to change
>
> ```
> org.apache.cassandra.*
> [blank line]
> java.*
> [blank line]
> javax.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 4. Total 5 groups, 1792 files to change
>
> ```
> java.*
> javax.*
> [blank line]
> com.*
> net.*
> org.*
> [blank line]
> org.apache.cassandra.*
> [blank line]
> all other imports
> [blank line]
> static all other imports
> ```
>
> 5. Total 2 groups, 3114 files to change
>
> ```
> java.*
> javax.*
> org.apache.cassandra.*
> all other imports
> [blank line]
> static all other imports
> ```
>
>
> Of course, any suggestions are really appreciated.
> Please, share your thoughts.
>
>> On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever wrote:
>>> Another angle I forgot to mention is that this is quite a big patch and
>>> there are quite big pieces of work coming, being it CEP-15, for example. So
>>> I am trying to figure out if we are ok to just merge this work first and
>>> devs doing CEP-15 will need to rework their imports or we merge this after
>>> them so we will fix their stuff. I do not know what is more preferable.
>>
>>
>>
>> Thank you for bringing this point up Stefan.
>>
>> I would be actively reaching out to all those engaged with current CEPs,
>> asking them the rebase impact this would cause and if they are ok with it.
>> The CEPs are our priority, and we have a significant amount of them in
>> progress compared to anything we've had for many years.
>>
>>
>>
Re: [DISCUSSION] Cassandra's code style and source code analysis
Hello everyone, have a great vacation and happy holidays to all!
I've completed a small research about how the classe's import order
rule are spread in the Apache projects. Some of the projects don't
have any restrictions over the imports even if they are using the
checkstyle configuration. The other ones may have only the consensus
over the imports, but they are not reflected in the checkstyle yet
(e.g. Kafka). The conclusion here can only be that there is a very
large variability in the classe's import order, so we have to agree on
the order on our own.
You can find the projects, IDEs and frameworks and their corresponding
classe's import order below:
https://mmuzaf.github.io/blog/Java_Import_Orders.html
Most of the time during development in an IDE the classe's imports
remains collapsed, so from my point of view the following things
related to the classe's import comes into the first place to consider:
- a PR review: newly imports must be clearly visible;
- try to minimize the total amount of affected files;
- the import order rule must be implemented in a simple way and well
supported by IDEs and its plugins;
In addition to the last mentioned option, the checkstyle itself has
some limitations also. For instance, the ImportOrder has a limitation
by design to enforce an empty line between groups ("java", "javax"),
or CustomImportOrder may have only up to 4 custom groups separated by
a blank line.
Based on all of the above I can propose the following classe's order.
All of them are tested on the latest changes from the trunk branch
(commit hash: b171b4ba294126e985d0ee629744516f19c8644e)
1. Total 2 groups, 3072 files to change
```
all other imports
[blank line]
static all other imports
```
2. Total 3 groups, 2345 files to change
```
java.*
javax.*
[blank line]
all other imports
[blank line]
static all other imports
```
3. Total 5 groups, 2968 files to change
```
org.apache.cassandra.*
[blank line]
java.*
[blank line]
javax.*
[blank line]
all other imports
[blank line]
static all other imports
```
4. Total 5 groups, 1792 files to change
```
java.*
javax.*
[blank line]
com.*
net.*
org.*
[blank line]
org.apache.cassandra.*
[blank line]
all other imports
[blank line]
static all other imports
```
5. Total 2 groups, 3114 files to change
```
java.*
javax.*
org.apache.cassandra.*
all other imports
[blank line]
static all other imports
```
Of course, any suggestions are really appreciated.
Please, share your thoughts.
On Thu, 15 Dec 2022 at 17:48, Mick Semb Wever wrote:
>>
>> Another angle I forgot to mention is that this is quite a big patch and
>> there are quite big pieces of work coming, being it CEP-15, for example. So
>> I am trying to figure out if we are ok to just merge this work first and
>> devs doing CEP-15 will need to rework their imports or we merge this after
>> them so we will fix their stuff. I do not know what is more preferable.
>
>
>
> Thank you for bringing this point up Stefan.
>
> I would be actively reaching out to all those engaged with current CEPs,
> asking them the rebase impact this would cause and if they are ok with it.
> The CEPs are our priority, and we have a significant amount of them in
> progress compared to anything we've had for many years.
>
>
>
Re: [DISCUSSION] Cassandra's code style and source code analysis
> > Another angle I forgot to mention is that this is quite a big patch and > there are quite big pieces of work coming, being it CEP-15, for example. So > I am trying to figure out if we are ok to just merge this work first and > devs doing CEP-15 will need to rework their imports or we merge this after > them so we will fix their stuff. I do not know what is more preferable. > Thank you for bringing this point up Stefan. I would be actively reaching out to all those engaged with current CEPs, asking them the rebase impact this would cause and if they are ok with it. The CEPs are our priority, and we have a significant amount of them in progress compared to anything we've had for many years.
Re: [DISCUSSION] Cassandra's code style and source code analysis
Hi Maxim, yes I will help you to review that. The approach you outlined sounds reasonable to me by I do not think we can merge this based on my opinion only. I would welcome more developers to discuss this. Another angle I forgot to mention is that this is quite a big patch and there are quite big pieces of work coming, being it CEP-15, for example. So I am trying to figure out if we are ok to just merge this work first and devs doing CEP-15 will need to rework their imports or we merge this after them so we will fix their stuff. I do not know what is more preferable. I do not think that the order of imports as we having right now is the same we want to go with. Check Benedict's comment here (1). I agree that the current ordering is strange. Like, why is there this "org.cliffc.high_scale_lib" import in particular? That seems to be completely arbitrary here (or it has some intrinsic logic but I do not see it nor that decision is nowhere to find documented). We need to gather more feedback here. I ll try to take a look at other Apache projects how their import order is. Regards Regards (1) https://issues.apache.org/jira/browse/CASSANDRA-17925 From: Maxim Muzafarov Sent: Monday, December 12, 2022 17:13 To: dev@cassandra.apache.org Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. > Technically it can be two commits which would be merged / pushed at once. I'll prepare a new pull request containing both of the changes. My previous experience says me that it's really hard to find a reviewer who will be able to go through huge pull requests, that's why initially I've split this into AvoidStarImport and CustomImportOrder rules. So, if you'll help with the review I'm happy to proceed the way you suggested :-) > One thing which needs extra care for ordering imports is that if you order it > in IDEA by right-clicking on a package and choosing organising imports, it > will remove special comments You're right, but this is quite unusual behaviour for me and this seems to be a bug, that hasn't been fixed for a long time [1]. I've tested the same thing for Eclipse and NetBeans and `optimize imports` working there as we expect (no comments removes), so the issue exists only for the IntelliJ IDEA [1]. Despite all of that, we are still on the safe side here - if these comments will be removed by the `optimized import` procedure the build with checkstyle will fail. > I think this is a great time to revisit this ordering. I would say that the imports order is pretty good (probably, except for the blank lines) and the imports order is not as important as it is important that it be the same in all files and automation `optimize imports`. I suggest going through a "minimum change" strategy here. The IntelliJ IDEA has the following configuration with the imports order that most of the classes already fit: import java import javax [blank line] import com.google.common import org.apache.log4j import org.apache.commons import org.cliffc.high_scale_lib import org.junit import org.slf4j [blank line] import all other imports [blank line] import static all other imports We can update the documentation page [2] with this order and implement the same for NetBeans and Eclipse IDE configuration files as well as for the checkstyle config. If everyone is OK with the plan above I'll prepare everything for it. Suggested summary: - use current IntelliJ IDEA imports order as defaults for other IDEs; - update the documentation page; - prepare a single pull request with AvoidStarImport and CustomImportOrder; [1] https://youtrack.jetbrains.com/issue/IDEA-128133/Optimize-Imports-disregards-line-comments [2] https://cassandra.apache.org/_/development/code_style.html On Sun, 11 Dec 2022 at 00:03, Miklosovic, Stefan wrote: > > Should the source code obey the AvoidStarImport rule? > > yes > > Should we implement AvoidStarImport and CustomImportOrder in a > single pull request or do it one by one? > > Technically it can be two commits which would be merged / pushed at once. > > One thing which needs extra care for ordering imports is that if you order it > in IDEA by right-clicking on a package and choosing organising imports, it > will remove special comments which are put at the end of the import > statement. We need to be sure you put them back. Look at changes in > CASSANDRA-17055. We need to preserve this. > > Also, we need to be sure that the importing style can be (roughly) set in > each major IDE. (eclipse / netbeans / idea) so if we require some import > style it can be set in IDE like that. I do not know if we have any strong > preference when it comes to this but it de
Re: [DISCUSSION] Cassandra's code style and source code analysis
> Technically it can be two commits which would be merged / pushed at once. I'll prepare a new pull request containing both of the changes. My previous experience says me that it's really hard to find a reviewer who will be able to go through huge pull requests, that's why initially I've split this into AvoidStarImport and CustomImportOrder rules. So, if you'll help with the review I'm happy to proceed the way you suggested :-) > One thing which needs extra care for ordering imports is that if you order it > in IDEA by right-clicking on a package and choosing organising imports, it > will remove special comments You're right, but this is quite unusual behaviour for me and this seems to be a bug, that hasn't been fixed for a long time [1]. I've tested the same thing for Eclipse and NetBeans and `optimize imports` working there as we expect (no comments removes), so the issue exists only for the IntelliJ IDEA [1]. Despite all of that, we are still on the safe side here - if these comments will be removed by the `optimized import` procedure the build with checkstyle will fail. > I think this is a great time to revisit this ordering. I would say that the imports order is pretty good (probably, except for the blank lines) and the imports order is not as important as it is important that it be the same in all files and automation `optimize imports`. I suggest going through a "minimum change" strategy here. The IntelliJ IDEA has the following configuration with the imports order that most of the classes already fit: import java import javax [blank line] import com.google.common import org.apache.log4j import org.apache.commons import org.cliffc.high_scale_lib import org.junit import org.slf4j [blank line] import all other imports [blank line] import static all other imports We can update the documentation page [2] with this order and implement the same for NetBeans and Eclipse IDE configuration files as well as for the checkstyle config. If everyone is OK with the plan above I'll prepare everything for it. Suggested summary: - use current IntelliJ IDEA imports order as defaults for other IDEs; - update the documentation page; - prepare a single pull request with AvoidStarImport and CustomImportOrder; [1] https://youtrack.jetbrains.com/issue/IDEA-128133/Optimize-Imports-disregards-line-comments [2] https://cassandra.apache.org/_/development/code_style.html On Sun, 11 Dec 2022 at 00:03, Miklosovic, Stefan wrote: > > Should the source code obey the AvoidStarImport rule? > > yes > > Should we implement AvoidStarImport and CustomImportOrder in a > single pull request or do it one by one? > > Technically it can be two commits which would be merged / pushed at once. > > One thing which needs extra care for ordering imports is that if you order it > in IDEA by right-clicking on a package and choosing organising imports, it > will remove special comments which are put at the end of the import > statement. We need to be sure you put them back. Look at changes in > CASSANDRA-17055. We need to preserve this. > > Also, we need to be sure that the importing style can be (roughly) set in > each major IDE. (eclipse / netbeans / idea) so if we require some import > style it can be set in IDE like that. I do not know if we have any strong > preference when it comes to this but it definitely does not hurt. > > Also, I see that the current import style is > > java > [blank line] > com.google.common > org.apache.commons > org.junit > org.slf4j > [blank line] > everything else alphabetically > > I think this is a great time to revisit this ordering. I am not particularly > persuaded on this order and why it was choosen. Where has that decision come > from? > > ________ > From: Maxim Muzafarov > Sent: Wednesday, December 7, 2022 18:29 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > > > Dear community, > > > I have created the epic with code-style activities to track the progress: > https://issues.apache.org/jira/browse/CASSANDRA-18090 > > In my understanding, there is no need to format whole the code base at > once according to the code style described on the page [1], and the > best strategy here is to go forward with small evolutionary changes. > Thus eventually we will come up with a set of rules convenient for all > members of the community. In my mind, having one commit per an added > code style rule should be easy to look at for a reviewer, the git > commits history as well as rebasing/merging other pull requests that > may be affected by
Re: [DISCUSSION] Cassandra's code style and source code analysis
Should the source code obey the AvoidStarImport rule? yes Should we implement AvoidStarImport and CustomImportOrder in a single pull request or do it one by one? Technically it can be two commits which would be merged / pushed at once. One thing which needs extra care for ordering imports is that if you order it in IDEA by right-clicking on a package and choosing organising imports, it will remove special comments which are put at the end of the import statement. We need to be sure you put them back. Look at changes in CASSANDRA-17055. We need to preserve this. Also, we need to be sure that the importing style can be (roughly) set in each major IDE. (eclipse / netbeans / idea) so if we require some import style it can be set in IDE like that. I do not know if we have any strong preference when it comes to this but it definitely does not hurt. Also, I see that the current import style is java [blank line] com.google.common org.apache.commons org.junit org.slf4j [blank line] everything else alphabetically I think this is a great time to revisit this ordering. I am not particularly persuaded on this order and why it was choosen. Where has that decision come from? From: Maxim Muzafarov Sent: Wednesday, December 7, 2022 18:29 To: dev@cassandra.apache.org Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. Dear community, I have created the epic with code-style activities to track the progress: https://issues.apache.org/jira/browse/CASSANDRA-18090 In my understanding, there is no need to format whole the code base at once according to the code style described on the page [1], and the best strategy here is to go forward with small evolutionary changes. Thus eventually we will come up with a set of rules convenient for all members of the community. In my mind, having one commit per an added code style rule should be easy to look at for a reviewer, the git commits history as well as rebasing/merging other pull requests that may be affected by the new rules. I want to raise one more question related to class imports and the classses import order for a wider discussion. The import order is well described on the code style page [1], but using wildcard imports is not mentioned at all. The wildcard imports with their drawbacks has has already been raised in the JIRA issue [2] and didn't get enough attention. The checkstyle has the rules we are interested in for import control and they must be considered together. We can implement them in a single pull request or one by one, or use only the last one: - AvoidStarImport - CustomImportOrder But still, I think that wildcard imports have more disadvantages (class names conflicts e.g. java.util.*, java.sql.* or a new version of a library has name clashes) than advantages and such problems will be found in later CI cycles. Currently, I've implemented the AvoidStarImport checkstyle rule in a dedicated pull request [3][4], so you will be able to see all amount of the changes with removing wildcard imports. The changes are made for the checkstyle configuration as well as for code style configurations for different IDEs we supported. So, the open questions here are: - Should the source code obey the AvoidStarImport rule [3]? (I think yes); - Should we implement AvoidStarImport and CustomImportOrder in a single pull request or do it one by one? Anyway, I will fix the result of the agreement over the AvoidStarImport rule on the documentation page [1]. [1] https://cassandra.apache.org/_/development/code_style.html [2] https://issues.apache.org/jira/browse/CASSANDRA-17925 [3] https://issues.apache.org/jira/browse/CASSANDRA-18089 [4] https://github.com/apache/cassandra/pull/2041 On Thu, 1 Dec 2022 at 11:55, Claude Warren, Jr via dev wrote: > > The last time I worked on a project that tried to implement a coding style > across the project it was "an education". The short story is that trying to > "mitigate" the code base, with respect to style, is either a massive change > or a long slow process. > > Arguments here have stated that earlier attempts to have the tooling reformat > the code did not go well. What we ended up doing was turned on the style > checker and looked at the number of issues across the project. When new code > was accepted the number of issues could not rise. Eventually most of the > code was clean, with a few well coded legacy bits still not up to standard. > We could do something similar here. Much like code coverage, you can't > perform a merge unless the number of style errors remains the same or > decreases. > > As with all software rules, this is a strong recommendation as I am certain > that there are edge/corner
Re: [DISCUSSION] Cassandra's code style and source code analysis
linting I will continue to push back. I > think linting our brace rules could make sense since they are atypical, but > more formatting rules than this likely just leads to atrophying style. > Authorship involves thinking about how to present your code; I don’t want > to either encourage lazy authorship or prevent experimentation with > presentation. Both would be bad, and I expect we would struggle to evolve > our style guide again in future as the language evolves. Our brace rules > are a good example everyone unilaterally ignored when lambdas arrived, as > we all recognised they materially harmed the brevity benefits, and we > eventually codified this. > >>> > >>> On migration: auto formatters applied to code that was not written > with the rules in mind will almost unerringly be made a mess of, so having > a tool do this is not acceptable IMO. > >>> > >>> The idea of checkstyle being the source of truth continues to be > untenable and anyone still pushing for this should please engage with my > earlier points on this. > >>> > >>> > >>> On 30 Nov 2022, at 04:06, Patrick McFadin wrote: > >>> > >>> > >>> I'm going to +1 what Stefan has said. I've heard on many occasions > from newcomers to the project that having to use Ant is a deterrent. As a > matter of fact, a few weeks ago, I spent a Sunday afternoon helping > somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. > ant really super clean this time" > >>> > >>> Sure it still works for people that have been doing this for years. I > drive a 20 year old Toyota truck, but I'm reminded by my kids often that > it's not cool. So in that spirit, I feel my saying we need to keep Ant is > like saying "You kids get off my lawn!" If it's something that will help > attract new contributors, I'm all for it. > >>> > >>> Patrick > >>> > >>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan < > stefan.mikloso...@netapp.com> wrote: > >>>> > >>>> I agree with what you wrote. How I understand it is that migrating to > Maven/Gradle makes the project more "attractive" for newcomers. If a > project is built on "that old un-cool Ant", it might be a little bit > off-putting and questionable if we are "stuck in the past on build systems > and not progressing". > >>>> > >>>> So in that sense I agree this is more "marketing" rather than > technological question but on the other hand, does not Maven/Gradle allow > us to modularize the project better? Maybe we would like to modularize but > nobody is up to that because build system makes it impossible or at least > quite inconvenient to do so. Do you really think there are not any > significant benefits to switch even if it "just works" now? > >>>> > >>>> > >>>> From: Benedict > >>>> Sent: Friday, November 25, 2022 11:07 > >>>> To: dev@cassandra.apache.org > >>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code > analysis > >>>> > >>>> NetApp Security WARNING: This is an external email. Do not click > links or open attachments unless you recognize the sender and know the > content is safe. > >>>> > >>>> > >>>> > >>>> > >>>> There’s always a handful of people asking for it, but notably few if > any of the full time contributors doing the majority of the core > development of Cassandra. It strikes me as something very appealing to > others, but less so to those wanting to get on with development. > >>>> > >>>> I never really see a good argument articulated for the migration, > besides general hand waving that ant is old, and people like newer build > systems. Ant is working fine, so there isn’t a strong technical reason to > replace it, and there are good organisational reasons not to. > >>>> > >>>> Why do you consider a migration inevitable? > >>>> > >>>> > >>>> > >>>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan < > stefan.mikloso...@netapp.com> wrote: > >>>> > > >>>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how > this unfolds. My long-term perception is that changing it to something else > is more or less inevitable but if there is a broader consensus to not do > that well. > >>>> > > >>>
Re: [DISCUSSION] Cassandra's code style and source code analysis
entation. Both would be bad, and I expect we would struggle to evolve >>> our style guide again in future as the language evolves. Our brace rules >>> are a good example everyone unilaterally ignored when lambdas arrived, as >>> we all recognised they materially harmed the brevity benefits, and we >>> eventually codified this. >>> >>> On migration: auto formatters applied to code that was not written with the >>> rules in mind will almost unerringly be made a mess of, so having a tool do >>> this is not acceptable IMO. >>> >>> The idea of checkstyle being the source of truth continues to be untenable >>> and anyone still pushing for this should please engage with my earlier >>> points on this. >>> >>> >>> On 30 Nov 2022, at 04:06, Patrick McFadin wrote: >>> >>> >>> I'm going to +1 what Stefan has said. I've heard on many occasions from >>> newcomers to the project that having to use Ant is a deterrent. As a matter >>> of fact, a few weeks ago, I spent a Sunday afternoon helping somebody >>> trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really >>> super clean this time" >>> >>> Sure it still works for people that have been doing this for years. I drive >>> a 20 year old Toyota truck, but I'm reminded by my kids often that it's not >>> cool. So in that spirit, I feel my saying we need to keep Ant is like >>> saying "You kids get off my lawn!" If it's something that will help attract >>> new contributors, I'm all for it. >>> >>> Patrick >>> >>> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan >>> wrote: >>>> >>>> I agree with what you wrote. How I understand it is that migrating to >>>> Maven/Gradle makes the project more "attractive" for newcomers. If a >>>> project is built on "that old un-cool Ant", it might be a little bit >>>> off-putting and questionable if we are "stuck in the past on build systems >>>> and not progressing". >>>> >>>> So in that sense I agree this is more "marketing" rather than >>>> technological question but on the other hand, does not Maven/Gradle allow >>>> us to modularize the project better? Maybe we would like to modularize but >>>> nobody is up to that because build system makes it impossible or at least >>>> quite inconvenient to do so. Do you really think there are not any >>>> significant benefits to switch even if it "just works" now? >>>> >>>> >>>> From: Benedict >>>> Sent: Friday, November 25, 2022 11:07 >>>> To: dev@cassandra.apache.org >>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >>>> >>>> NetApp Security WARNING: This is an external email. Do not click links or >>>> open attachments unless you recognize the sender and know the content is >>>> safe. >>>> >>>> >>>> >>>> >>>> There’s always a handful of people asking for it, but notably few if any >>>> of the full time contributors doing the majority of the core development >>>> of Cassandra. It strikes me as something very appealing to others, but >>>> less so to those wanting to get on with development. >>>> >>>> I never really see a good argument articulated for the migration, besides >>>> general hand waving that ant is old, and people like newer build systems. >>>> Ant is working fine, so there isn’t a strong technical reason to replace >>>> it, and there are good organisational reasons not to. >>>> >>>> Why do you consider a migration inevitable? >>>> >>>> >>>> >>>> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan >>>> > wrote: >>>> > >>>> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this >>>> > unfolds. My long-term perception is that changing it to something else >>>> > is more or less inevitable but if there is a broader consensus to not do >>>> > that well. >>>> > >>>> > >>>> > From: Benedict >>>> > Sent: Friday, November 25, 2022 10:52 >>>> > To: dev@cass
Re: [DISCUSSION] Cassandra's code style and source code analysis
The last time I worked on a project that tried to implement a coding style across the project it was "an education". The short story is that trying to "mitigate" the code base, with respect to style, is either a massive change or a long slow process. Arguments here have stated that earlier attempts to have the tooling reformat the code did not go well. What we ended up doing was turned on the style checker and looked at the number of issues across the project. When new code was accepted the number of issues could not rise. Eventually most of the code was clean, with a few well coded legacy bits still not up to standard. We could do something similar here. Much like code coverage, you can't perform a merge unless the number of style errors remains the same or decreases. As with all software rules, this is a strong recommendation as I am certain that there are edge/corner case exceptions to be found. On Wed, Nov 30, 2022 at 3:30 PM Patrick McFadin wrote: > Why are we still debating build tooling? I think you’re wrong, but I’ve > conceded - on the assumption that we can get enough volunteers willing to > adopt responsibility for the new world order. > > Not debating. I am just throwing in my support since I have been in the > Camp of Ant. > > On Wed, Nov 30, 2022 at 1:29 AM Benedict wrote: > >> Why are we still debating build tooling? I think you’re wrong, but I’ve >> conceded - on the assumption that we can get enough volunteers willing to >> adopt responsibility for the new world order. >> >> I suggest five long term contributors nominate themselves as the build >> file maintainers, and collectively manage a safe and painless migration for >> the rest of us - and agree to maintain and develop the new build file going >> forwards, and support the community as they adopt it. >> >> On the topic of over-exuberant linting I will continue to push back. I >> think linting our brace rules could make sense since they are atypical, but >> more formatting rules than this likely just leads to atrophying style. >> Authorship involves thinking about how to present your code; I don’t want >> to either encourage lazy authorship or prevent experimentation with >> presentation. Both would be bad, and I expect we would struggle to evolve >> our style guide again in future as the language evolves. Our brace rules >> are a good example everyone unilaterally ignored when lambdas arrived, as >> we all recognised they materially harmed the brevity benefits, and we >> eventually codified this. >> >> On migration: auto formatters applied to code that was not written with >> the rules in mind will almost unerringly be made a mess of, so having a >> tool do this is not acceptable IMO. >> >> The idea of checkstyle being the source of truth continues to be >> untenable and anyone still pushing for this should please engage with my >> earlier points on this. >> >> >> On 30 Nov 2022, at 04:06, Patrick McFadin wrote: >> >> >> I'm going to +1 what Stefan has said. I've heard on many occasions from >> newcomers to the project that having to use Ant is a deterrent. As a matter >> of fact, a few weeks ago, I spent a Sunday afternoon helping somebody >> trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really >> super clean this time" >> >> Sure it still works for people that have been doing this for years. I >> drive a 20 year old Toyota truck, but I'm reminded by my kids often that >> it's not cool. So in that spirit, I feel my saying we need to keep Ant is >> like saying "You kids get off my lawn!" If it's something that will help >> attract new contributors, I'm all for it. >> >> Patrick >> >> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan < >> stefan.mikloso...@netapp.com> wrote: >> >>> I agree with what you wrote. How I understand it is that migrating to >>> Maven/Gradle makes the project more "attractive" for newcomers. If a >>> project is built on "that old un-cool Ant", it might be a little bit >>> off-putting and questionable if we are "stuck in the past on build systems >>> and not progressing". >>> >>> So in that sense I agree this is more "marketing" rather than >>> technological question but on the other hand, does not Maven/Gradle allow >>> us to modularize the project better? Maybe we would like to modularize but >>> nobody is up to that because build system makes it impossible or at least >>> quite inconvenient to do so. Do you really think there are not any >>> significant benef
Re: [DISCUSSION] Cassandra's code style and source code analysis
Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order. Not debating. I am just throwing in my support since I have been in the Camp of Ant. On Wed, Nov 30, 2022 at 1:29 AM Benedict wrote: > Why are we still debating build tooling? I think you’re wrong, but I’ve > conceded - on the assumption that we can get enough volunteers willing to > adopt responsibility for the new world order. > > I suggest five long term contributors nominate themselves as the build > file maintainers, and collectively manage a safe and painless migration for > the rest of us - and agree to maintain and develop the new build file going > forwards, and support the community as they adopt it. > > On the topic of over-exuberant linting I will continue to push back. I > think linting our brace rules could make sense since they are atypical, but > more formatting rules than this likely just leads to atrophying style. > Authorship involves thinking about how to present your code; I don’t want > to either encourage lazy authorship or prevent experimentation with > presentation. Both would be bad, and I expect we would struggle to evolve > our style guide again in future as the language evolves. Our brace rules > are a good example everyone unilaterally ignored when lambdas arrived, as > we all recognised they materially harmed the brevity benefits, and we > eventually codified this. > > On migration: auto formatters applied to code that was not written with > the rules in mind will almost unerringly be made a mess of, so having a > tool do this is not acceptable IMO. > > The idea of checkstyle being the source of truth continues to be untenable > and anyone still pushing for this should please engage with my earlier > points on this. > > > On 30 Nov 2022, at 04:06, Patrick McFadin wrote: > > > I'm going to +1 what Stefan has said. I've heard on many occasions from > newcomers to the project that having to use Ant is a deterrent. As a matter > of fact, a few weeks ago, I spent a Sunday afternoon helping somebody > trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really > super clean this time" > > Sure it still works for people that have been doing this for years. I > drive a 20 year old Toyota truck, but I'm reminded by my kids often that > it's not cool. So in that spirit, I feel my saying we need to keep Ant is > like saying "You kids get off my lawn!" If it's something that will help > attract new contributors, I'm all for it. > > Patrick > > On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan < > stefan.mikloso...@netapp.com> wrote: > >> I agree with what you wrote. How I understand it is that migrating to >> Maven/Gradle makes the project more "attractive" for newcomers. If a >> project is built on "that old un-cool Ant", it might be a little bit >> off-putting and questionable if we are "stuck in the past on build systems >> and not progressing". >> >> So in that sense I agree this is more "marketing" rather than >> technological question but on the other hand, does not Maven/Gradle allow >> us to modularize the project better? Maybe we would like to modularize but >> nobody is up to that because build system makes it impossible or at least >> quite inconvenient to do so. Do you really think there are not any >> significant benefits to switch even if it "just works" now? >> >> >> From: Benedict >> Sent: Friday, November 25, 2022 11:07 >> To: dev@cassandra.apache.org >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >> >> NetApp Security WARNING: This is an external email. Do not click links or >> open attachments unless you recognize the sender and know the content is >> safe. >> >> >> >> >> There’s always a handful of people asking for it, but notably few if any >> of the full time contributors doing the majority of the core development of >> Cassandra. It strikes me as something very appealing to others, but less so >> to those wanting to get on with development. >> >> I never really see a good argument articulated for the migration, besides >> general hand waving that ant is old, and people like newer build systems. >> Ant is working fine, so there isn’t a strong technical reason to replace >> it, and there are good organisational reasons not to. >> >> Why do you consider a migration inevitable? >> >> >> >> > On 25 Nov 2022, at 09:58, M
Re: [DISCUSSION] Cassandra's code style and source code analysis
I again agree with Benedict when it comes to the code formatters. It will certainly bring a lot of mess to the code base and for what benefit? I think I am looking into the Cassandra code long enough to see that we are progressively making it look better. It seems to me that over last few years there was the influx of top-notch developers to the project (not saying the previous generation was less "talented", that is not my point at all) and I feel that we are taking great care of how the code looks like while doing reviews and so on. I feel like, lets say, 8 years ago, the development was way more "wild" and "how it looks" was more of an afterthought rather than something inherently considerate during reviews / contribution. One can feel the difference when looking into the newer subsystems almost immediately. Massively formatting the code would just bring unnecessary big-bang effect for what benefit actually ... Lets just do this naturally. It is something different to fix unused imports or order them the same way, same as introducing spotbugs. But I am big -1 for formatting the code as a whole in one go if that is what we are contemplating about to do as well as I am against to try to change the current code and / or make build-time rules about how the code should look like. I doubt we will ever introduce a reasonable set of rules without changing a lot of already existing code in order to comply with that policy. When it comes to build systems, what I would like to see is to do some "feasability-study" - how much time it would take, what has to change, how Jenkins would look like ... basically what effort we are talking about there. Maybe it could be as simple as trying to have compillable Maven / Gradle project and nothing else so people could gradually add to that. And maybe once we will flip the switch, who knows. From: Benedict Sent: Wednesday, November 30, 2022 10:29 To: dev@cassandra.apache.org Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order. I suggest five long term contributors nominate themselves as the build file maintainers, and collectively manage a safe and painless migration for the rest of us - and agree to maintain and develop the new build file going forwards, and support the community as they adopt it. On the topic of over-exuberant linting I will continue to push back. I think linting our brace rules could make sense since they are atypical, but more formatting rules than this likely just leads to atrophying style. Authorship involves thinking about how to present your code; I don’t want to either encourage lazy authorship or prevent experimentation with presentation. Both would be bad, and I expect we would struggle to evolve our style guide again in future as the language evolves. Our brace rules are a good example everyone unilaterally ignored when lambdas arrived, as we all recognised they materially harmed the brevity benefits, and we eventually codified this. On migration: auto formatters applied to code that was not written with the rules in mind will almost unerringly be made a mess of, so having a tool do this is not acceptable IMO. The idea of checkstyle being the source of truth continues to be untenable and anyone still pushing for this should please engage with my earlier points on this. On 30 Nov 2022, at 04:06, Patrick McFadin wrote: I'm going to +1 what Stefan has said. I've heard on many occasions from newcomers to the project that having to use Ant is a deterrent. As a matter of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really super clean this time" Sure it still works for people that have been doing this for years. I drive a 20 year old Toyota truck, but I'm reminded by my kids often that it's not cool. So in that spirit, I feel my saying we need to keep Ant is like saying "You kids get off my lawn!" If it's something that will help attract new contributors, I'm all for it. Patrick On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan mailto:stefan.mikloso...@netapp.com>> wrote: I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing". So in that sense
Re: [DISCUSSION] Cassandra's code style and source code analysis
Why are we still debating build tooling? I think you’re wrong, but I’ve conceded - on the assumption that we can get enough volunteers willing to adopt responsibility for the new world order. I suggest five long term contributors nominate themselves as the build file maintainers, and collectively manage a safe and painless migration for the rest of us - and agree to maintain and develop the new build file going forwards, and support the community as they adopt it. On the topic of over-exuberant linting I will continue to push back. I think linting our brace rules could make sense since they are atypical, but more formatting rules than this likely just leads to atrophying style. Authorship involves thinking about how to present your code; I don’t want to either encourage lazy authorship or prevent experimentation with presentation. Both would be bad, and I expect we would struggle to evolve our style guide again in future as the language evolves. Our brace rules are a good example everyone unilaterally ignored when lambdas arrived, as we all recognised they materially harmed the brevity benefits, and we eventually codified this. On migration: auto formatters applied to code that was not written with the rules in mind will almost unerringly be made a mess of, so having a tool do this is not acceptable IMO. The idea of checkstyle being the source of truth continues to be untenable and anyone still pushing for this should please engage with my earlier points on this. > On 30 Nov 2022, at 04:06, Patrick McFadin wrote: > > > I'm going to +1 what Stefan has said. I've heard on many occasions from > newcomers to the project that having to use Ant is a deterrent. As a matter > of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying > to build Cassandra and Ant caused a ton of problems. "Ok. ant really super > clean this time" > > Sure it still works for people that have been doing this for years. I drive a > 20 year old Toyota truck, but I'm reminded by my kids often that it's not > cool. So in that spirit, I feel my saying we need to keep Ant is like saying > "You kids get off my lawn!" If it's something that will help attract new > contributors, I'm all for it. > > Patrick > >> On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan >> wrote: >> I agree with what you wrote. How I understand it is that migrating to >> Maven/Gradle makes the project more "attractive" for newcomers. If a project >> is built on "that old un-cool Ant", it might be a little bit off-putting and >> questionable if we are "stuck in the past on build systems and not >> progressing". >> >> So in that sense I agree this is more "marketing" rather than technological >> question but on the other hand, does not Maven/Gradle allow us to modularize >> the project better? Maybe we would like to modularize but nobody is up to >> that because build system makes it impossible or at least quite inconvenient >> to do so. Do you really think there are not any significant benefits to >> switch even if it "just works" now? >> >> >> From: Benedict >> Sent: Friday, November 25, 2022 11:07 >> To: dev@cassandra.apache.org >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >> >> NetApp Security WARNING: This is an external email. Do not click links or >> open attachments unless you recognize the sender and know the content is >> safe. >> >> >> >> >> There’s always a handful of people asking for it, but notably few if any of >> the full time contributors doing the majority of the core development of >> Cassandra. It strikes me as something very appealing to others, but less so >> to those wanting to get on with development. >> >> I never really see a good argument articulated for the migration, besides >> general hand waving that ant is old, and people like newer build systems. >> Ant is working fine, so there isn’t a strong technical reason to replace it, >> and there are good organisational reasons not to. >> >> Why do you consider a migration inevitable? >> >> >> >> > On 25 Nov 2022, at 09:58, Miklosovic, Stefan >> > wrote: >> > >> > Interesting take on Ant / no-Ant, Benedict. I am very curious how this >> > unfolds. My long-term perception is that changing it to something else is >> > more or less inevitable but if there is a broader consensus to not do that >> > well. >> > >> > >> > From: Benedict
Re: [DISCUSSION] Cassandra's code style and source code analysis
I'm going to +1 what Stefan has said. I've heard on many occasions from newcomers to the project that having to use Ant is a deterrent. As a matter of fact, a few weeks ago, I spent a Sunday afternoon helping somebody trying to build Cassandra and Ant caused a ton of problems. "Ok. ant really super clean this time" Sure it still works for people that have been doing this for years. I drive a 20 year old Toyota truck, but I'm reminded by my kids often that it's not cool. So in that spirit, I feel my saying we need to keep Ant is like saying "You kids get off my lawn!" If it's something that will help attract new contributors, I'm all for it. Patrick On Fri, Nov 25, 2022 at 2:22 AM Miklosovic, Stefan < stefan.mikloso...@netapp.com> wrote: > I agree with what you wrote. How I understand it is that migrating to > Maven/Gradle makes the project more "attractive" for newcomers. If a > project is built on "that old un-cool Ant", it might be a little bit > off-putting and questionable if we are "stuck in the past on build systems > and not progressing". > > So in that sense I agree this is more "marketing" rather than > technological question but on the other hand, does not Maven/Gradle allow > us to modularize the project better? Maybe we would like to modularize but > nobody is up to that because build system makes it impossible or at least > quite inconvenient to do so. Do you really think there are not any > significant benefits to switch even if it "just works" now? > > ____ > From: Benedict > Sent: Friday, November 25, 2022 11:07 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is > safe. > > > > > There’s always a handful of people asking for it, but notably few if any > of the full time contributors doing the majority of the core development of > Cassandra. It strikes me as something very appealing to others, but less so > to those wanting to get on with development. > > I never really see a good argument articulated for the migration, besides > general hand waving that ant is old, and people like newer build systems. > Ant is working fine, so there isn’t a strong technical reason to replace > it, and there are good organisational reasons not to. > > Why do you consider a migration inevitable? > > > > > On 25 Nov 2022, at 09:58, Miklosovic, Stefan < > stefan.mikloso...@netapp.com> wrote: > > > > Interesting take on Ant / no-Ant, Benedict. I am very curious how this > unfolds. My long-term perception is that changing it to something else is > more or less inevitable but if there is a broader consensus to not do that > well. > > > > > > From: Benedict > > Sent: Friday, November 25, 2022 10:52 > > To: dev@cassandra.apache.org > > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > > > NetApp Security WARNING: This is an external email. Do not click links > or open attachments unless you recognize the sender and know the content is > safe. > > > > > > > > > > I was in a bit of a rush last night. I should say that I’m of course +1 > a general endeavour to clean this up, and to expand our use of linters, and > I appreciate your volunteering to help out in this way Maxim. > > > > However, responding to Stefan, I’m pretty -1 migrating from ant to > another build system without really good reason. Migration has a real cost > to productivity for all existing contributors, and the phantom of > increasing new contributions has never paid off historically. I’m all for > easing people into participation, but not at penalty to the existing > contributor base. > > > > If the only reason is to make it easier to open in a different IDE, we > can perhaps have some basic build files outlining code structure for > importing, that are compatible with our canonical ant build? We could > perhaps even generate them. > > > > > >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan < > stefan.mikloso...@netapp.com> wrote: > >> > >> For the record, I was testing that same combo Claude mentioned and it > did not work out of the box but it is definitely possible to set up > successfully. I do not remember the details. > >> > >> To replay to Maxim, it all seems good to me, roughly, but I humbly > think it all boils down to Maven/Gradle refactoring and on top of that we > can do
Re: [DISCUSSION] Cassandra's code style and source code analysis
So much awesome here. Big +1 to having checkstyle be the source of truth. On 2022/11/24 17:10:28 Maxim Muzafarov wrote: > Hello everyone, > > > First of all, thank you all for this awesome project which I have > often been inspired by. My name is Maxim Muzafarov I'm a Committer and > PMC of Apache Ignite hence you most likely don't know me as I come > from another part of the ASF. Perhaps, I did almost the same things > with B-Trees, off-heap memory management, rebalancing, checkpointing, > snapshotting, and IEPs (you are calling it CEPs) but on a slightly > different distributed database architecture. > > Nevertheless, > > I was chasing down for my first issue to get experience with Cassandra > and found a bunch of opened JIRAs related to the source code analysis > (static analysis as well as the code style). These issues still appear > in JIRA from time to time [1][2][3][4]. It seems to me there not > enough attention has been paid to this topic and all possible options > for this analysis and code style haven't been widely discussed before. > I'd like to summarize everything that I have found and offer my skills > and my experience for solving some of such issues we'll agree on. > > > = Motivation = > > The goal is to make small contributions easier and safer to apply with > GitHub PRs for both a contributor and a committer by adding automation > code checks for each new Cassandra contribution. This also will help > to reduce the time required for reviewing and applying such PRs by an > experienced developer. > > As you may know, the infrastructure team has disabled public sign-up > to ASF JIRA (the GitHub issues are recommended instead). Thus the > following things become more important if we are still interested in > attracting new contributions as it was discussed earlier [6]. > > I do not want to add extra steps to the existing workflow with code > review or make GitHub pull requests as default for patches as it also > was discussed already [7], just to improve the automation checks in it > and make checks more convenient. > > > = Proposed Solution = > > == 1. Make the checkstyle config a single point of truth for the > source code style. == > > The checkstyle is already used and included in the Cassandra project > build lifecycle (ant command line, Jenkins, CircleCI). There is no > need to maintain code style configurations for different types of IDEs > (e.g. IntelliJ inspections configuration) since the checkstyle.xml > file can be directly imported to IDE used by a developer. This is fair > for Intellij Idea, NetBeans, and Eclipse. > > So, I propose to focus on the checks themselves and checking pull > requests with automation scripts, rather than maintaining these > integrations. The benefits here are avoiding all issues with > maintaining configurations for different IDEs. Another good advantage > of this approach would be the ability to add new checkstyle rules > without touching IDE configuration - and such tickets will be LFH and > easy to commit. > > The actions points here are: > > - create an umbrella JIRA ticket for all checkstyle issues e.g. [8] > (or label checkstyle); > - add checkstyle to GitHub pull requests using GitHub actions (execute > ant command); > - include checkstyle to the build (already done); > - remove redundant code style configurations related to IDEs from the > source code e.g. [9]; > > > == 2. Add additional tooling for code analysis to the build and GitHub > pull requests. == > > The source code static analysis and automated checks have been > discussed recently in the "SpotBugs to the build" topic [10]. I'd like > to add my 50 cents here. > > Before discussing the pros and cons of each solution, let's focus on > the criteria that such solutions must meet. You can find the most > complete list of such tooling here [11]. > > From my point of view, the crucial criteria are: > - free for open-source (at least licenses should allow such usages); > - popularity in the ASF ecosystems; > - convenient integration and/or plugins for IDEs and GitHub; > - we must be able to integrate with CirleCI, and Jenkins as well as > launch from a command line; > > > === Sonar === > > pros > + this tool is free for open-source and recommended by the ASF > infrastructure team [12]; > + was already used for the Cassandra project some time ago at > sonarcloud.io [13]; > + it has GitHub pull requests analysis [14]; > > cons > - run locally requires additional configuration and TOKEN_ID due to > the analysis results stored in the ext database (infra will not > provide it for local usage); > > === SpotBugs (FindBugs) === > > pros > + license is allowed to use it and run it as a library (should be legal for > us); > + it analyses the bytecode that differs from how the checkstyle works; > + can be executed from the command line as well as integrated into the build; > > cons > - requires compiled source code; > > === PMD === > > pros > + BSD licenses more
Re: [DISCUSSION] Cassandra's code style and source code analysis
lator and others I think. I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it. I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else. On 25 Nov 2022, at 10:22, Miklosovic, Stefan <mailto:stefan.mikloso...@netapp.com>> wrote: I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing". So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now? From: Benedict mailto:bened...@apache.org>> Sent: Friday, November 25, 2022 11:07 To: dev@cassandra.apache.org <mailto:dev@cassandra.apache.org> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development. I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to. Why do you consider a migration inevitable? On 25 Nov 2022, at 09:58, Miklosovic, Stefan <mailto:stefan.mikloso...@netapp.com>> wrote: Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that well. From: Benedict mailto:bened...@apache.org>> Sent: Friday, November 25, 2022 10:52 To: dev@cassandra.apache.org <mailto:dev@cassandra.apache.org> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim. However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base. If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them. On 25 Nov 2022, at 09:35, Miklosovic, Stefan <mailto:stefan.mikloso...@netapp.com>> wrote: For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details. To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else. For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to loo
Re: [DISCUSSION] Cassandra's code style and source code analysis
n of lints that can reject patches. >>> >>> +1. It may be worthwhile to configure 2 tiers of lints, optional and >>> required, so we can move to a more gradual process of cleaning up lint >>> violations for those that are interested in that type of work. I know in >>> the past we've seen value in looking at the diff in linting violations even >>> w/a 1k+ noisy violation environment. >>> >>> >>> On Fri, Nov 25, 2022, at 12:41 PM, sc...@paradoxica.net wrote: >>> >>> For me, the strongest arguments in favor of adopting a modern build tool >>> like Maven or Gradle are their ecosystems - both explicit (in terms of >>> plugins), and implicit (in terms of nearly all build tooling supporting >>> both of them, but not ant). >>> >>> Investment in Ant - and in tooling that integrates with Ant - fell off >>> years ago. This makes integrating build-phase aspects of Cassandra with >>> other tooling a very frustrating task that users of most build tools get >>> for free. Many tools built in the last several years don’t support it, or >>> do so only as an afterthought. >>> >>> Two recent examples that have caused pain for me, which I suspect are felt >>> by many: >>> >>> – Integration with internal build systems at many companies that develop >>> Cassandra. Because ant has fallen into disuse, this integration is heavily >>> manual instead of automatic and free. It usually requires forking the >>> project’s build.xml, developing custom tooling around it, or creating a >>> mock Gradle build that wraps ant lifecycle tasks (which also requires >>> overriding ant tasks whose names clash). >>> >>> – Security toolchain integration. Many users and developers of Cassandra >>> also integrate with security tooling at their respective companies. Because >>> Ant has fallen into disuse, most tooling commonly used by security >>> organizations doesn’t support it. SBOMs are a good example, as their >>> introduction postdates ant’s decline. Maven plugins exist to generate them >>> in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra >>> users and developers who need them must manually write shims to produce >>> SBOMs that users of modern build tools get for free. >>> >>> These might not be use cases anticipated by the project, but they represent >>> work I suspect a large number of contributors to the project are required >>> to perform to make Cassandra usable for them. >>> >>> The ecosystem point means that the fix for this has to be external to the >>> project if the project continues to use Ant: lots of one-off scripts and >>> shims unsuitable for contribution with sole maintainers at their respective >>> companies to provide functionality that users of modern build tools get for >>> free. It also demands continuous, incremental work to adapt to changes in >>> security and build tooling in use at many companies that don’t need to be >>> made for projects using well-supported tools like Maven or Gradle. >>> >>> – Scott >>> >>>> On Nov 25, 2022, at 4:56 AM, Benedict wrote: >>>> >>>> I think modules are already fairly well supported - we in effect already >>>> have several? FQL, Simulator and others I think. >>>> >>>> I can anyway say with absolute certainty that the main impediment to >>>> modularising is not build tooling, it’s the difficulty of the task, the >>>> cost to the project of undertaking it, and therefore its relative payoff >>>> versus other things that could be undertaken by the folk with the relevant >>>> expertise to do it. >>>> >>>> I’ve also been around long enough to see enough efforts to broaden >>>> contributions fail to have an impact, that basing costly decisions on this >>>> kind of reasoning doesn’t resonate. The main impediments to contributions >>>> are the complexity of the codebase (and problem domain) and our limited >>>> capacity to respond promptly to high quality contributions. Until we fix >>>> those fundamental issues, this kind of rearranging of chairs seems more >>>> like a branding exercise to ourselves than to anyone else. >>>> >>>> >>>>> On 25 Nov 2022, at 10:22, Miklosovic, Stefan >>>>> wrote: >>>>> >>>>> I agree with what you wrote. How I
Re: [DISCUSSION] Cassandra's code style and source code analysis
f lints, optional and >>> required, so we can move to a more gradual process of cleaning up lint >>> violations for those that are interested in that type of work. I know in >>> the past we've seen value in looking at the diff in linting violations even >>> w/a 1k+ noisy violation environment. >>> >>> >>> On Fri, Nov 25, 2022, at 12:41 PM, sc...@paradoxica.net wrote: >>> >>> For me, the strongest arguments in favor of adopting a modern build tool >>> like Maven or Gradle are their ecosystems - both explicit (in terms of >>> plugins), and implicit (in terms of nearly all build tooling supporting >>> both of them, but not ant). >>> >>> Investment in Ant - and in tooling that integrates with Ant - fell off >>> years ago. This makes integrating build-phase aspects of Cassandra with >>> other tooling a very frustrating task that users of most build tools get >>> for free. Many tools built in the last several years don’t support it, or >>> do so only as an afterthought. >>> >>> Two recent examples that have caused pain for me, which I suspect are felt >>> by many: >>> >>> – Integration with internal build systems at many companies that develop >>> Cassandra. Because ant has fallen into disuse, this integration is heavily >>> manual instead of automatic and free. It usually requires forking the >>> project’s build.xml, developing custom tooling around it, or creating a >>> mock Gradle build that wraps ant lifecycle tasks (which also requires >>> overriding ant tasks whose names clash). >>> >>> – Security toolchain integration. Many users and developers of Cassandra >>> also integrate with security tooling at their respective companies. Because >>> Ant has fallen into disuse, most tooling commonly used by security >>> organizations doesn’t support it. SBOMs are a good example, as their >>> introduction postdates ant’s decline. Maven plugins exist to generate them >>> in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra >>> users and developers who need them must manually write shims to produce >>> SBOMs that users of modern build tools get for free. >>> >>> These might not be use cases anticipated by the project, but they represent >>> work I suspect a large number of contributors to the project are required >>> to perform to make Cassandra usable for them. >>> >>> The ecosystem point means that the fix for this has to be external to the >>> project if the project continues to use Ant: lots of one-off scripts and >>> shims unsuitable for contribution with sole maintainers at their respective >>> companies to provide functionality that users of modern build tools get for >>> free. It also demands continuous, incremental work to adapt to changes in >>> security and build tooling in use at many companies that don’t need to be >>> made for projects using well-supported tools like Maven or Gradle. >>> >>> – Scott >>> >>>> On Nov 25, 2022, at 4:56 AM, Benedict wrote: >>>> >>>> I think modules are already fairly well supported - we in effect already >>>> have several? FQL, Simulator and others I think. >>>> >>>> I can anyway say with absolute certainty that the main impediment to >>>> modularising is not build tooling, it’s the difficulty of the task, the >>>> cost to the project of undertaking it, and therefore its relative payoff >>>> versus other things that could be undertaken by the folk with the relevant >>>> expertise to do it. >>>> >>>> I’ve also been around long enough to see enough efforts to broaden >>>> contributions fail to have an impact, that basing costly decisions on this >>>> kind of reasoning doesn’t resonate. The main impediments to contributions >>>> are the complexity of the codebase (and problem domain) and our limited >>>> capacity to respond promptly to high quality contributions. Until we fix >>>> those fundamental issues, this kind of rearranging of chairs seems more >>>> like a branding exercise to ourselves than to anyone else. >>>> >>>> >>>>> On 25 Nov 2022, at 10:22, Miklosovic, Stefan >>>>> wrote: >>>>> >>>>> I agree with what you wrote. How I understand it is that migrating to >>>>> Maven/Gradle makes the project more "attrac
Re: [DISCUSSION] Cassandra's code style and source code analysis
s anticipated by the project, but they represent >> work I suspect a large number of contributors to the project are required to >> perform to make Cassandra usable for them. >> >> The ecosystem point means that the fix for this has to be external to the >> project if the project continues to use Ant: lots of one-off scripts and >> shims unsuitable for contribution with sole maintainers at their respective >> companies to provide functionality that users of modern build tools get for >> free. It also demands continuous, incremental work to adapt to changes in >> security and build tooling in use at many companies that don’t need to be >> made for projects using well-supported tools like Maven or Gradle. >> >> – Scott >> >>> On Nov 25, 2022, at 4:56 AM, Benedict wrote: >>> >>> I think modules are already fairly well supported - we in effect already >>> have several? FQL, Simulator and others I think. >>> >>> I can anyway say with absolute certainty that the main impediment to >>> modularising is not build tooling, it’s the difficulty of the task, the >>> cost to the project of undertaking it, and therefore its relative payoff >>> versus other things that could be undertaken by the folk with the relevant >>> expertise to do it. >>> >>> I’ve also been around long enough to see enough efforts to broaden >>> contributions fail to have an impact, that basing costly decisions on this >>> kind of reasoning doesn’t resonate. The main impediments to contributions >>> are the complexity of the codebase (and problem domain) and our limited >>> capacity to respond promptly to high quality contributions. Until we fix >>> those fundamental issues, this kind of rearranging of chairs seems more >>> like a branding exercise to ourselves than to anyone else. >>> >>> >>>> On 25 Nov 2022, at 10:22, Miklosovic, Stefan >>>> wrote: >>>> >>>> I agree with what you wrote. How I understand it is that migrating to >>>> Maven/Gradle makes the project more "attractive" for newcomers. If a >>>> project is built on "that old un-cool Ant", it might be a little bit >>>> off-putting and questionable if we are "stuck in the past on build systems >>>> and not progressing". >>>> >>>> So in that sense I agree this is more "marketing" rather than >>>> technological question but on the other hand, does not Maven/Gradle allow >>>> us to modularize the project better? Maybe we would like to modularize but >>>> nobody is up to that because build system makes it impossible or at least >>>> quite inconvenient to do so. Do you really think there are not any >>>> significant benefits to switch even if it "just works" now? >>>> >>>> >>>> From: Benedict >>>> Sent: Friday, November 25, 2022 11:07 >>>> To: dev@cassandra.apache.org >>>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >>>> >>>> NetApp Security WARNING: This is an external email. Do not click links or >>>> open attachments unless you recognize the sender and know the content is >>>> safe. >>>> >>>> >>>> >>>> >>>> There’s always a handful of people asking for it, but notably few if any >>>> of the full time contributors doing the majority of the core development >>>> of Cassandra. It strikes me as something very appealing to others, but >>>> less so to those wanting to get on with development. >>>> >>>> I never really see a good argument articulated for the migration, besides >>>> general hand waving that ant is old, and people like newer build systems. >>>> Ant is working fine, so there isn’t a strong technical reason to replace >>>> it, and there are good organisational reasons not to. >>>> >>>> Why do you consider a migration inevitable? >>>> >>>> >>>> >>>>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan >>>>> wrote: >>>>> >>>>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this >>>>> unfolds. My long-term perception is that changing it to something else is >>>>> more or less inevitable but if there is a broader cons
Re: [DISCUSSION] Cassandra's code style and source code analysis
ild.xml, developing custom tooling around it, or creating a mock > Gradle build that wraps ant lifecycle tasks (which also requires overriding > ant tasks whose names clash). > > – Security toolchain integration. Many users and developers of Cassandra also > integrate with security tooling at their respective companies. Because Ant > has fallen into disuse, most tooling commonly used by security organizations > doesn’t support it. SBOMs are a good example, as their introduction postdates > ant’s decline. Maven plugins exist to generate them in CycloneDX and SPDX > format, but no such plugins exist for ant. Cassandra users and developers who > need them must manually write shims to produce SBOMs that users of modern > build tools get for free. > > These might not be use cases anticipated by the project, but they represent > work I suspect a large number of contributors to the project are required to > perform to make Cassandra usable for them. > > The ecosystem point means that the fix for this has to be external to the > project if the project continues to use Ant: lots of one-off scripts and > shims unsuitable for contribution with sole maintainers at their respective > companies to provide functionality that users of modern build tools get for > free. It also demands continuous, incremental work to adapt to changes in > security and build tooling in use at many companies that don’t need to be > made for projects using well-supported tools like Maven or Gradle. > > – Scott > > > On Nov 25, 2022, at 4:56 AM, Benedict wrote: > > > > I think modules are already fairly well supported - we in effect already > > have several? FQL, Simulator and others I think. > > > > I can anyway say with absolute certainty that the main impediment to > > modularising is not build tooling, it’s the difficulty of the task, the > > cost to the project of undertaking it, and therefore its relative payoff > > versus other things that could be undertaken by the folk with the relevant > > expertise to do it. > > > > I’ve also been around long enough to see enough efforts to broaden > > contributions fail to have an impact, that basing costly decisions on this > > kind of reasoning doesn’t resonate. The main impediments to contributions > > are the complexity of the codebase (and problem domain) and our limited > > capacity to respond promptly to high quality contributions. Until we fix > > those fundamental issues, this kind of rearranging of chairs seems more > > like a branding exercise to ourselves than to anyone else. > > > > > >> On 25 Nov 2022, at 10:22, Miklosovic, Stefan > >> wrote: > >> > >> I agree with what you wrote. How I understand it is that migrating to > >> Maven/Gradle makes the project more "attractive" for newcomers. If a > >> project is built on "that old un-cool Ant", it might be a little bit > >> off-putting and questionable if we are "stuck in the past on build systems > >> and not progressing". > >> > >> So in that sense I agree this is more "marketing" rather than > >> technological question but on the other hand, does not Maven/Gradle allow > >> us to modularize the project better? Maybe we would like to modularize but > >> nobody is up to that because build system makes it impossible or at least > >> quite inconvenient to do so. Do you really think there are not any > >> significant benefits to switch even if it "just works" now? > >> > >> > >> From: Benedict > >> Sent: Friday, November 25, 2022 11:07 > >> To: dev@cassandra.apache.org > >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > >> > >> NetApp Security WARNING: This is an external email. Do not click links or > >> open attachments unless you recognize the sender and know the content is > >> safe. > >> > >> > >> > >> > >> There’s always a handful of people asking for it, but notably few if any > >> of the full time contributors doing the majority of the core development > >> of Cassandra. It strikes me as something very appealing to others, but > >> less so to those wanting to get on with development. > >> > >> I never really see a good argument articulated for the migration, besides > >> general hand waving that ant is old, and people like newer build systems. > >> Ant is working fine, so there isn’t a strong technical reason to replace > >> it, and th
Re: [DISCUSSION] Cassandra's code style and source code analysis
ing and questionable if we are "stuck in the past on build systems > >> and not progressing". > >> > >> So in that sense I agree this is more "marketing" rather than > >> technological question but on the other hand, does not Maven/Gradle allow > >> us to modularize the project better? Maybe we would like to modularize but > >> nobody is up to that because build system makes it impossible or at least > >> quite inconvenient to do so. Do you really think there are not any > >> significant benefits to switch even if it "just works" now? > >> > >> > >> From: Benedict > >> Sent: Friday, November 25, 2022 11:07 > >> To: dev@cassandra.apache.org > >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > >> > >> NetApp Security WARNING: This is an external email. Do not click links or > >> open attachments unless you recognize the sender and know the content is > >> safe. > >> > >> > >> > >> > >> There’s always a handful of people asking for it, but notably few if any > >> of the full time contributors doing the majority of the core development > >> of Cassandra. It strikes me as something very appealing to others, but > >> less so to those wanting to get on with development. > >> > >> I never really see a good argument articulated for the migration, besides > >> general hand waving that ant is old, and people like newer build systems. > >> Ant is working fine, so there isn’t a strong technical reason to replace > >> it, and there are good organisational reasons not to. > >> > >> Why do you consider a migration inevitable? > >> > >> > >> > >>> On 25 Nov 2022, at 09:58, Miklosovic, Stefan > >>> wrote: > >>> > >>> Interesting take on Ant / no-Ant, Benedict. I am very curious how this > >>> unfolds. My long-term perception is that changing it to something else is > >>> more or less inevitable but if there is a broader consensus to not do > >>> that well. > >>> > >>> > >>> From: Benedict > >>> Sent: Friday, November 25, 2022 10:52 > >>> To: dev@cassandra.apache.org > >>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > >>> > >>> NetApp Security WARNING: This is an external email. Do not click links or > >>> open attachments unless you recognize the sender and know the content is > >>> safe. > >>> > >>> > >>> > >>> > >>> I was in a bit of a rush last night. I should say that I’m of course +1 a > >>> general endeavour to clean this up, and to expand our use of linters, and > >>> I appreciate your volunteering to help out in this way Maxim. > >>> > >>> However, responding to Stefan, I’m pretty -1 migrating from ant to > >>> another build system without really good reason. Migration has a real > >>> cost to productivity for all existing contributors, and the phantom of > >>> increasing new contributions has never paid off historically. I’m all for > >>> easing people into participation, but not at penalty to the existing > >>> contributor base. > >>> > >>> If the only reason is to make it easier to open in a different IDE, we > >>> can perhaps have some basic build files outlining code structure for > >>> importing, that are compatible with our canonical ant build? We could > >>> perhaps even generate them. > >>> > >>> > >>>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan > >>>>> wrote: > >>>> > >>>> For the record, I was testing that same combo Claude mentioned and it > >>>> did not work out of the box but it is definitely possible to set up > >>>> successfully. I do not remember the details. > >>>> > >>>> To replay to Maxim, it all seems good to me, roughly, but I humbly think > >>>> it all boils down to Maven/Gradle refactoring and on top of that we can > >>>> do all else. > >>>> > >>>> For example, there is (1) where the solution, besides fixing the tests, > >>>> is to introduce an Ant task which would check this on build. That
Re: [DISCUSSION] Cassandra's code style and source code analysis
For me, the strongest arguments in favor of adopting a modern build tool like Maven or Gradle are their ecosystems - both explicit (in terms of plugins), and implicit (in terms of nearly all build tooling supporting both of them, but not ant). Investment in Ant - and in tooling that integrates with Ant - fell off years ago. This makes integrating build-phase aspects of Cassandra with other tooling a very frustrating task that users of most build tools get for free. Many tools built in the last several years don’t support it, or do so only as an afterthought. Two recent examples that have caused pain for me, which I suspect are felt by many: – Integration with internal build systems at many companies that develop Cassandra. Because ant has fallen into disuse, this integration is heavily manual instead of automatic and free. It usually requires forking the project’s build.xml, developing custom tooling around it, or creating a mock Gradle build that wraps ant lifecycle tasks (which also requires overriding ant tasks whose names clash). – Security toolchain integration. Many users and developers of Cassandra also integrate with security tooling at their respective companies. Because Ant has fallen into disuse, most tooling commonly used by security organizations doesn’t support it. SBOMs are a good example, as their introduction postdates ant’s decline. Maven plugins exist to generate them in CycloneDX and SPDX format, but no such plugins exist for ant. Cassandra users and developers who need them must manually write shims to produce SBOMs that users of modern build tools get for free. These might not be use cases anticipated by the project, but they represent work I suspect a large number of contributors to the project are required to perform to make Cassandra usable for them. The ecosystem point means that the fix for this has to be external to the project if the project continues to use Ant: lots of one-off scripts and shims unsuitable for contribution with sole maintainers at their respective companies to provide functionality that users of modern build tools get for free. It also demands continuous, incremental work to adapt to changes in security and build tooling in use at many companies that don’t need to be made for projects using well-supported tools like Maven or Gradle. – Scott > On Nov 25, 2022, at 4:56 AM, Benedict wrote: > > I think modules are already fairly well supported - we in effect already have > several? FQL, Simulator and others I think. > > I can anyway say with absolute certainty that the main impediment to > modularising is not build tooling, it’s the difficulty of the task, the cost > to the project of undertaking it, and therefore its relative payoff versus > other things that could be undertaken by the folk with the relevant expertise > to do it. > > I’ve also been around long enough to see enough efforts to broaden > contributions fail to have an impact, that basing costly decisions on this > kind of reasoning doesn’t resonate. The main impediments to contributions are > the complexity of the codebase (and problem domain) and our limited capacity > to respond promptly to high quality contributions. Until we fix those > fundamental issues, this kind of rearranging of chairs seems more like a > branding exercise to ourselves than to anyone else. > > >> On 25 Nov 2022, at 10:22, Miklosovic, Stefan >> wrote: >> >> I agree with what you wrote. How I understand it is that migrating to >> Maven/Gradle makes the project more "attractive" for newcomers. If a project >> is built on "that old un-cool Ant", it might be a little bit off-putting and >> questionable if we are "stuck in the past on build systems and not >> progressing". >> >> So in that sense I agree this is more "marketing" rather than technological >> question but on the other hand, does not Maven/Gradle allow us to modularize >> the project better? Maybe we would like to modularize but nobody is up to >> that because build system makes it impossible or at least quite inconvenient >> to do so. Do you really think there are not any significant benefits to >> switch even if it "just works" now? >> >> >> From: Benedict >> Sent: Friday, November 25, 2022 11:07 >> To: dev@cassandra.apache.org >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >> >> NetApp Security WARNING: This is an external email. Do not click links or >> open attachments unless you recognize the sender and know the content is >> safe. >> >> >> >> >> There’s always a handful of people asking for it, but notably few if any of >>
Re: [DISCUSSION] Cassandra's code style and source code analysis
I think modules are already fairly well supported - we in effect already have several? FQL, Simulator and others I think. I can anyway say with absolute certainty that the main impediment to modularising is not build tooling, it’s the difficulty of the task, the cost to the project of undertaking it, and therefore its relative payoff versus other things that could be undertaken by the folk with the relevant expertise to do it. I’ve also been around long enough to see enough efforts to broaden contributions fail to have an impact, that basing costly decisions on this kind of reasoning doesn’t resonate. The main impediments to contributions are the complexity of the codebase (and problem domain) and our limited capacity to respond promptly to high quality contributions. Until we fix those fundamental issues, this kind of rearranging of chairs seems more like a branding exercise to ourselves than to anyone else. > On 25 Nov 2022, at 10:22, Miklosovic, Stefan > wrote: > > I agree with what you wrote. How I understand it is that migrating to > Maven/Gradle makes the project more "attractive" for newcomers. If a project > is built on "that old un-cool Ant", it might be a little bit off-putting and > questionable if we are "stuck in the past on build systems and not > progressing". > > So in that sense I agree this is more "marketing" rather than technological > question but on the other hand, does not Maven/Gradle allow us to modularize > the project better? Maybe we would like to modularize but nobody is up to > that because build system makes it impossible or at least quite inconvenient > to do so. Do you really think there are not any significant benefits to > switch even if it "just works" now? > > > From: Benedict > Sent: Friday, November 25, 2022 11:07 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > > > There’s always a handful of people asking for it, but notably few if any of > the full time contributors doing the majority of the core development of > Cassandra. It strikes me as something very appealing to others, but less so > to those wanting to get on with development. > > I never really see a good argument articulated for the migration, besides > general hand waving that ant is old, and people like newer build systems. Ant > is working fine, so there isn’t a strong technical reason to replace it, and > there are good organisational reasons not to. > > Why do you consider a migration inevitable? > > > >> On 25 Nov 2022, at 09:58, Miklosovic, Stefan >> wrote: >> >> Interesting take on Ant / no-Ant, Benedict. I am very curious how this >> unfolds. My long-term perception is that changing it to something else is >> more or less inevitable but if there is a broader consensus to not do that >> .... well. >> >> >> From: Benedict >> Sent: Friday, November 25, 2022 10:52 >> To: dev@cassandra.apache.org >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >> >> NetApp Security WARNING: This is an external email. Do not click links or >> open attachments unless you recognize the sender and know the content is >> safe. >> >> >> >> >> I was in a bit of a rush last night. I should say that I’m of course +1 a >> general endeavour to clean this up, and to expand our use of linters, and I >> appreciate your volunteering to help out in this way Maxim. >> >> However, responding to Stefan, I’m pretty -1 migrating from ant to another >> build system without really good reason. Migration has a real cost to >> productivity for all existing contributors, and the phantom of increasing >> new contributions has never paid off historically. I’m all for easing people >> into participation, but not at penalty to the existing contributor base. >> >> If the only reason is to make it easier to open in a different IDE, we can >> perhaps have some basic build files outlining code structure for importing, >> that are compatible with our canonical ant build? We could perhaps even >> generate them. >> >> >>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan >>>> wrote: >>> >>> For the record, I was testing that same combo Claude mentioned and it did >>> not work out of the box but it is definit
Re: [DISCUSSION] Cassandra's code style and source code analysis
I agree with what you wrote. How I understand it is that migrating to Maven/Gradle makes the project more "attractive" for newcomers. If a project is built on "that old un-cool Ant", it might be a little bit off-putting and questionable if we are "stuck in the past on build systems and not progressing". So in that sense I agree this is more "marketing" rather than technological question but on the other hand, does not Maven/Gradle allow us to modularize the project better? Maybe we would like to modularize but nobody is up to that because build system makes it impossible or at least quite inconvenient to do so. Do you really think there are not any significant benefits to switch even if it "just works" now? From: Benedict Sent: Friday, November 25, 2022 11:07 To: dev@cassandra.apache.org Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development. I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to. Why do you consider a migration inevitable? > On 25 Nov 2022, at 09:58, Miklosovic, Stefan > wrote: > > Interesting take on Ant / no-Ant, Benedict. I am very curious how this > unfolds. My long-term perception is that changing it to something else is > more or less inevitable but if there is a broader consensus to not do that > well. > > > From: Benedict > Sent: Friday, November 25, 2022 10:52 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > > > I was in a bit of a rush last night. I should say that I’m of course +1 a > general endeavour to clean this up, and to expand our use of linters, and I > appreciate your volunteering to help out in this way Maxim. > > However, responding to Stefan, I’m pretty -1 migrating from ant to another > build system without really good reason. Migration has a real cost to > productivity for all existing contributors, and the phantom of increasing new > contributions has never paid off historically. I’m all for easing people into > participation, but not at penalty to the existing contributor base. > > If the only reason is to make it easier to open in a different IDE, we can > perhaps have some basic build files outlining code structure for importing, > that are compatible with our canonical ant build? We could perhaps even > generate them. > > >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan >> wrote: >> >> For the record, I was testing that same combo Claude mentioned and it did >> not work out of the box but it is definitely possible to set up >> successfully. I do not remember the details. >> >> To replay to Maxim, it all seems good to me, roughly, but I humbly think it >> all boils down to Maven/Gradle refactoring and on top of that we can do all >> else. >> >> For example, there is (1) where the solution, besides fixing the tests, is >> to introduce an Ant task which would check this on build. That being said, >> how is that going to look like when we change Ant for something else? That >> stuff suddenly becomes obsolete. >> >> This case maybe applies to other problems we want to solve as well. I do not >> want to do something tailored for one build system just to rewrite it all or >> to spend significant amount of time on that again when we switch the build >> system. >> >> For that reason I think changing Ant for something else should be top >> priority (as I understand that it the hot topic for community for very long >> time) and then everything else should follow. We should spend time on things >> mentioned only in case they do not collide with any build system at all. >> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964 >> >> Stefan >> >> >> From: Claude W
Re: [DISCUSSION] Cassandra's code style and source code analysis
I should say, if a majority of the core developers are secretly really keen to see one of the other build systems takeover, I won’t stand in the way. But my preference is not to take the productivity hit, and I know many of the core developers prefer to avoid active involvement on list, and I suspect most are at best ambivalent to this. So without evidence of strong support for the migration from those folk, I’ll continue to voice my concerns as one of the more vocal ones. > On 25 Nov 2022, at 10:07, Benedict wrote: > > There’s always a handful of people asking for it, but notably few if any of > the full time contributors doing the majority of the core development of > Cassandra. It strikes me as something very appealing to others, but less so > to those wanting to get on with development. > > I never really see a good argument articulated for the migration, besides > general hand waving that ant is old, and people like newer build systems. Ant > is working fine, so there isn’t a strong technical reason to replace it, and > there are good organisational reasons not to. > > Why do you consider a migration inevitable? > > > >> On 25 Nov 2022, at 09:58, Miklosovic, Stefan >> wrote: >> >> Interesting take on Ant / no-Ant, Benedict. I am very curious how this >> unfolds. My long-term perception is that changing it to something else is >> more or less inevitable but if there is a broader consensus to not do that >> well. >> >> >> From: Benedict >> Sent: Friday, November 25, 2022 10:52 >> To: dev@cassandra.apache.org >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >> >> NetApp Security WARNING: This is an external email. Do not click links or >> open attachments unless you recognize the sender and know the content is >> safe. >> >> >> >> >> I was in a bit of a rush last night. I should say that I’m of course +1 a >> general endeavour to clean this up, and to expand our use of linters, and I >> appreciate your volunteering to help out in this way Maxim. >> >> However, responding to Stefan, I’m pretty -1 migrating from ant to another >> build system without really good reason. Migration has a real cost to >> productivity for all existing contributors, and the phantom of increasing >> new contributions has never paid off historically. I’m all for easing people >> into participation, but not at penalty to the existing contributor base. >> >> If the only reason is to make it easier to open in a different IDE, we can >> perhaps have some basic build files outlining code structure for importing, >> that are compatible with our canonical ant build? We could perhaps even >> generate them. >> >> >>>> On 25 Nov 2022, at 09:35, Miklosovic, Stefan >>>> wrote: >>> >>> For the record, I was testing that same combo Claude mentioned and it did >>> not work out of the box but it is definitely possible to set up >>> successfully. I do not remember the details. >>> >>> To replay to Maxim, it all seems good to me, roughly, but I humbly think it >>> all boils down to Maven/Gradle refactoring and on top of that we can do all >>> else. >>> >>> For example, there is (1) where the solution, besides fixing the tests, is >>> to introduce an Ant task which would check this on build. That being said, >>> how is that going to look like when we change Ant for something else? That >>> stuff suddenly becomes obsolete. >>> >>> This case maybe applies to other problems we want to solve as well. I do >>> not want to do something tailored for one build system just to rewrite it >>> all or to spend significant amount of time on that again when we switch the >>> build system. >>> >>> For that reason I think changing Ant for something else should be top >>> priority (as I understand that it the hot topic for community for very long >>> time) and then everything else should follow. We should spend time on >>> things mentioned only in case they do not collide with any build system at >>> all. >>> >>> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964 >>> >>> Stefan >>> >>> >>> From: Claude Warren, Jr via dev >>> Sent: Friday, November 25, 2022 10:16 >>> To: dev@cassandra.apache.org >>> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >
Re: [DISCUSSION] Cassandra's code style and source code analysis
There’s always a handful of people asking for it, but notably few if any of the full time contributors doing the majority of the core development of Cassandra. It strikes me as something very appealing to others, but less so to those wanting to get on with development. I never really see a good argument articulated for the migration, besides general hand waving that ant is old, and people like newer build systems. Ant is working fine, so there isn’t a strong technical reason to replace it, and there are good organisational reasons not to. Why do you consider a migration inevitable? > On 25 Nov 2022, at 09:58, Miklosovic, Stefan > wrote: > > Interesting take on Ant / no-Ant, Benedict. I am very curious how this > unfolds. My long-term perception is that changing it to something else is > more or less inevitable but if there is a broader consensus to not do that > well. > > > From: Benedict > Sent: Friday, November 25, 2022 10:52 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > > > I was in a bit of a rush last night. I should say that I’m of course +1 a > general endeavour to clean this up, and to expand our use of linters, and I > appreciate your volunteering to help out in this way Maxim. > > However, responding to Stefan, I’m pretty -1 migrating from ant to another > build system without really good reason. Migration has a real cost to > productivity for all existing contributors, and the phantom of increasing new > contributions has never paid off historically. I’m all for easing people into > participation, but not at penalty to the existing contributor base. > > If the only reason is to make it easier to open in a different IDE, we can > perhaps have some basic build files outlining code structure for importing, > that are compatible with our canonical ant build? We could perhaps even > generate them. > > >> On 25 Nov 2022, at 09:35, Miklosovic, Stefan >> wrote: >> >> For the record, I was testing that same combo Claude mentioned and it did >> not work out of the box but it is definitely possible to set up >> successfully. I do not remember the details. >> >> To replay to Maxim, it all seems good to me, roughly, but I humbly think it >> all boils down to Maven/Gradle refactoring and on top of that we can do all >> else. >> >> For example, there is (1) where the solution, besides fixing the tests, is >> to introduce an Ant task which would check this on build. That being said, >> how is that going to look like when we change Ant for something else? That >> stuff suddenly becomes obsolete. >> >> This case maybe applies to other problems we want to solve as well. I do not >> want to do something tailored for one build system just to rewrite it all or >> to spend significant amount of time on that again when we switch the build >> system. >> >> For that reason I think changing Ant for something else should be top >> priority (as I understand that it the hot topic for community for very long >> time) and then everything else should follow. We should spend time on things >> mentioned only in case they do not collide with any build system at all. >> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-17964 >> >> Stefan >> >> >> From: Claude Warren, Jr via dev >> Sent: Friday, November 25, 2022 10:16 >> To: dev@cassandra.apache.org >> Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis >> >> NetApp Security WARNING: This is an external email. Do not click links or >> open attachments unless you recognize the sender and know the content is >> safe. >> >> >> >> +1 for the concept as a whole. I am certain I could find nits to pick if I >> looked deeply. >> >> @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath). >> I gave up and am spending time trying to learn IntelliJ. I also mentioned >> it in one of the discussion areas. >> >> Claude >> >> On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever >> mailto:m...@apache.org>> wrote: >> Thank you for a solid write up Maxim. And welcome to Cassandra, it's >> very positive to see you here. >> >> I whole-heartedly agree with nearly everything you write. Some input >> and q
Re: [DISCUSSION] Cassandra's code style and source code analysis
Interesting take on Ant / no-Ant, Benedict. I am very curious how this unfolds. My long-term perception is that changing it to something else is more or less inevitable but if there is a broader consensus to not do that well. From: Benedict Sent: Friday, November 25, 2022 10:52 To: dev@cassandra.apache.org Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim. However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base. If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them. > On 25 Nov 2022, at 09:35, Miklosovic, Stefan > wrote: > > For the record, I was testing that same combo Claude mentioned and it did > not work out of the box but it is definitely possible to set up successfully. > I do not remember the details. > > To replay to Maxim, it all seems good to me, roughly, but I humbly think it > all boils down to Maven/Gradle refactoring and on top of that we can do all > else. > > For example, there is (1) where the solution, besides fixing the tests, is to > introduce an Ant task which would check this on build. That being said, how > is that going to look like when we change Ant for something else? That stuff > suddenly becomes obsolete. > > This case maybe applies to other problems we want to solve as well. I do not > want to do something tailored for one build system just to rewrite it all or > to spend significant amount of time on that again when we switch the build > system. > > For that reason I think changing Ant for something else should be top > priority (as I understand that it the hot topic for community for very long > time) and then everything else should follow. We should spend time on things > mentioned only in case they do not collide with any build system at all. > > (1) https://issues.apache.org/jira/browse/CASSANDRA-17964 > > Stefan > > > From: Claude Warren, Jr via dev > Sent: Friday, November 25, 2022 10:16 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > > +1 for the concept as a whole. I am certain I could find nits to pick if I > looked deeply. > > @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath). > I gave up and am spending time trying to learn IntelliJ. I also mentioned it > in one of the discussion areas. > > Claude > > On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever > mailto:m...@apache.org>> wrote: > Thank you for a solid write up Maxim. And welcome to Cassandra, it's > very positive to see you here. > > I whole-heartedly agree with nearly everything you write. Some input > and questions inline. > > >> >> As you may know, the infrastructure team has disabled public sign-up >> to ASF JIRA (the GitHub issues are recommended instead). >> > > > I suspect (based on chatter in general, but not on dev@ AFAIK) is to > avoid GH issues and stick with jira. The sign-up hurdle we will > document on the website: CASSANDRA-18064 > > > >> == 1. Make the checkstyle config a single point of truth for the >> source code style. == >> >> The checkstyle is already used and included in the Cassandra project >> build lifecycle (ant command line, Jenkins, CircleCI). There is no >> need to maintain code style configurations for different types of IDEs >> (e.g. IntelliJ inspections configuration) since the checkstyle.xml >> file can be directly imported to IDE used by a developer. This is fair >> for Intellij Idea, NetBeans, and Eclipse. > > > Big +1 > > >> So, I propose to focus on the checks themselves and checking pull >> requests with automation s
Re: [DISCUSSION] Cassandra's code style and source code analysis
I was in a bit of a rush last night. I should say that I’m of course +1 a general endeavour to clean this up, and to expand our use of linters, and I appreciate your volunteering to help out in this way Maxim. However, responding to Stefan, I’m pretty -1 migrating from ant to another build system without really good reason. Migration has a real cost to productivity for all existing contributors, and the phantom of increasing new contributions has never paid off historically. I’m all for easing people into participation, but not at penalty to the existing contributor base. If the only reason is to make it easier to open in a different IDE, we can perhaps have some basic build files outlining code structure for importing, that are compatible with our canonical ant build? We could perhaps even generate them. > On 25 Nov 2022, at 09:35, Miklosovic, Stefan > wrote: > > For the record, I was testing that same combo Claude mentioned and it did > not work out of the box but it is definitely possible to set up successfully. > I do not remember the details. > > To replay to Maxim, it all seems good to me, roughly, but I humbly think it > all boils down to Maven/Gradle refactoring and on top of that we can do all > else. > > For example, there is (1) where the solution, besides fixing the tests, is to > introduce an Ant task which would check this on build. That being said, how > is that going to look like when we change Ant for something else? That stuff > suddenly becomes obsolete. > > This case maybe applies to other problems we want to solve as well. I do not > want to do something tailored for one build system just to rewrite it all or > to spend significant amount of time on that again when we switch the build > system. > > For that reason I think changing Ant for something else should be top > priority (as I understand that it the hot topic for community for very long > time) and then everything else should follow. We should spend time on things > mentioned only in case they do not collide with any build system at all. > > (1) https://issues.apache.org/jira/browse/CASSANDRA-17964 > > Stefan > > > From: Claude Warren, Jr via dev > Sent: Friday, November 25, 2022 10:16 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > > +1 for the concept as a whole. I am certain I could find nits to pick if I > looked deeply. > > @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath). > I gave up and am spending time trying to learn IntelliJ. I also mentioned it > in one of the discussion areas. > > Claude > > On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever > mailto:m...@apache.org>> wrote: > Thank you for a solid write up Maxim. And welcome to Cassandra, it's > very positive to see you here. > > I whole-heartedly agree with nearly everything you write. Some input > and questions inline. > > >> >> As you may know, the infrastructure team has disabled public sign-up >> to ASF JIRA (the GitHub issues are recommended instead). >> > > > I suspect (based on chatter in general, but not on dev@ AFAIK) is to > avoid GH issues and stick with jira. The sign-up hurdle we will > document on the website: CASSANDRA-18064 > > > >> == 1. Make the checkstyle config a single point of truth for the >> source code style. == >> >> The checkstyle is already used and included in the Cassandra project >> build lifecycle (ant command line, Jenkins, CircleCI). There is no >> need to maintain code style configurations for different types of IDEs >> (e.g. IntelliJ inspections configuration) since the checkstyle.xml >> file can be directly imported to IDE used by a developer. This is fair >> for Intellij Idea, NetBeans, and Eclipse. > > > Big +1 > > >> So, I propose to focus on the checks themselves and checking pull >> requests with automation scripts, rather than maintaining these >> integrations. The benefits here are avoiding all issues with >> maintaining configurations for different IDEs. Another good advantage >> of this approach would be the ability to add new checkstyle rules >> without touching IDE configuration - and such tickets will be LFH and >> easy to commit. >> >> The actions points here are: >> >> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8] >> (or label checkstyle); >> - add checkstyle to GitHub
Re: [DISCUSSION] Cassandra's code style and source code analysis
For the record, I was testing that same combo Claude mentioned and it did not work out of the box but it is definitely possible to set up successfully. I do not remember the details. To replay to Maxim, it all seems good to me, roughly, but I humbly think it all boils down to Maven/Gradle refactoring and on top of that we can do all else. For example, there is (1) where the solution, besides fixing the tests, is to introduce an Ant task which would check this on build. That being said, how is that going to look like when we change Ant for something else? That stuff suddenly becomes obsolete. This case maybe applies to other problems we want to solve as well. I do not want to do something tailored for one build system just to rewrite it all or to spend significant amount of time on that again when we switch the build system. For that reason I think changing Ant for something else should be top priority (as I understand that it the hot topic for community for very long time) and then everything else should follow. We should spend time on things mentioned only in case they do not collide with any build system at all. (1) https://issues.apache.org/jira/browse/CASSANDRA-17964 Stefan From: Claude Warren, Jr via dev Sent: Friday, November 25, 2022 10:16 To: dev@cassandra.apache.org Subject: Re: [DISCUSSION] Cassandra's code style and source code analysis NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. +1 for the concept as a whole. I am certain I could find nits to pick if I looked deeply. @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath). I gave up and am spending time trying to learn IntelliJ. I also mentioned it in one of the discussion areas. Claude On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever mailto:m...@apache.org>> wrote: Thank you for a solid write up Maxim. And welcome to Cassandra, it's very positive to see you here. I whole-heartedly agree with nearly everything you write. Some input and questions inline. > > As you may know, the infrastructure team has disabled public sign-up > to ASF JIRA (the GitHub issues are recommended instead). > I suspect (based on chatter in general, but not on dev@ AFAIK) is to avoid GH issues and stick with jira. The sign-up hurdle we will document on the website: CASSANDRA-18064 > == 1. Make the checkstyle config a single point of truth for the > source code style. == > > The checkstyle is already used and included in the Cassandra project > build lifecycle (ant command line, Jenkins, CircleCI). There is no > need to maintain code style configurations for different types of IDEs > (e.g. IntelliJ inspections configuration) since the checkstyle.xml > file can be directly imported to IDE used by a developer. This is fair > for Intellij Idea, NetBeans, and Eclipse. Big +1 > So, I propose to focus on the checks themselves and checking pull > requests with automation scripts, rather than maintaining these > integrations. The benefits here are avoiding all issues with > maintaining configurations for different IDEs. Another good advantage > of this approach would be the ability to add new checkstyle rules > without touching IDE configuration - and such tickets will be LFH and > easy to commit. > > The actions points here are: > > - create an umbrella JIRA ticket for all checkstyle issues e.g. [8] > (or label checkstyle); > - add checkstyle to GitHub pull requests using GitHub actions (execute > ant command); Instead of custom GHA scripting, please use our existing cassandra-artifact.sh (which should already include all such checks). Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11 > == 3. Enable pushing backwards build results for both Jenkins and > CircleCI to GitHub pull requests. == > > The goal here is to have a "green checkbox" for those GitHub pull > requests that have corresponding Jenkins/CircleCI runs. According to > the following links, it is completely possible to have those. > > https://github.com/jenkinsci/github-branch-source-plugin > https://circleci.com/docs/enable-checks/ > > Moreover, the GitHub Branch Source Plugin is already enabled for the > Cassandra project [16]. The same seems should work the same way for > CirleCI, but I have faced the infrastructure team comment [17] that > describes admin permissions are required for that, so the question is > still open here. I could dig a bit more once we've agreed on it. > > The actions points here are: > - enable Jenkins integration for GitHub pull requests; > - enable CircleCI integration for GitHub pull requests; Some folk use CircleCI, some use ci-cassandra. T
Re: [DISCUSSION] Cassandra's code style and source code analysis
+1 for the concept as a whole. I am certain I could find nits to pick if I looked deeply. @mck -- I did have a problem with Cassandra + Eclipse + Java11 (Classpath). I gave up and am spending time trying to learn IntelliJ. I also mentioned it in one of the discussion areas. Claude On Thu, Nov 24, 2022 at 8:55 PM Mick Semb Wever wrote: > Thank you for a solid write up Maxim. And welcome to Cassandra, it's > very positive to see you here. > > I whole-heartedly agree with nearly everything you write. Some input > and questions inline. > > > > > > As you may know, the infrastructure team has disabled public sign-up > > to ASF JIRA (the GitHub issues are recommended instead). > > > > > I suspect (based on chatter in general, but not on dev@ AFAIK) is to > avoid GH issues and stick with jira. The sign-up hurdle we will > document on the website: CASSANDRA-18064 > > > > > == 1. Make the checkstyle config a single point of truth for the > > source code style. == > > > > The checkstyle is already used and included in the Cassandra project > > build lifecycle (ant command line, Jenkins, CircleCI). There is no > > need to maintain code style configurations for different types of IDEs > > (e.g. IntelliJ inspections configuration) since the checkstyle.xml > > file can be directly imported to IDE used by a developer. This is fair > > for Intellij Idea, NetBeans, and Eclipse. > > > Big +1 > > > > So, I propose to focus on the checks themselves and checking pull > > requests with automation scripts, rather than maintaining these > > integrations. The benefits here are avoiding all issues with > > maintaining configurations for different IDEs. Another good advantage > > of this approach would be the ability to add new checkstyle rules > > without touching IDE configuration - and such tickets will be LFH and > > easy to commit. > > > > The actions points here are: > > > > - create an umbrella JIRA ticket for all checkstyle issues e.g. [8] > > (or label checkstyle); > > - add checkstyle to GitHub pull requests using GitHub actions (execute > > ant command); > > > Instead of custom GHA scripting, please use our existing > cassandra-artifact.sh (which should already include all such checks). > > Something like > https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11 > > > > > == 3. Enable pushing backwards build results for both Jenkins and > > CircleCI to GitHub pull requests. == > > > > The goal here is to have a "green checkbox" for those GitHub pull > > requests that have corresponding Jenkins/CircleCI runs. According to > > the following links, it is completely possible to have those. > > > > https://github.com/jenkinsci/github-branch-source-plugin > > https://circleci.com/docs/enable-checks/ > > > > Moreover, the GitHub Branch Source Plugin is already enabled for the > > Cassandra project [16]. The same seems should work the same way for > > CirleCI, but I have faced the infrastructure team comment [17] that > > describes admin permissions are required for that, so the question is > > still open here. I could dig a bit more once we've agreed on it. > > > > The actions points here are: > > - enable Jenkins integration for GitHub pull requests; > > - enable CircleCI integration for GitHub pull requests; > > > Some folk use CircleCI, some use ci-cassandra. The green checkbox idea > is great, so long as it's optional. We don't want PRs triggering the > runs either (there are other mechanisms for triggering for now). > > > > The actions points here are: > > > > - initiate a wide survey for user and dev lists, to get to know about > > the usages; > > - remove those configurations that are not used anymore; > > - force migration from Ant to Gradle/Maven; > > > Let's leave this out for now. There's too many unknowns here. If > there's an IDE configuration that's broken, no one has reported it for > ages, and no one is around to fix it, then I say we should raise the > discussion to remove it. > > The Gradle/Maven migration is a hot one, contribute to that discussion > but let's not tangle this work up with it, IMHO. > > Totally agree that IDE project files should be as light weight as possible. > > > > Summarizing everything proposed above I think it is possible to > > simplify adding small contributions easier to the codebase as well as > > save a bunch of committer's time. > > > > So, > > WDYT about the things described above? > > Should I create a CEP for this? > > > I see no need for a CEP here. An epic and tickets will work. > Again, thanks for the input Maxim! >
Re: [DISCUSSION] Cassandra's code style and source code analysis
> == 1. Make the checkstyle config a single point of truth for the > source code style. == -1, the single point of truth is the wiki - everything else follows this. The code style is a community consensus, and as previously discussed the last time it came up, mostly not strict. If we intend to impose hard checks here that reject patches we will need to debate carefully which elements of the code style are to become hard rules. > There is no > need to maintain code style configurations for different types of IDEs > (e.g. IntelliJ inspections configuration) since the checkstyle.xml > file can be directly imported to IDE used by a developer. This is fair >> >> for Intellij Idea, NetBeans, and Eclipse. Does this update IntelliJ’s own linter, or does a plugin run check style over the codebase? I couldn’t find a native import, and the latter is probably a downgrade for most active developers. >> == 2. Add additional tooling for code analysis to the build and GitHub >> pull requests. == I’m generally in favour of carefully expanding our use of additional linters, but we should activate new lints very cautiously, as not all lints are equally good. I think there needs to be a lot of community consensus on the broad expansion of lints that can reject patches. > On 24 Nov 2022, at 20:56, Mick Semb Wever wrote: > > Thank you for a solid write up Maxim. And welcome to Cassandra, it's > very positive to see you here. > > I whole-heartedly agree with nearly everything you write. Some input > and questions inline. > > >> >> As you may know, the infrastructure team has disabled public sign-up >> to ASF JIRA (the GitHub issues are recommended instead). >> > > > I suspect (based on chatter in general, but not on dev@ AFAIK) is to > avoid GH issues and stick with jira. The sign-up hurdle we will > document on the website: CASSANDRA-18064 > > > >> == 1. Make the checkstyle config a single point of truth for the >> source code style. == >> >> The checkstyle is already used and included in the Cassandra project >> build lifecycle (ant command line, Jenkins, CircleCI). There is no >> need to maintain code style configurations for different types of IDEs >> (e.g. IntelliJ inspections configuration) since the checkstyle.xml >> file can be directly imported to IDE used by a developer. This is fair >> for Intellij Idea, NetBeans, and Eclipse. > > > Big +1 > > >> So, I propose to focus on the checks themselves and checking pull >> requests with automation scripts, rather than maintaining these >> integrations. The benefits here are avoiding all issues with >> maintaining configurations for different IDEs. Another good advantage >> of this approach would be the ability to add new checkstyle rules >> without touching IDE configuration - and such tickets will be LFH and >> easy to commit. >> >> The actions points here are: >> >> - create an umbrella JIRA ticket for all checkstyle issues e.g. [8] >> (or label checkstyle); >> - add checkstyle to GitHub pull requests using GitHub actions (execute >> ant command); > > > Instead of custom GHA scripting, please use our existing > cassandra-artifact.sh (which should already include all such checks). > > Something like > https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11 > > > >> == 3. Enable pushing backwards build results for both Jenkins and >> CircleCI to GitHub pull requests. == >> >> The goal here is to have a "green checkbox" for those GitHub pull >> requests that have corresponding Jenkins/CircleCI runs. According to >> the following links, it is completely possible to have those. >> >> https://github.com/jenkinsci/github-branch-source-plugin >> https://circleci.com/docs/enable-checks/ >> >> Moreover, the GitHub Branch Source Plugin is already enabled for the >> Cassandra project [16]. The same seems should work the same way for >> CirleCI, but I have faced the infrastructure team comment [17] that >> describes admin permissions are required for that, so the question is >> still open here. I could dig a bit more once we've agreed on it. >> >> The actions points here are: >> - enable Jenkins integration for GitHub pull requests; >> - enable CircleCI integration for GitHub pull requests; > > > Some folk use CircleCI, some use ci-cassandra. The green checkbox idea > is great, so long as it's optional. We don't want PRs triggering the > runs either (there are other mechanisms for triggering for now). > > >> The actions points here are: >> >> - initiate a wide survey for user and dev lists, to get to know about >> the usages; >> - remove those configurations that are not used anymore; >> - force migration from Ant to Gradle/Maven; > > > Let's leave this out for now. There's too many unknowns here. If > there's an IDE configuration that's broken, no one has reported it for > ages, and no one is around to fix it, then I say we should raise the > discussion to remove it. > > The
Re: [DISCUSSION] Cassandra's code style and source code analysis
Thank you for a solid write up Maxim. And welcome to Cassandra, it's very positive to see you here. I whole-heartedly agree with nearly everything you write. Some input and questions inline. > > As you may know, the infrastructure team has disabled public sign-up > to ASF JIRA (the GitHub issues are recommended instead). > I suspect (based on chatter in general, but not on dev@ AFAIK) is to avoid GH issues and stick with jira. The sign-up hurdle we will document on the website: CASSANDRA-18064 > == 1. Make the checkstyle config a single point of truth for the > source code style. == > > The checkstyle is already used and included in the Cassandra project > build lifecycle (ant command line, Jenkins, CircleCI). There is no > need to maintain code style configurations for different types of IDEs > (e.g. IntelliJ inspections configuration) since the checkstyle.xml > file can be directly imported to IDE used by a developer. This is fair > for Intellij Idea, NetBeans, and Eclipse. Big +1 > So, I propose to focus on the checks themselves and checking pull > requests with automation scripts, rather than maintaining these > integrations. The benefits here are avoiding all issues with > maintaining configurations for different IDEs. Another good advantage > of this approach would be the ability to add new checkstyle rules > without touching IDE configuration - and such tickets will be LFH and > easy to commit. > > The actions points here are: > > - create an umbrella JIRA ticket for all checkstyle issues e.g. [8] > (or label checkstyle); > - add checkstyle to GitHub pull requests using GitHub actions (execute > ant command); Instead of custom GHA scripting, please use our existing cassandra-artifact.sh (which should already include all such checks). Something like https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:cassandra:mck/github-actions/3.11 > == 3. Enable pushing backwards build results for both Jenkins and > CircleCI to GitHub pull requests. == > > The goal here is to have a "green checkbox" for those GitHub pull > requests that have corresponding Jenkins/CircleCI runs. According to > the following links, it is completely possible to have those. > > https://github.com/jenkinsci/github-branch-source-plugin > https://circleci.com/docs/enable-checks/ > > Moreover, the GitHub Branch Source Plugin is already enabled for the > Cassandra project [16]. The same seems should work the same way for > CirleCI, but I have faced the infrastructure team comment [17] that > describes admin permissions are required for that, so the question is > still open here. I could dig a bit more once we've agreed on it. > > The actions points here are: > - enable Jenkins integration for GitHub pull requests; > - enable CircleCI integration for GitHub pull requests; Some folk use CircleCI, some use ci-cassandra. The green checkbox idea is great, so long as it's optional. We don't want PRs triggering the runs either (there are other mechanisms for triggering for now). > The actions points here are: > > - initiate a wide survey for user and dev lists, to get to know about > the usages; > - remove those configurations that are not used anymore; > - force migration from Ant to Gradle/Maven; Let's leave this out for now. There's too many unknowns here. If there's an IDE configuration that's broken, no one has reported it for ages, and no one is around to fix it, then I say we should raise the discussion to remove it. The Gradle/Maven migration is a hot one, contribute to that discussion but let's not tangle this work up with it, IMHO. Totally agree that IDE project files should be as light weight as possible. > Summarizing everything proposed above I think it is possible to > simplify adding small contributions easier to the codebase as well as > save a bunch of committer's time. > > So, > WDYT about the things described above? > Should I create a CEP for this? I see no need for a CEP here. An epic and tickets will work. Again, thanks for the input Maxim!
[DISCUSSION] Cassandra's code style and source code analysis
Hello everyone, First of all, thank you all for this awesome project which I have often been inspired by. My name is Maxim Muzafarov I'm a Committer and PMC of Apache Ignite hence you most likely don't know me as I come from another part of the ASF. Perhaps, I did almost the same things with B-Trees, off-heap memory management, rebalancing, checkpointing, snapshotting, and IEPs (you are calling it CEPs) but on a slightly different distributed database architecture. Nevertheless, I was chasing down for my first issue to get experience with Cassandra and found a bunch of opened JIRAs related to the source code analysis (static analysis as well as the code style). These issues still appear in JIRA from time to time [1][2][3][4]. It seems to me there not enough attention has been paid to this topic and all possible options for this analysis and code style haven't been widely discussed before. I'd like to summarize everything that I have found and offer my skills and my experience for solving some of such issues we'll agree on. = Motivation = The goal is to make small contributions easier and safer to apply with GitHub PRs for both a contributor and a committer by adding automation code checks for each new Cassandra contribution. This also will help to reduce the time required for reviewing and applying such PRs by an experienced developer. As you may know, the infrastructure team has disabled public sign-up to ASF JIRA (the GitHub issues are recommended instead). Thus the following things become more important if we are still interested in attracting new contributions as it was discussed earlier [6]. I do not want to add extra steps to the existing workflow with code review or make GitHub pull requests as default for patches as it also was discussed already [7], just to improve the automation checks in it and make checks more convenient. = Proposed Solution = == 1. Make the checkstyle config a single point of truth for the source code style. == The checkstyle is already used and included in the Cassandra project build lifecycle (ant command line, Jenkins, CircleCI). There is no need to maintain code style configurations for different types of IDEs (e.g. IntelliJ inspections configuration) since the checkstyle.xml file can be directly imported to IDE used by a developer. This is fair for Intellij Idea, NetBeans, and Eclipse. So, I propose to focus on the checks themselves and checking pull requests with automation scripts, rather than maintaining these integrations. The benefits here are avoiding all issues with maintaining configurations for different IDEs. Another good advantage of this approach would be the ability to add new checkstyle rules without touching IDE configuration - and such tickets will be LFH and easy to commit. The actions points here are: - create an umbrella JIRA ticket for all checkstyle issues e.g. [8] (or label checkstyle); - add checkstyle to GitHub pull requests using GitHub actions (execute ant command); - include checkstyle to the build (already done); - remove redundant code style configurations related to IDEs from the source code e.g. [9]; == 2. Add additional tooling for code analysis to the build and GitHub pull requests. == The source code static analysis and automated checks have been discussed recently in the "SpotBugs to the build" topic [10]. I'd like to add my 50 cents here. Before discussing the pros and cons of each solution, let's focus on the criteria that such solutions must meet. You can find the most complete list of such tooling here [11]. >From my point of view, the crucial criteria are: - free for open-source (at least licenses should allow such usages); - popularity in the ASF ecosystems; - convenient integration and/or plugins for IDEs and GitHub; - we must be able to integrate with CirleCI, and Jenkins as well as launch from a command line; === Sonar === pros + this tool is free for open-source and recommended by the ASF infrastructure team [12]; + was already used for the Cassandra project some time ago at sonarcloud.io [13]; + it has GitHub pull requests analysis [14]; cons - run locally requires additional configuration and TOKEN_ID due to the analysis results stored in the ext database (infra will not provide it for local usage); === SpotBugs (FindBugs) === pros + license is allowed to use it and run it as a library (should be legal for us); + it analyses the bytecode that differs from how the checkstyle works; + can be executed from the command line as well as integrated into the build; cons - requires compiled source code; === PMD === pros + BSD licenses more permissive than LGPL (SpotBugs); + analyses the source code like the checkstyle does; + have an extended rule sets for source code analyses; cons - the checkstyle is already used in the project, and should be enough for now; === IntelliJ IDEA === pros + free for open-source and can be used from the command line [15]; + local checks are performed with the same code
