Re: [DISCUSS] putting versions into Deprecated annotations

[Francisco Guerrero]

To me this seems insufficient. As a developer, I'd like to see what the 
alternative is when reading the javadoc without having to go to Jira.

What I would prefer is to know what the alternative is and how to use it. For 
example:

/** @deprecated Use {@link #alternative} instead. See CASSANDRA-6504 */
@Deprecated(since = "2.1")
public Integer concurrent_replicates = null;

I am not sure if checkstyle can enforce the above, so the mechanisms to enforce 
it would still need to be laid out, unless we can easily support something like 
the above with checkstyle rules.

On 2023/10/10 20:34:27 Maxim Muzafarov wrote:
> Hello everyone,
> 
> 
> I've discussed with Stefan some steps we can take to improve the final
> solution, so the final version might look like this:
> 
> /** @deprecated See CASSANDRA-6504 */
> @Deprecated(since = "2.1")
> public Integer concurrent_replicates = null;
> 
> The issue number will be taken from the git blame comment. I doubt I
> can generate and/or create a meaningful comment for every deprecation
> annotation, but providing a link to the issue that was retrieved from
> the git blame is not too big a problem. This also improves the
> visibility.
> 
> In addition, we can add two checkstyle rules [1] [2] to ensure that
> any future deprecations will have a "since" element and a JavaDoc
> comment.
> WDYT?
> 
> [1] 
> https://checkstyle.sourceforge.io/apidocs/com/puppycrawl/tools/checkstyle/checks/annotation/MissingDeprecatedCheck.html
> [2] 
> https://checkstyle.org/apidocs/com/puppycrawl/tools/checkstyle/checks/coding/MatchXpathCheck.html
> 
> On Tue, 10 Oct 2023 at 14:50, Josh McKenzie  wrote:
> >
> > Sounds like we're relitigating the basics of how @Deprecated, forRemoval, 
> > since, and javadoc @link all intersect to make deprecation less painful ;)
> >
> > So:
> >
> > Built-in java.lang.Deprecated: required
> > Can use since and forRemoval if you have that info handy and think it'd be 
> > useful (would make it a lot easier to grep for things to pull before a 
> > major)
> > If it's being replaced by something, you should {@link #} the javadoc for 
> > it so people know where to bounce over to
> >
> > I've been leaning pretty heavily on the functionality of point 3 for 
> > documenting cross-module implicit dependencies as I come across them lately 
> > so that one resonates with me.
> >
> > On Tue, Oct 10, 2023, at 4:38 AM, Miklosovic, Stefan wrote:
> >
> > OK.
> >
> > Let's go with in-built java.lang.Deprecated annotation. If somebody wants 
> > to document that in more detail, there are Javadocs as mentioned. Let's 
> > just stick with the standard stuff.
> >
> > I will try to implement this for 5.0 (versions since it was deprecated) 
> > with my take on what should be removed (forRemoval = true) but that should 
> > be definitely cross-checked on review as Mick mentioned.
> >
> > 
> > From: Mick Semb Wever 
> > Sent: Monday, October 9, 2023 10:55
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSS] putting versions into Deprecated annotations
> >
> > NetApp Security WARNING: This is an external email. Do not click links or 
> > open attachments unless you recognize the sender and know the content is 
> > safe.
> >
> >
> >
> > Tangential question to this is if everything we deprecated is eligible for 
> > removal? In other words, are there any cases when forRemoval would be 
> > false? Could you elaborate on that and give such examples or do you all 
> > think that everything which is deprecated will be eventually removed?
> >
> >
> > Removal cannot be default.  This came up in the subtickets of 
> > CASSANDRA-18306.
> >
> > I suggest that adding " forRemoval = true" and the later actual removal of 
> > the code both require broader consensus.  I'm open to that being on the 
> > ticket or needing a thread on the ML.  Small stuff, common sense says on 
> > the ticket is enough, but a few folk have already stated that deprecated 
> > code that has minimal maintenance overhead should not be removed.
> >
> >
> 

Re: [DISCUSS] putting versions into Deprecated annotations

[Maxim Muzafarov]

Hello everyone,


I've discussed with Stefan some steps we can take to improve the final
solution, so the final version might look like this:

/** @deprecated See CASSANDRA-6504 */
@Deprecated(since = "2.1")
public Integer concurrent_replicates = null;

The issue number will be taken from the git blame comment. I doubt I
can generate and/or create a meaningful comment for every deprecation
annotation, but providing a link to the issue that was retrieved from
the git blame is not too big a problem. This also improves the
visibility.

In addition, we can add two checkstyle rules [1] [2] to ensure that
any future deprecations will have a "since" element and a JavaDoc
comment.
WDYT?

[1] 
https://checkstyle.sourceforge.io/apidocs/com/puppycrawl/tools/checkstyle/checks/annotation/MissingDeprecatedCheck.html
[2] 
https://checkstyle.org/apidocs/com/puppycrawl/tools/checkstyle/checks/coding/MatchXpathCheck.html

On Tue, 10 Oct 2023 at 14:50, Josh McKenzie  wrote:
>
> Sounds like we're relitigating the basics of how @Deprecated, forRemoval, 
> since, and javadoc @link all intersect to make deprecation less painful ;)
>
> So:
>
> Built-in java.lang.Deprecated: required
> Can use since and forRemoval if you have that info handy and think it'd be 
> useful (would make it a lot easier to grep for things to pull before a major)
> If it's being replaced by something, you should {@link #} the javadoc for it 
> so people know where to bounce over to
>
> I've been leaning pretty heavily on the functionality of point 3 for 
> documenting cross-module implicit dependencies as I come across them lately 
> so that one resonates with me.
>
> On Tue, Oct 10, 2023, at 4:38 AM, Miklosovic, Stefan wrote:
>
> OK.
>
> Let's go with in-built java.lang.Deprecated annotation. If somebody wants to 
> document that in more detail, there are Javadocs as mentioned. Let's just 
> stick with the standard stuff.
>
> I will try to implement this for 5.0 (versions since it was deprecated) with 
> my take on what should be removed (forRemoval = true) but that should be 
> definitely cross-checked on review as Mick mentioned.
>
> 
> From: Mick Semb Wever 
> Sent: Monday, October 9, 2023 10:55
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSS] putting versions into Deprecated annotations
>
> NetApp Security WARNING: This is an external email. Do not click links or 
> open attachments unless you recognize the sender and know the content is safe.
>
>
>
> Tangential question to this is if everything we deprecated is eligible for 
> removal? In other words, are there any cases when forRemoval would be false? 
> Could you elaborate on that and give such examples or do you all think that 
> everything which is deprecated will be eventually removed?
>
>
> Removal cannot be default.  This came up in the subtickets of CASSANDRA-18306.
>
> I suggest that adding " forRemoval = true" and the later actual removal of 
> the code both require broader consensus.  I'm open to that being on the 
> ticket or needing a thread on the ML.  Small stuff, common sense says on the 
> ticket is enough, but a few folk have already stated that deprecated code 
> that has minimal maintenance overhead should not be removed.
>
>

Re: Need Confluent "Create" permission for filing a CEP

[Jaydeep Chovatia]

Thank you!

On Tue, Oct 10, 2023 at 2:58 AM Brandon Williams  wrote:

> I've added you, you should have access now.
>
> Kind Regards,
> Brandon
>
> On Tue, Oct 10, 2023 at 1:24 AM Jaydeep Chovatia
>  wrote:
> >
> > Hi,
> >
> > I want to create a new CEP request but do not see the "Create" page
> permission on Confluent. Could someone permit me?
> > Here is the CEP draft: [DRAFT] CEP - Apache Cassandra Official Repair
> Solution - Google Docs
> >
> > My confluent user-id is: chovatia.jayd...@gmail.com
> >
> > Jaydeep
>

Re: Avoiding pushes to broken branches

[Josh McKenzie]

What about having a nag-bot that notifies #cassandra-dev on ASF slack hourly if 
the builds are broken?

On Tue, Oct 10, 2023, at 8:02 AM, Mick Semb Wever wrote:
> I'd like to suggest some improvements for identifying and announcing
> broken branches and to avoid pushing commits to broken branches.
> 
> For CI we should have two gates.
> 
> The first is pre-commit testing, which we have already discussed, e.g.
> either ci-cassandra or circleci can be used (and repeated tests are
> expected to be run on circleci).
> 
> The second gate is whether the branch the commit is being merged to is
> in a healthy state.  Our canonical CI system is ci-cassandra, and for
> post-commit health it is our only CI.  Last week ci-cassandra was
> broken on all branches from 4.0 up.  The cause was two reasons,
> neither our fault: debian packaging (CASSANDRA-18910) and xerces2 xml
> file processing OOM (cassandra-builds:8d11eea).
> 
> Knowing if a branch is broken (before pushing) is just to check
> ci-cassandra.apache.org
> 
> Folk have suggested this is not enough, and that a message to the dev@
> would also help, but part of the problem is that there's no one place
> that people check before pushing (there's no requirement on anyone to
> be keeping up to date with dev@ at all times).
> 
> To summarise, I feel we currently don't have good practices for
> - identifying and announcing a broken CI,
> - knowing who is investigating it,
> - labelling it with the cause,
> - knowing who is working on a fix
> 
> 
> The suggested actions I'm proposing for us all to adopt are:
> 
> 1. Before committing please check dev@ and ci-cassandra.a.o
> 2. If you see ci-cassandra is red on a branch, and no dev@ thread has
> been started, please start the dev@ thread and create the ticket,
> 3. Put the ticket id into the description of the first red build
> ("Add description")
> 4. By default, hold off on pushing to broken branches.
> 
> 
> WRT (2), we should just be able to send the automated build failures
> to dev@ instead of builds@, but failed builds are often not sending
> such notifications, so this isn't something we can rely on yet.
> 
> 
> Reference slack threads:
> - https://the-asf.slack.com/archives/CK23JSY2K/p1696693542832489
> - https://the-asf.slack.com/archives/CK23JSY2K/p1696599019480519
> - https://the-asf.slack.com/archives/CK23JSY2K/p1696351208371029
> - https://the-asf.slack.com/archives/CK23JSY2K/p1695878499669699
> 

Re: [DISCUSS] putting versions into Deprecated annotations

[Josh McKenzie]

Sounds like we're relitigating the basics of how @Deprecated, forRemoval, 
since, and javadoc @link all intersect to make deprecation less painful ;)

So:
 1. Built-in java.lang.Deprecated: required
 2. Can use since and forRemoval if you have that info handy and think it'd be 
useful (would make it a lot easier to grep for things to pull before a major)
 3. If it's being replaced by something, you should {@link #} the javadoc for 
it so people know where to bounce over to
I've been leaning pretty heavily on the functionality of point 3 for 
documenting cross-module implicit dependencies as I come across them lately so 
that one resonates with me.

On Tue, Oct 10, 2023, at 4:38 AM, Miklosovic, Stefan wrote:
> OK.
> 
> Let's go with in-built java.lang.Deprecated annotation. If somebody wants to 
> document that in more detail, there are Javadocs as mentioned. Let's just 
> stick with the standard stuff.
> 
> I will try to implement this for 5.0 (versions since it was deprecated) with 
> my take on what should be removed (forRemoval = true) but that should be 
> definitely cross-checked on review as Mick mentioned.
> 
> 
> From: Mick Semb Wever 
> Sent: Monday, October 9, 2023 10:55
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSS] putting versions into Deprecated annotations
> 
> NetApp Security WARNING: This is an external email. Do not click links or 
> open attachments unless you recognize the sender and know the content is safe.
> 
> 
> 
> Tangential question to this is if everything we deprecated is eligible for 
> removal? In other words, are there any cases when forRemoval would be false? 
> Could you elaborate on that and give such examples or do you all think that 
> everything which is deprecated will be eventually removed?
> 
> 
> Removal cannot be default.  This came up in the subtickets of CASSANDRA-18306.
> 
> I suggest that adding " forRemoval = true" and the later actual removal of 
> the code both require broader consensus.  I'm open to that being on the 
> ticket or needing a thread on the ML.  Small stuff, common sense says on the 
> ticket is enough, but a few folk have already stated that deprecated code 
> that has minimal maintenance overhead should not be removed.
> 

Avoiding pushes to broken branches

[Mick Semb Wever]

I'd like to suggest some improvements for identifying and announcing
broken branches and to avoid pushing commits to broken branches.

For CI we should have two gates.

The first is pre-commit testing, which we have already discussed, e.g.
either ci-cassandra or circleci can be used (and repeated tests are
expected to be run on circleci).

The second gate is whether the branch the commit is being merged to is
in a healthy state.  Our canonical CI system is ci-cassandra, and for
post-commit health it is our only CI.  Last week ci-cassandra was
broken on all branches from 4.0 up.  The cause was two reasons,
neither our fault: debian packaging (CASSANDRA-18910) and xerces2 xml
file processing OOM (cassandra-builds:8d11eea).

Knowing if a branch is broken (before pushing) is just to check
ci-cassandra.apache.org

Folk have suggested this is not enough, and that a message to the dev@
would also help, but part of the problem is that there's no one place
that people check before pushing (there's no requirement on anyone to
be keeping up to date with dev@ at all times).

To summarise, I feel we currently don't have good practices for
 - identifying and announcing a broken CI,
 - knowing who is investigating it,
 - labelling it with the cause,
 - knowing who is working on a fix


The suggested actions I'm proposing for us all to adopt are:

 1. Before committing please check dev@ and ci-cassandra.a.o
 2. If you see ci-cassandra is red on a branch, and no dev@ thread has
been started, please start the dev@ thread and create the ticket,
 3. Put the ticket id into the description of the first red build
("Add description")
 4. By default, hold off on pushing to broken branches.


WRT (2), we should just be able to send the automated build failures
to dev@ instead of builds@, but failed builds are often not sending
such notifications, so this isn't something we can rely on yet.


Reference slack threads:
 - https://the-asf.slack.com/archives/CK23JSY2K/p1696693542832489
 - https://the-asf.slack.com/archives/CK23JSY2K/p1696599019480519
 - https://the-asf.slack.com/archives/CK23JSY2K/p1696351208371029
 - https://the-asf.slack.com/archives/CK23JSY2K/p1695878499669699

Re: Need Confluent "Create" permission for filing a CEP

[Brandon Williams]

I've added you, you should have access now.

Kind Regards,
Brandon

On Tue, Oct 10, 2023 at 1:24 AM Jaydeep Chovatia
 wrote:
>
> Hi,
>
> I want to create a new CEP request but do not see the "Create" page 
> permission on Confluent. Could someone permit me?
> Here is the CEP draft: [DRAFT] CEP - Apache Cassandra Official Repair 
> Solution - Google Docs
>
> My confluent user-id is: chovatia.jayd...@gmail.com
>
> Jaydeep

Re: [DISCUSS] putting versions into Deprecated annotations

[Miklosovic, Stefan]

OK.

Let's go with in-built java.lang.Deprecated annotation. If somebody wants to 
document that in more detail, there are Javadocs as mentioned. Let's just stick 
with the standard stuff.

I will try to implement this for 5.0 (versions since it was deprecated) with my 
take on what should be removed (forRemoval = true) but that should be 
definitely cross-checked on review as Mick mentioned.


From: Mick Semb Wever 
Sent: Monday, October 9, 2023 10:55
To: dev@cassandra.apache.org
Subject: Re: [DISCUSS] putting versions into Deprecated annotations

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.



Tangential question to this is if everything we deprecated is eligible for 
removal? In other words, are there any cases when forRemoval would be false? 
Could you elaborate on that and give such examples or do you all think that 
everything which is deprecated will be eventually removed?


Removal cannot be default.  This came up in the subtickets of CASSANDRA-18306.

I suggest that adding " forRemoval = true" and the later actual removal of the 
code both require broader consensus.  I'm open to that being on the ticket or 
needing a thread on the ML.  Small stuff, common sense says on the ticket is 
enough, but a few folk have already stated that deprecated code that has 
minimal maintenance overhead should not be removed.

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[Claude Warren, Jr via dev]

I have been exploring adding a second Path to the Cassandra File object.
The original path being the path within the standard Cassandra directory
tree and the second being a translated path when there is what was called a
ChannelProxy in place.

A problem arises when the Directories.getLocationForDisk() is called.  It
seems to be looking for locations that start with the data directory
absolute path.   I can change it to make it look for the original path not
the translated path.  But in other cases the translated path is the one
that is needed.

I notice that there is a concept of multiple file locations in the code
base, particularly in the Directories.DataDirectories class where there are
"locationsForNonSystemKeyspaces" and "locationsForSystemKeyspace" in the
constructor, and in the
DatabaseDescriptor.getNonLocalSystemKeyspacesDataFileLocations() method
which returns an array of String and is populated from the cassandra.yaml
file.

The DatabaseDescriptor.getNonLocalSystemKeyspacesDataFileLocations()  only
ever seems to return an array of one item.

Why does DatabaseDescriptor.getNonLocalSystemKeyspacesDataFileLocations()
return an array?

Should the system set the path to the root of the ColumnFamilyStore in the
ColumnFamilyStore directories instance?
Should the Directories.getLocationForDisk() do the proxy to the other file
system?

Where is the proper location to change from the standard internal
representation to the remote location?


On Fri, Sep 29, 2023 at 8:07 AM Claude Warren, Jr 
wrote:

> Sorry I was out sick and did not respond yesterday.
>
> Henrik,  How does your system work?  What is the design strategy?  Also is
> your code available somewhere?
>
> After looking at the code some more I think that the best solution is not
> a FileChannelProxy but to modify the Cassandra File class to get a
> FileSystem object for a Factory to build the Path that is used within that
> object.  I think that this makes if very small change that will pick up
> 90+% of the cases.  We then just need to find the edge cases.
>
>
>
>
>
> On Fri, Sep 29, 2023 at 1:14 AM German Eichberger via dev <
> dev@cassandra.apache.org> wrote:
>
>> Super excited about this as well. Happy to help test with Azure and any
>> other way needed.
>>
>> Thanks,
>> German
>> --
>> *From:* guo Maxwell 
>> *Sent:* Wednesday, September 27, 2023 7:38 PM
>> *To:* dev@cassandra.apache.org 
>> *Subject:* [EXTERNAL] Re: [DISCUSS] CEP-36: A Configurable ChannelProxy
>> to alias external storage locations
>>
>> Thanks , So I think a jira can be created now. And I'd be happy to
>> provide some help with this as well if needed.
>>
>> Henrik Ingo  于2023年9月28日周四 00:21写道：
>>
>> It seems I was volunteered to rebase the Astra implementation of this
>> functionality (FileSystemProvider) onto Cassandra trunk. (And publish it,
>> of course) I'll try to get going today or tomorrow, so that this
>> discussion can then benefit from having that code available for inspection.
>> And potentially using it as a soluttion to this use case.
>>
>> On Tue, Sep 26, 2023 at 8:04 PM Jake Luciani  wrote:
>>
>> We (DataStax) have a FileSystemProvider for Astra we can provide.
>> Works with S3/GCS/Azure.
>>
>> I'll ask someone on our end to make it accessible.
>>
>> This would work by having a bucket prefix per node. But there are lots
>> of details needed to support things like out of bound compaction
>> (mentioned in CEP).
>>
>> Jake
>>
>> On Tue, Sep 26, 2023 at 12:56 PM Benedict  wrote:
>> >
>> > I agree with Ariel, the more suitable insertion point is probably the
>> JDK level FileSystemProvider and FileSystem abstraction.
>> >
>> > It might also be that we can reuse existing work here in some cases?
>> >
>> > On 26 Sep 2023, at 17:49, Ariel Weisberg  wrote:
>> >
>> > 
>> > Hi,
>> >
>> > Support for multiple storage backends including remote storage backends
>> is a pretty high value piece of functionality. I am happy to see there is
>> interest in that.
>> >
>> > I think that `ChannelProxyFactory` as an integration point is going to
>> quickly turn into a dead end as we get into really using multiple storage
>> backends. We need to be able to list files and really the full range of
>> filesystem interactions that Java supports should work with any backend to
>> make development, testing, and using existing code straightforward.
>> >
>> > It's a little more work to get C* to creates paths for alternate
>> backends where appropriate, but that works is probably necessary even with
>> `ChanelProxyFactory` and munging UNIX paths (vs supporting multiple
>> Fileystems). There will probably also be backend specific behaviors that
>> show up above the `ChannelProxy` layer that will depend on the backend.
>> >
>> > Ideally there would be some config to specify several backend
>> filesystems and their individual configuration that can be used, as well as
>> configuration and support for a "backend file router" for file creation
>> (and opening) 

[RESULT][VOTE] Accept java-driver

[Mick Semb Wever]

>
> The vote will be open for 72 hours (or longer). Votes by PMC members are 
> considered binding. A vote passes if there are at least three binding +1s and 
> no -1's.



Vote passes with 30 +1s (ten binding).


This page is updated:
https://incubator.apache.org/ip-clearance/cassandra-java-driver.html


I've created https://issues.apache.org/jira/browse/INFRA-25069

Need Confluent "Create" permission for filing a CEP

[Jaydeep Chovatia]

Hi,

I want to create a new CEP request but do not see the "Create" page
permission on Confluent
.
Could someone permit me?
Here is the CEP draft: [DRAFT] CEP - Apache Cassandra Official Repair
Solution - Google Docs


My confluent user-id is: chovatia.jayd...@gmail.com

Jaydeep