Re: Supporting multiple JDKs

[Sumanth Pasupuleti]

> I also can't remember any reports on regressions in 2.2 bug fix releases
specific to 1.7. So what's the actual problem we want to solve here?

Discussion for 2.2 came up from the ticket CASSANDRA-14563
, which I think came
up from an incident where a JDK1.7 incompatible commit was made into 2.2
branch. But I see the consensus on 2.2 that, there is no ROI on spending
time on 2.2 given that its reaching EOL, which can mean (CASSANDRA-14563
 (in discussion
state) and CASSANDRA-14625
 (in patch available
state) may be resolved as won't fix)

As for 4.0/4.x, I think there are a couple of takeaways from this thread,
and I am happy to pursue them.
1. Make CI jobs use "_build_multi_java" and adding optional workflow to run
tests against JAVA 11 - CASSANDRA-14609

2. Add CI job to validate "ant artifacts" - CASSANDRA-14718

3. Auto-update of documentation from CI builds - CASSANDRA-14719


Thanks,
Sumanth

On Sat, Sep 8, 2018 at 4:59 AM Stefan Podkowinski  wrote:

> I really don't see any benefit at all in having any additional Java 1.7
> specific build and testing changes for the 2.2 branch. The 2.2 version
> is reaching EOL and will only get critical patches until then anyways. I
> also can't remember any reports on regressions in 2.2 bug fix releases
> specific to 1.7. So what's the actual problem we want to solve here?
>
> As for 4.0, we're going to ship multi-release jars, which are targeted
> against Java 8, but also contain Java 11 classes that will only be used
> when executed under Java 11 (also currently just a single class). I can
> see two issues that need our attention with that:
>  * We should make sure to use the "_build_multi_java" target for our CI
> jobs, so we're really testing the same build that we would ship. It's
> probably not going to make a real difference, but who knows..
>  * It would also be nice to have the option to run tests on CI under
> Java 11, although we only provide "experimental" support for that Java
> version. Nice to have at this point, as there will be plenty of bugs in
> 4.0 to fix, until we should spend time looking more closely for more
> subtle Java 11 issues. But if someone wants to contribute any work to
> make this happen, I'd be glad to have that option running tests on Java
> 11, so don't get me wrong.
>
>
> On 07.09.18 04:10, Sumanth Pasupuleti wrote:
> >> And I would suggest to go further and crash the build with JDK1.7 so we
> > can take away the possibility for users to shoot their foot off this way.
> >
> > I like this suggestion. Either we should be on the side of NO support to
> > JDK 1.7, or if we say we support JDK1.7, I believe we should be building
> > against JDK1.7 to make sure we are compliant.
> > I have a quick clarifying question here - I believe origin of
> > CASSANDRA-14563 is from the introduction of an API in 2.2 that is
> > incompatible with 1.7, that has then been manually detected and fixed.
> Are
> > you suggesting, going further, we would not support 1.7?
> >
> >> Currently I'm unclear on how we would make a stable release using only
> > JDK8, maybe their are plans on the table i don't know about?
> >
> > From the current state of build.xml and from the past discussions, I do
> > believe as well, that we need both JDKs to make a 4.0 release using
> > ‘_build_multi_java’. Bonus would be that, the release would also be able
> to
> > run against Java11, but that would be an experimental release.
> >
> >> I'm not familiar with optional jobs or workflows in CircleCi, do you
> have
> > an example of what you mean at hand?
> >
> > By optional, I was referring to having workflow definitions in place, but
> > calls to those workflows commented out. Basically similar to what we have
> > today.
> > workflows:
> > version: 2
> > build_and_run_tests: *default_jobs
> > #build_and_run_tests: *with_dtest_jobs_only
> > #build_and_run_tests_java11: *with_dtest_jobs_java11
> > Jason created CASSANDRA-14609 for this purpose I believe.
> >
> >> Off-topic, but what are your thoughts on this? Can we add `ant
> > artifacts`, and the building of the docs, as a separate jobs into the
> > existing default CircleCI workflow? I think we should also be looking
> into
> > getting https://cassandra.apache.org/doc/latest/ automatically updated
> > after each successful trunk build, and have
> > https://cassandra.apache.org/doc/X.Y versions on the docs in place
> (which
> > are only updated after each patch release).
> >
> > I like all these ideas! I believe we should be able to add a workflow to
> > test out artifact generation. Will create a JIRA for this. Your
> suggestions
> > around auto-update of docs provides a way to keep 

Re: UDF

[Jeremiah D Jordan]

Be careful when pulling in source files from the DataStax Java Driver (or 
anywhere) to make sure and respect its Apache License, Version 2.0 and keep all 
Copyright's etc with said files.

-Jeremiah

> On Sep 11, 2018, at 12:29 PM, Jeff Jirsa  wrote:
> 
> +1 as well.
> 
> On Tue, Sep 11, 2018 at 10:27 AM Aleksey Yeschenko 
> wrote:
> 
>> If this is about inclusion in 4.0, then I support it.
>> 
>> Technically this is *mostly* just a move+donation of some code from
>> java-driver to Cassandra. Given how important this seemingly is to the
>> board and PMC for us to not have the dependency on the driver, the sooner
>> it’s gone, the better.
>> 
>> I’d be +1 for committing to trunk.
>> 
>> —
>> AY
>> 
>> On 11 September 2018 at 14:43:29, Robert Stupp (sn...@snazy.de) wrote:
>> 
>> The patch is technically complete - i.e. it works and does its thing.
>> 
>> It's not strictly a bug fix but targets trunk. That's why I started the
>> discussion.
>> 
>> 
>> On 09/11/2018 02:53 PM, Jason Brown wrote:
>>> Hi Robert,
>>> 
>>> Thanks for taking on this work. Is this message a heads up that a patch
>> is
>>> coming/complete, or to spawn a discussion about including this in 4.0?
>>> 
>>> Thanks,
>>> 
>>> -Jason
>>> 
>>> On Tue, Sep 11, 2018 at 2:32 AM, Robert Stupp  wrote:
>>> 
 In an effort to clean up our hygiene and limit the dependencies used
>> by
 UDFs/UDAs, I think we should refactor the UDF code parts and remove
>> the
 dependency to the Java Driver in that area without breaking existing
 UDFs/UDAs.
 
 A working prototype is in this branch: 
 https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_snazy_=DwIFaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=Gesm79MRSHznQEKqQabvh3Ie1L3xzqlPsfLfEfadHTM=6lUpmmETCKbmt_zcp_DCLIxCGPjVyf7zdX0UjBVOZX4=
 cassandra/tree/feature/remove-udf-driver-dep-trunk <
 https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_snazy_cassandra_tree_feature_remove-2D=DwIFaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g=Gesm79MRSHznQEKqQabvh3Ie1L3xzqlPsfLfEfadHTM=fBx64l59d8Y9Q7m9j0nNH9VvcaHc3QfoCAx4st5UJDM=
 udf-driver-dep-trunk> . The changes are rather trivial and provide
>> 100%
 backwards compatibility for existing UDFs.
 
 The prototype copies the necessary parts from the Java Driver into the
>> C*
 source tree to org.apache.cassandra.cql3.functions.types and adopts
>> its
 usages - i.e. UDF/UDA code plus CQLSSTableWriter +
>> StressCQLSSTableWriter.
 The latter two classes have a reference to UDF’s UDHelper and had to
>> be
 changed as well.
 
 Some functionality, like type parsing & handling, is duplicated in the
 code base with this prototype - once in the “current” source tree and
>> once
 for UDFs. However, unifying the code paths is not trivial, since the
>> UDF
 sandbox prohibits the use of internal classes (direct and likely
>> indirect
 dependencies).
 
 Robert
 
 —
 Robert Stupp
 @snazy
 
 
>> 
>> --
>> Robert Stupp
>> @snazy
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: UDF

[Jeff Jirsa]

+1 as well.

On Tue, Sep 11, 2018 at 10:27 AM Aleksey Yeschenko 
wrote:

> If this is about inclusion in 4.0, then I support it.
>
> Technically this is *mostly* just a move+donation of some code from
> java-driver to Cassandra. Given how important this seemingly is to the
> board and PMC for us to not have the dependency on the driver, the sooner
> it’s gone, the better.
>
> I’d be +1 for committing to trunk.
>
> —
> AY
>
> On 11 September 2018 at 14:43:29, Robert Stupp (sn...@snazy.de) wrote:
>
> The patch is technically complete - i.e. it works and does its thing.
>
> It's not strictly a bug fix but targets trunk. That's why I started the
> discussion.
>
>
> On 09/11/2018 02:53 PM, Jason Brown wrote:
> > Hi Robert,
> >
> > Thanks for taking on this work. Is this message a heads up that a patch
> is
> > coming/complete, or to spawn a discussion about including this in 4.0?
> >
> > Thanks,
> >
> > -Jason
> >
> > On Tue, Sep 11, 2018 at 2:32 AM, Robert Stupp  wrote:
> >
> >> In an effort to clean up our hygiene and limit the dependencies used
> by
> >> UDFs/UDAs, I think we should refactor the UDF code parts and remove
> the
> >> dependency to the Java Driver in that area without breaking existing
> >> UDFs/UDAs.
> >>
> >> A working prototype is in this branch: https://github.com/snazy/
> >> cassandra/tree/feature/remove-udf-driver-dep-trunk <
> >> https://github.com/snazy/cassandra/tree/feature/remove-
> >> udf-driver-dep-trunk> . The changes are rather trivial and provide
> 100%
> >> backwards compatibility for existing UDFs.
> >>
> >> The prototype copies the necessary parts from the Java Driver into the
> C*
> >> source tree to org.apache.cassandra.cql3.functions.types and adopts
> its
> >> usages - i.e. UDF/UDA code plus CQLSSTableWriter +
> StressCQLSSTableWriter.
> >> The latter two classes have a reference to UDF’s UDHelper and had to
> be
> >> changed as well.
> >>
> >> Some functionality, like type parsing & handling, is duplicated in the
> >> code base with this prototype - once in the “current” source tree and
> once
> >> for UDFs. However, unifying the code paths is not trivial, since the
> UDF
> >> sandbox prohibits the use of internal classes (direct and likely
> indirect
> >> dependencies).
> >>
> >> Robert
> >>
> >> —
> >> Robert Stupp
> >> @snazy
> >>
> >>
>
> --
> Robert Stupp
> @snazy
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: UDF

[Aleksey Yeschenko]

If this is about inclusion in 4.0, then I support it.

Technically this is *mostly* just a move+donation of some code from java-driver 
to Cassandra. Given how important this seemingly is to the board and PMC for us 
to not have the dependency on the driver, the sooner it’s gone, the better.

I’d be +1 for committing to trunk.

—
AY

On 11 September 2018 at 14:43:29, Robert Stupp (sn...@snazy.de) wrote:

The patch is technically complete - i.e. it works and does its thing.  

It's not strictly a bug fix but targets trunk. That's why I started the  
discussion.  


On 09/11/2018 02:53 PM, Jason Brown wrote:  
> Hi Robert,  
>  
> Thanks for taking on this work. Is this message a heads up that a patch is  
> coming/complete, or to spawn a discussion about including this in 4.0?  
>  
> Thanks,  
>  
> -Jason  
>  
> On Tue, Sep 11, 2018 at 2:32 AM, Robert Stupp  wrote:  
>  
>> In an effort to clean up our hygiene and limit the dependencies used by  
>> UDFs/UDAs, I think we should refactor the UDF code parts and remove the  
>> dependency to the Java Driver in that area without breaking existing  
>> UDFs/UDAs.  
>>  
>> A working prototype is in this branch: https://github.com/snazy/  
>> cassandra/tree/feature/remove-udf-driver-dep-trunk <  
>> https://github.com/snazy/cassandra/tree/feature/remove-  
>> udf-driver-dep-trunk> . The changes are rather trivial and provide 100%  
>> backwards compatibility for existing UDFs.  
>>  
>> The prototype copies the necessary parts from the Java Driver into the C*  
>> source tree to org.apache.cassandra.cql3.functions.types and adopts its  
>> usages - i.e. UDF/UDA code plus CQLSSTableWriter + StressCQLSSTableWriter.  
>> The latter two classes have a reference to UDF’s UDHelper and had to be  
>> changed as well.  
>>  
>> Some functionality, like type parsing & handling, is duplicated in the  
>> code base with this prototype - once in the “current” source tree and once  
>> for UDFs. However, unifying the code paths is not trivial, since the UDF  
>> sandbox prohibits the use of internal classes (direct and likely indirect  
>> dependencies).  
>>  
>> Robert  
>>  
>> —  
>> Robert Stupp  
>> @snazy  
>>  
>>  

--  
Robert Stupp  
@snazy  


-  
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org  
For additional commands, e-mail: dev-h...@cassandra.apache.org  

Speakers needed for Apache DC Roadshow

[Rich Bowen]

We need your help to make the Apache Washington DC Roadshow on Dec 4th a 
success.


What do we need most? Speakers!

We're bringing a unique DC flavor to this event by mixing Open Source 
Software with talks about Apache projects as well as OSS CyberSecurity, 
OSS in Government and and OSS Career advice.


Please take a look at: http://www.apachecon.com/usroadshow18/

(Note: You are receiving this message because you are subscribed to one 
or more mailing lists at The Apache Software Foundation.)


Rich, for the ApacheCon Planners

--
rbo...@apache.org
http://apachecon.com
@ApacheCon

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: UDF

[Robert Stupp]

The patch is technically complete - i.e. it works and does its thing.

It's not strictly a bug fix but targets trunk. That's why I started the 
discussion.



On 09/11/2018 02:53 PM, Jason Brown wrote:

Hi Robert,

Thanks for taking on this work. Is this message a heads up that a patch is
coming/complete, or to spawn a discussion about including this in 4.0?

Thanks,

-Jason

On Tue, Sep 11, 2018 at 2:32 AM, Robert Stupp  wrote:


In an effort to clean up our hygiene and limit the dependencies used by
UDFs/UDAs, I think we should refactor the UDF code parts and remove the
dependency to the Java Driver in that area without breaking existing
UDFs/UDAs.

A working prototype is in this branch: https://github.com/snazy/
cassandra/tree/feature/remove-udf-driver-dep-trunk <
https://github.com/snazy/cassandra/tree/feature/remove-
udf-driver-dep-trunk> . The changes are rather trivial and provide 100%
backwards compatibility for existing UDFs.

The prototype copies the necessary parts from the Java Driver into the C*
source tree to org.apache.cassandra.cql3.functions.types and adopts its
usages - i.e. UDF/UDA code plus CQLSSTableWriter + StressCQLSSTableWriter.
The latter two classes have a reference to UDF’s UDHelper and had to be
changed as well.

Some functionality, like type parsing & handling, is duplicated in the
code base with this prototype - once in the “current” source tree and once
for UDFs. However, unifying the code paths is not trivial, since the UDF
sandbox prohibits the use of internal classes (direct and likely indirect
dependencies).

Robert

—
Robert Stupp
@snazy




--
Robert Stupp
@snazy


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Yet another repair solution

[Marcus Olsson]

Sure thing!

Up until now it has been running in an OSGi environment, so among other
things I'm working towards both OSGi and a standalone application.

It's designed to be tightly coupled with a single instance, where it
keeps track of the repair state and performs repair of tables for that
node only.
The current features include alarms, "pausing repairs", metrics,
dynamic scheduling and "pluggability" for each of them (as well as some
other components like connection management, lease management, etc).

The design is based on CASSANDRA-10070 with Cassandra (and LWT) as a
default backend for the lease management. It utilizes the repair
history from Cassandra to determine repair state of tables in order to
prioritize and schedule them. This also means that a manual "nodetool
repair" would be counted towards the repair state of the tables.

Best Regards
Marcus Olsson

On tor, 2018-08-30 at 07:55 -0700, Dinesh Joshi wrote:
> In the meanwhile, do you think you could highlight the features of
> your repair solution / sidecar?
> 
> Dinesh
> 
> > 
> > On Aug 30, 2018, at 4:57 AM, Marcus Olsson  > com> wrote:
> > 
> > Great to see that there is an interest! As there currently are some
> > internal dependencies etc. in place there is still some work to be
> > done before we can publish it. I would expect this to take at least
> > a few weeks, to try set the correct expectations.
> > 
> > Best Regards
> > Marcus Olsson
> > 
> > On tis, 2018-08-28 at 23:18 -0700, Vinay Chella wrote:
> > I am excited to see that the community is working on solving the
> > critical
> > problems in C* operations (e.g., repair, backups etc.,) with
> > different
> > solutions. Of course, learnings from these systems are key to
> > designing the
> > robust solution which works for everyone.
> > 
> > 
> > Thanks,
> > Vinay Chella
> > 
> > 
> > On Tue, Aug 28, 2018 at 1:23 PM Roopa  > id>
> > wrote:
> > 
> > 
> > +1 interested in seeing and understanding another repair solution.
> > 
> > 
> > On Aug 28, 2018, at 1:03 PM, Joseph Lynch  > ilto:joe.e.ly...@gmail.com>> wrote:
> > 
> > I'm pretty interested in seeing and understanding your solution!
> > When we
> > started on CASSANDRA-14346 reading your design documents and plan
> > you
> > sketched out in CASSANDRA-10070 were really helpful in improving
> > our
> > design. I'm particularly interested in how the Scheduler/Job/Task
> > APIs
> > turned out (we're working on something similar internally and would
> > love
> > to
> > 
> > compare notes and figure out the best way to implement that kind of
> > abstraction)?
> > 
> > -Joey
> > 
> > 
> > On Tue, Aug 28, 2018 at 6:34 AM Marcus Olsson <
> > marcus.ols...@ericsson.com>
> > 
> > wrote:
> > 
> > 
> > Hi,
> > 
> > With the risk of stirring the repair/side-car topic  even further
> > I'd
> > just
> > 
> > 
> > like to mention that we have recently gotten approval to contribute
> > our
> > repair management side-car solution.
> > It's based on the proposal in
> > https://issues.apache.org/jira/browse/CASSANDRA-10070 as a
> > standalone
> > application sitting next to each instance.
> > With the recent discussions in mind I'd just like to hear the
> > thoughts
> > from the community on this before we put in the effort of bringing
> > our
> > solution into open source.
> > 
> > Would there be an interest of having yet another repair solution in
> > the
> > discussion?
> > 
> > Best Regards
> > Marcus Olsson
> > 
> > -
> > 
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > :dev-unsubscr...@cassandra.apache.org>
> > For additional commands, e-mail: dev-h...@cassandra.apache.org > to:dev-h...@cassandra.apache.org>
> > 
> > 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 

Re: UDF

[Jason Brown]

Hi Robert,

Thanks for taking on this work. Is this message a heads up that a patch is
coming/complete, or to spawn a discussion about including this in 4.0?

Thanks,

-Jason

On Tue, Sep 11, 2018 at 2:32 AM, Robert Stupp  wrote:

> In an effort to clean up our hygiene and limit the dependencies used by
> UDFs/UDAs, I think we should refactor the UDF code parts and remove the
> dependency to the Java Driver in that area without breaking existing
> UDFs/UDAs.
>
> A working prototype is in this branch: https://github.com/snazy/
> cassandra/tree/feature/remove-udf-driver-dep-trunk <
> https://github.com/snazy/cassandra/tree/feature/remove-
> udf-driver-dep-trunk> . The changes are rather trivial and provide 100%
> backwards compatibility for existing UDFs.
>
> The prototype copies the necessary parts from the Java Driver into the C*
> source tree to org.apache.cassandra.cql3.functions.types and adopts its
> usages - i.e. UDF/UDA code plus CQLSSTableWriter + StressCQLSSTableWriter.
> The latter two classes have a reference to UDF’s UDHelper and had to be
> changed as well.
>
> Some functionality, like type parsing & handling, is duplicated in the
> code base with this prototype - once in the “current” source tree and once
> for UDFs. However, unifying the code paths is not trivial, since the UDF
> sandbox prohibits the use of internal classes (direct and likely indirect
> dependencies).
>
> Robert
>
> —
> Robert Stupp
> @snazy
>
>

UDF

[Robert Stupp]

In an effort to clean up our hygiene and limit the dependencies used by 
UDFs/UDAs, I think we should refactor the UDF code parts and remove the 
dependency to the Java Driver in that area without breaking existing UDFs/UDAs.

A working prototype is in this branch: 
https://github.com/snazy/cassandra/tree/feature/remove-udf-driver-dep-trunk 
 . 
The changes are rather trivial and provide 100% backwards compatibility for 
existing UDFs.

The prototype copies the necessary parts from the Java Driver into the C* 
source tree to org.apache.cassandra.cql3.functions.types and adopts its usages 
- i.e. UDF/UDA code plus CQLSSTableWriter + StressCQLSSTableWriter. The latter 
two classes have a reference to UDF’s UDHelper and had to be changed as well.

Some functionality, like type parsing & handling, is duplicated in the code 
base with this prototype - once in the “current” source tree and once for UDFs. 
However, unifying the code paths is not trivial, since the UDF sandbox 
prohibits the use of internal classes (direct and likely indirect dependencies).

Robert

—
Robert Stupp
@snazy