Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

2023-09-27 Thread Henrik Ingo
> still to be solved and much logging to be reduced. > > > > I look forward to productive discussions, > > Claude > > > > [1] > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations > > [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory > > > > > > > > > -- > http://twitter.com/tjake > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [VOTE] Release Apache Cassandra 5.0-alpha1

2023-09-05 Thread Henrik Ingo
tools/fqltool/src/org/apache/cassandra/fqltool/commands/Dump.java > 10. ./src/java/org/apache/cassandra/net/Crc.java > 11. https://www.apache.org/legal/resolved.html#cc-by > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [DISCUSS] The future of CREATE INDEX

2023-05-17 Thread Henrik Ingo
h to force the user to make a decision (and wouldn't > change the legacy behavior of the existing CREATE INDEX). In this world, > creating a legacy 2i might look something like CREATE INDEX...USING > `legacy`. > 3.) Eventually deprecate CREATE CUSTOM INDEX...USING. > > Eventual

Re: [DISCUSS] New data type for vector search

2023-04-28 Thread Henrik Ingo
gt;> On Apr 27, 2023, at 9:00 AM, Benedict wrote: >>>> >>>> >>>> That’s a bounded ring buffer, not a fixed length array. >>>> >>>> This definitely isn’t a tuple because the types are all the same, which >>>> is pret

Re: [DISCUSS] New data type for vector search

2023-04-28 Thread Henrik Ingo
gt; we do here, it is fundamentally a frozen list with a restriction on its >>> size. What we’re defining here are “statically” sized arrays, whereas a >>> frozen list is essentially a dynamically sized array. >>> >>> I do not think vector is a good name because vector is us

Re: Adding vector search to SAI with heirarchical navigable small world graph index

2023-04-25 Thread Henrik Ingo
a different node than the >> first. Elastic requires specifying it up front like I propose here.2. >> Driver support3. Support for updating the vector in a row that hasn’t been >> flushed yet, either by updating HNSW to support deletes, or by adding some >> kind of invalidation

Re: [DISCUSS] Next release date

2023-04-19 Thread Henrik Ingo
sks our desired GA date. Also, again, I don't understand how cutting a > 5.0 branch makes anything substantially easier to start testing. Perhaps > I'm the only one who thinks this. If so, I'm not going to make further > noise about it. > > On Tue, Apr 18, 2023 at 7:26 AM Henrik

Re: [DISCUSS] Next release date

2023-04-18 Thread Henrik Ingo
n we are talking about merges where a single merge / rebase takes more than one day. We will have to stop merging smaller work to trunk anyway, when CEP-21 is being merged. No? henrik On Tue, Apr 18, 2023 at 3:24 AM Henrik Ingo wrote: > Trying to collect a few loose ends from across thi

Re: [DISCUSS] Next release date

2023-04-17 Thread Henrik Ingo
freeze. This last question is basically a form of saying I hope we aren't discussing a problem that doesn't even exist? henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [EXTERNAL] Re: (CVE only) support for 3,11 beyond published EOL

2023-04-17 Thread Henrik Ingo
out this? > > > I am also asking specifically for 3.11 since this release has been around > so long that it might warrant longer support than what we would offer for > 4.0. > > > > This logic can also be the other way around :-) > > We should be sending a clear signal

Re: [DISCUSS] CEP-29 CQL NOT Operator

2023-04-13 Thread Henrik Ingo
I get what you all mean: No, I actually think both are unnecessary. But yeah, certainly this latter case is a bug? henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [DISCUSS] CEP-29 CQL NOT Operator

2023-04-13 Thread Henrik Ingo
ion keys). >> >> > On Apr 4, 2023, at 2:28 AM, Piotr Kołaczkowski >> wrote: >> > >> > Hi everyone! >> > >> > I created a new CEP for adding NOT support to the query language and >> > want to start discussion around it: >>

Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE

2023-04-06 Thread Henrik Ingo
> threads being collaborative in an open-mode. Sometimes the best idea is on > the tail end of a sequence of bad and/or unpopular ideas. > > > > > > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE

2023-04-04 Thread Henrik Ingo
t;> for a group of tables. Nearly all traditional SQL databases use DATABASE as >>>> the container name for a group of tables so it would make sense for >>>> Cassandra to adopt this naming as well. >>>> >>>> KEYSPACE would be kept in the grammar but we wo

Re: [DISCUSS] cep-15-accord, cep-21-tcm, and trunk

2023-03-27 Thread Henrik Ingo
te: > > > Accord doesn’t have a hard dependency on CEP-21 fwiw, it just needs > linearizable epochs. This could be achieved with a much more modest patch, > essentially avoiding almost all of the insertion points of cep-21, just > making sure that joining and leaving nodes update some

Re: [EXTERNAL] [DISCUSS] Next release date

2023-03-27 Thread Henrik Ingo
letting the branch/release date slip too much into the > unknown, squeezing GA QA efforts, while putting in place exceptional > waivers for CEP-21 and CEP-15. > >>> > >>> I hope that in the future we will be more willing to commit to the > release train model: less

Re: [DISCUSS] cep-15-accord, cep-21-tcm, and trunk

2023-03-22 Thread Henrik Ingo
ebase it on cep-21-tcm until such time as the latter merges to trunk, at > which point cep-15-accord can be rebased to trunk again and then merged > when ready, nominally taking up the work of CASSANDRA-18196 > <https://issues.apache.org/jira/browse/CASSANDRA-18196> again. > > Any objec

Re: [DISCUSS] Next release date

2023-03-01 Thread Henrik Ingo
e? > 2b. If the final components to specified CEPs are not > approved/appropriate to land during alpha, would it be better if the > project commits to a one-off half-year release later in the year? > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.co

Re: Downgradability

2023-02-23 Thread Henrik Ingo
sure > we haven’t talked about switching the default format? > > On 23 Feb 2023, at 12:12, Henrik Ingo wrote: > >  > On Thu, Feb 23, 2023 at 11:57 AM Benedict wrote: > >> Can somebody explain to me why this is being fought tooth and nail, when >> the work invol

Re: Downgradability

2023-02-23 Thread Henrik Ingo
ompatibility solution first. And possibly better testing, etc. Letting them merge would be justified by the desire to have more frequent and smaller increments of work merged into trunk... well, I'm not going to repeat everything from that discussion but the same pro's and con's apply. henrik -- He

Re: Downgradability

2023-02-22 Thread Henrik Ingo
to be downgraded later. henrik On Thu, Feb 23, 2023 at 1:14 AM Henrik Ingo wrote: > Just this once I'm going to be really brief :-) > > Just wanted to share for reference how Mongodb implemented > downgradeability around their 4.4 version: > https://www.mongodb.com/docs/manual/re

Re: Downgradability

2023-02-22 Thread Henrik Ingo
d needing >>>> a bump for a 4.1 bugfix might require porting over support for new >>>> features); >>>> - it requires separate and uncoordinated solutions to the problem and >>>> switching mechanisms for each individual change. >>>> >>>> An alternative solution is to implement/complete CASSANDRA-8110 >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8110>, which provides >>>> a method of writing sstables for a target version. During upgrades, a node >>>> could be set to produce sstables corresponding to the older version, and >>>> there is a very straightforward way to implement modifications to formats >>>> like the tickets above to conform to its requirements. >>>> >>>> What do people think should be the way forward? >>>> >>>> Regards, >>>> Branimir >>>> >>>> >>>> -- >>> you are the apple of my eye ! >>> >> -- >> http://twitter.com/tjake >> >> -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: Announcement: Performance testing for Cassandra

2023-02-06 Thread Henrik Ingo
> is the link to the fallout-tests repository > <https://github.com/datastax/fallout-tests>. > > One thing to note is that, at the moment, it is not possible to run > performance tests on trunk but rather on specific versions of Cassandra. > However, we are currently wor

Re: CASSANDRA-14227 removing the 2038 limit

2023-02-03 Thread Henrik Ingo
the current 2038 limit since the 1970 Unix epoch. I > wouldn't make it a configurable value, off the top of my head it would make > for some interesting bugs and debugging sessions when nodes had different > values. Food for another ticket in any case imo. > > Regards > On 3

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Henrik Ingo
if this is more > controversial, so I won't burden this email with enumerating pros and cons > of the two approaches until I get a gauge of the community's temperature. > > So - what do we think? > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.c

Re: CASSANDRA-14227 removing the 2038 limit

2023-02-03 Thread Henrik Ingo
flow policy is the default but everything is backwards > compatible by keeping the previous ones in place. Think upgrade scenarios > or apps relying on the previous behavior. > > - The new limit is around year 292,471,208,677 which sounds ok given the > Sun will start collapsing in 3 to 5 billion years :-) > > - Please feel free to drop by the ticket and take a look at the PR even if > it's cursory > > Thx in advance. > > > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: Welcome Patrick McFadin as Cassandra Committer

2023-02-02 Thread Henrik Ingo
din has accepted > the invitation to become committer today. > > Thanks a lot, Patrick, for everything you have done for this project and > its community through the years. > > Congratulations and welcome! > > The Apache Cassandra PMC members > -- Henrik Ingo c. +358 40

Re: Merging CEP-15 to trunk

2023-01-30 Thread Henrik Ingo
Circle OR Jenkins. > > Sure. And thanks. But during development, did you ever run nightly tests / all tests? I wouldn't want the night after merging to trunk to be the first time those are run. henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/data

Re: Merging CEP-15 to trunk

2023-01-30 Thread Henrik Ingo
ate that the implementation follows the CEP. But I wouldn't expect a debate on competing approaches at this stage. henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: Merging CEP-15 to trunk

2023-01-30 Thread Henrik Ingo
that presumably should be uncommented (now or later)... But the discussion on remaining items should have been delegated to the more relevant contributors than myself. If you don't see further communications from me, it just means I'm satisfied for my part and expect others to speak up (or not)

Re: Merging CEP-15 to trunk

2023-01-27 Thread Henrik Ingo
at 5:06 PM Henrik Ingo wrote: > Thanks Benedict > > For brevity I'll respond to your email, although indirectly this is also a > continuation of my debate with Josh: > > At least on my scorecard, one issue was raised regarding the actual code: > CASSANDRA-18193 Provide desi

Re: Merging CEP-15 to trunk

2023-01-25 Thread Henrik Ingo
be > individually involved in specific code level changes. If folks feel like > our current processes, CI infrastructure, and committer pool risks > maintaining our bar of quality that's definitely something we should talk > about in depth, as in my mind that's the backbone of us s

Re: Merging CEP-15 to trunk

2023-01-25 Thread Henrik Ingo
1. > > One thing that I wanted to ask for is when you push to CI, you or whoever > does it, to approve all jobs. > > > Thanks Ekaterina, we will be sure to fully qualify the CI result, and I > will make sure we also run your flaky test runner on the newly introduced > tests.

Re: Merging CEP-15 to trunk

2023-01-24 Thread Henrik Ingo
gt; are dismissed or people not trusted or anything like that. > But considering the size and the importance of this work, I can really see > only benefit of a final cross-check. > As Henrik mentioned me, I am not sure I will have the chance to review > this work any time soon (just setting the

Re: Merging CEP-15 to trunk

2023-01-24 Thread Henrik Ingo
deserves a quick check by > another. Ideally this would be one of the existing reviewers (but like any > other review step, no matter how short and trivial it is, that's still an > open process). I see others already doing this when rebasing larger patches > before the final merge. > &

Re: Merging CEP-15 to trunk

2023-01-24 Thread Henrik Ingo
ewer is a committer or a PMC member? henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: Merging CEP-15 to trunk

2023-01-20 Thread Henrik Ingo
merge into trunk deserves extra eyeballs than a merge into a feature > branch. > > We can refer to this as a "higher pre-commit gateway" or a "second pass". > Either way I believe it is a good thing. > > > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com

Re: Intra-project dependencies

2023-01-20 Thread Henrik Ingo
Thanks Mick and David. I've been following this silently for a few days because we already exhausted my knowledge on the topic. But it seems your collective knowledge is uncovering a nice solution. If I summarize, I like all of this: - link to SHA, not library version - use git submodules

Re: Intra-project dependencies

2023-01-17 Thread Henrik Ingo
ild from cassandra commit 123abc, should checkout/download/use the correct versions of other modules at the time 123abc was committed. henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://

Re: Intra-project dependencies

2023-01-17 Thread Henrik Ingo
H5S7Ebw!elHEL4NgggbLsBElKkANZ1KhFuuOpnWrUZe8RESgdwNEkPdmWuw_JQEVFlxZbPTkq7FsCt5BAn_x5pBGaqIJig$ > and | > > | > https://urldefense.com/v3/__https://pgp.mit.edu/pks/lookup?search=derek*40chen-becker.org__;JQ!!PbtH5S7Ebw!elHEL4NgggbLsBElKkANZ1KhFuuOpnWrUZe8RESgdwNEkPdmWuw_JQEVFlxZbP

Re: Intra-project dependencies

2023-01-16 Thread Henrik Ingo
e complicated (our source tarballs are the asf >> releases) >> >> How? > > - handling patches cover submodules >> >> How is this different to patches affecting multiple versions in C*? > > - switching branches, and using git worktrees, during dv >> >

Re: Intra-project dependencies

2023-01-16 Thread Henrik Ingo
upgrade tests? As the library > itself no longer has an explicit version, what I presume you meant by > logical version. > > To begin with, I'm leaning towards (2) because it is a cognitive re-use of > our release branches, and the problems aro

Re: [DISCUSS] Taking another(other(other)) stab at performance testing

2023-01-10 Thread Henrik Ingo
of 4.0. https://arxiv.org/abs/2301.03034 henrik On Sun, Jan 8, 2023 at 5:12 AM Henrik Ingo wrote: > Hi Josh, all > > I'm sitting at an airport, so rather than participating in the comment > threads in the doc, I will just post some high level principles I've > derived during my

Re: [DISCUSS] Taking another(other(other)) stab at performance testing

2023-01-07 Thread Henrik Ingo
s month (1st page, > estimated reading time 5 minutes): > https://docs.google.com/document/d/1X5C0dQdl6-oGRr9mXVPwAJTPjkS8lyt2Iz3hWTI4yIk/edit# > > Curious to hear what other perspectives there are out there on the topic. > > Early Happy New Years everyone! > > ~Josh > > > --

Re: [DISCUSS] CEP-26: Unified Compaction Strategy

2022-12-20 Thread Henrik Ingo
I noticed the CEP doesn't link to this, so it should be worth mentioning that the UCS documentation is available here: https://github.com/datastax/cassandra/blob/ds-trunk/doc/unified_compaction.md Both of the above seem to do a poor job referencing the literature we've been inspired by. I will

Re: [DISCUSS] CEP-21: Transactional Cluster Metadata

2022-09-05 Thread Henrik Ingo
d version isn't that big of a deal? Also, it should make sense to minimize the dependencies from the previous major version (without CEP-21) to the new major version (with CEP-21). If a bug is found, it's much easier to fix code in the new major version than the old and supposedly stable one. henrik

Re: [DISCUSS] CEP-20: Dynamic Data Masking

2022-08-24 Thread Henrik Ingo
d it be for SUPERUSERs to *not* automatically get the >>>> UNMASK permission? >>>> >>>> I'll also echo the concerns around masking primary key components. >>>> It's highly likely that certain personal data properties would be used as a >>>> partitio

Re: [DISCUSS] CEP-20: Dynamic Data Masking

2022-08-23 Thread Henrik Ingo
olumn and role. However, I'm not sure we > need that type of granularity instead of the simplicity of attaching the > masking to the column declaration. wdyt? > > > My gut feeling likewise is that this adds complexity but little value. > >> -- Henrik Ingo +358 40 569 7354 &l

Re: [DISCUSS] CEP-20: Dynamic Data Masking

2022-08-22 Thread Henrik Ingo
> by using UDFs as masking functions. > > Thanks, > -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.com/DataStaxEng> [image: Visit us on YouTube.] <https://urldefense.

Re: [DISCUSS] Cassandra ecosystem site

2022-08-07 Thread Henrik Ingo
ink > <https://urldefense.com/v3/__http://cassandra.link/__;!!PbtH5S7Ebw!e2PlRJmfEuZkacYBgAqGI0rJLJeePGh6jCyohtgYzSrBH-TIOesfuqmPrfneRqQ-dJiC5v14Sj-oXRnMlcMBN00cLVV0aQ$>* > : > The best resources for Apache Cassandra > > -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit u

Re: Cassandra project status update 2022-07-18

2022-07-19 Thread Henrik Ingo
On Mon, Jul 18, 2022 at 11:42 PM Josh McKenzie wrote: > [CI Trends] > https://butler.cassandra.apache.org/#/ > > The last three weeks show us this delightful trend: > > 3.0: 10 -> 10 > 3.11: 23 -> 15 > 4.0: 2 -> 1 > 4.1: 8 -> 10 -> 5 > trunk: 20 -> 5 > > That's 36 total failures across all our

Re: [Marketing] For Review: Pluggable Memtable Implementations blog

2022-07-14 Thread Henrik Ingo
X1EhqWihEb35Sevv-JDE/edit > -- > > Chris Thornett > -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.com/DataStaxEng> [image: Visit us on YouT

Re: Thanks to Nate for his service as PMC Chair

2022-07-14 Thread Henrik Ingo
of Apache Cassandra PMC members can be found on: > https://cassandra.apache.org/_/community.html > > Kind regards, > > Paulo > -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.c

[DISCUSS] Cassandra ecosystem site

2022-07-04 Thread Henrik Ingo
do want to start using this, maybe an immediate need would be to setup a backup of the MySQL database that hosts the contents of the site. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twit

Re: CEP-15 multi key transaction syntax

2022-06-06 Thread Henrik Ingo
; it’s fine to require a client upgrade to get multiple result sets. > > Possibly. I just wanted to share an idea for consideration. IMO the temp table idea might not be too hard to implement*, but sure the syntax does feel a bit bolted on. *) I'm maybe the wrong person to judge that, of course :-)

Re: CEP-15 multi key transaction syntax

2022-06-06 Thread Henrik Ingo
, and a result set for each named select (but not > named update). We could also support an optional RETURN keyword, which > would allow the user to only return specific named selects (ie: RETURN > select1, select2). > > Let me know what you think! > > Blake >

Pluggability improvements in 4.1

2022-04-26 Thread Henrik Ingo
ership is not merged.) CEP-16 <https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-16%3A+Auth+Plugin+Support+for+CQLSH> While client side, worth mentioning: Pluggable auth for CQLSH If there are more that I don't know about, please reply and add to the list. henrik -- Henrik Ingo +35

Re: [DISCUSS] CEP-7 Storage Attached Index

2022-02-14 Thread Henrik Ingo
On Fri, Feb 11, 2022 at 8:47 PM Caleb Rackliffe wrote: > Just finished reading the latest version of the CEP. Here are my thoughts: > > - We've already talked about OR queries, so I won't rehash that, but > tokenization support seems like it might be another one of those places > where we can

Re: [GSOC] Call for Mentors

2022-02-11 Thread Henrik Ingo
red to complete the task? > - What warm-up tasks can the candidate do to ramp up for the project? > > The mentor will work with potential participants to refine the high level > description into smaller subtasks at a later stage (during candidate > application period). > > Cheers,

Re: [DISCUSS] CEP-7 Storage Attached Index

2022-02-07 Thread Henrik Ingo
Thanks Benjamin for reviewing and raising this. While I don't speak for the CEP authors, just some thoughts from me: On Mon, Feb 7, 2022 at 11:18 AM Benjamin Lerer wrote: > I would like to raise 2 points regarding the current CEP proposal: > > 1. There are mention of some target versions and

Re: [GSOC] Call for Mentors

2022-02-02 Thread Henrik Ingo
;> that so prospective participants can start engaging with the project and >> working with mentors to refine the project before submitting an application. >> >> This year I will not be able to participate as a primary mentor but I >> would be happy to co-mentor other

Re: [DISCUSS] CEP-7 Storage Attached Index

2022-02-02 Thread Henrik Ingo
onflation >> >> on >> >> this one, but maybe it's the right time to raise >> >> it >> >> :shrug:) >> >> Agreed that modularization is the way to go and will >> >> speed up >> >>

Re: Google Summer of Code 2022 - Kick-off

2022-01-21 Thread Henrik Ingo
as submit project ideas. > > Em dom., 16 de jan. de 2022 às 19:13, Henrik Ingo < > henrik.i...@datastax.com> escreveu: > >> Hi Paulo >> >> In my experience, it works best when the mentoring organization is clear >> about who is the mentor, and

Re: Google Summer of Code 2022 - Kick-off

2022-01-16 Thread Henrik Ingo
tasks with the "gsoc" tag. >>> >> >>> >> Please note that mentoring a GSoC project is not only a good way to >>> attract >>> >> new members to the community but also a great way of recruiting new >>> members >>> >> to your p

Re: [DISCUSS] Releasable trunk and quality

2022-01-06 Thread Henrik Ingo
nk, then in descending order to each stable branch, you can just check the oldest branch to verify the chain. Not everyone followed this style, but I would always append the cherry pick message, so that the last commit (to the oldest stable branch) would contain the chain of githashes all the way t

Re: [DISCUSS] Periodic snapshot publishing with minor version bumps

2021-12-21 Thread Henrik Ingo
gt; > ----- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] &l

Re: [DISCUSS] Releasable trunk and quality

2021-12-21 Thread Henrik Ingo
nt > > > branches that are developed separately (no merge tracking) is more > > > complicated. So yeah, I see value in those merge commits. I'm not > against > > > trying something new, just would appreciate a bit more exposure to it > > > before making a project wide

Re: Paxos repairs in CEP-14

2021-12-05 Thread Henrik Ingo
plained, did I understand correctly that (focusing on a single partition and only LWT writes...) I can in any event stream commit logs from a majority of replicas, merge them, and such a merged log must contain all committed transactions to that partition. (And this should have nothing to do with the

Re: Paxos repairs in CEP-14

2021-12-05 Thread Henrik Ingo
an have other uses. Even with all of the above, I'm still left wondering: does the repair process (with the above modification, say) result in a node having all writes that happened before t, or is it only guaranteed to have the most recent value for each primary key? Henrik > > From: Henrik I

Paxos repairs in CEP-14

2021-12-04 Thread Henrik Ingo
contain a complete log? In particular, I'm trying to parse the significance of "or witnessing something newer"? (Use case for this last question could be point in time restore, aka continuous backup, or also streaming writes to a downstream system.) henrik -- Henrik Ingo +358 4

Re: [DISCUSS] Nested YAML configs for new features

2021-11-24 Thread Henrik Ingo
on > > user preference (example: > > track_warnings.row_index_size.abort_threshold, using the '.' notation > > to represent nested structures) > > > > Thoughts? > > > > ----- > > To

Re: [DISCUSS] Releasable trunk and quality

2021-11-17 Thread Henrik Ingo
ure: running face-first into 60+ failing tests on trunk when > going through the commit process for denylisting this week brought this > topic back up for me (reminds me of when I went to merge CDC back in 3.6 > and those test failures riled me up... I sense a pattern ;)) > > Looking forwar

Re: [DISCUSS] Creating a new slack channel for newcomers

2021-11-12 Thread Henrik Ingo
bably have a specific channel for > > > > > newcomers. > > > > > This proposal makes total sense to me. > > > > > > > > > > What is your opinion on this? Do you have any concerns about it? > > >

Re: [DISCUSS] CEP-18: Improving Modularity

2021-10-25 Thread Henrik Ingo
stems. > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-18%3A+Improving+Modularity > > CASSANDRA-17044 has already been created for Schema Storage changes related > to this work and more JIRAs and PRs are to follow for the other subsystems > proposed in the CEP. > >

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Henrik Ingo
server side, and it's not possible to have any additional roundtrips to the client. So the use of "interactive transactions" is supposed to distinguish from those. But you're right I may have invented the word. Historically such transactions were the norm so no additional qualifier has been needed.

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Henrik Ingo
ction. And this is traditionally the common case, even if recent NewSQL and NoSQL databases have introduced some intriguing outside of the box thinking in this area. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: V

Re: [IDEA] Read committed transaction with Accord

2021-10-14 Thread Henrik Ingo
ialize access" error. > > > > It looks to me like the proposed design here would (1) not allow NOC to > > make progress at READ COMMITTED, but also (2) does not provide the tools > to > > achieve progress with SERIALIZABLE either since locking outside of the > >

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Henrik Ingo
, the server cannot retry the transaction when concurrent > changes are found to have been applied after the reconnaissance reads (what > you call the conversational phase). > > On Tue, Oct 12, 2021 at 3:55 PM Henrik Ingo > wrote: > > > Hi all > > > > I was e

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-13 Thread Henrik Ingo
esult in "finer grained" dependency checking and therefore cause less aborts due to occ. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.com/DataStaxEn

Re: [IDEA] Read committed transaction with Accord

2021-10-13 Thread Henrik Ingo
omically as part of the Accord operation, and then the transaction > concurrency control arbitration mechanism kicks in. > And now I teased you to outline a different read committed transaction approach :-) henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [imag

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Henrik Ingo
sandra is coming from, I'm not particularly alarmed by this limitation as I would expect operations on a Cassandra database to be fast and small, but it's an important limitation to call out for sure. Indeed, those who have been worried Accord will not be able to serve well all possible future

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-13 Thread Henrik Ingo
execute itself, and nothing more. The above is saying that for a given snapshot/timestamp, the result of a statement is equally well defined by the secondary index keys used as it is by the primary keys returned from those secondary index keys. henrik -- Henrik Ingo +358 40 569 7354 <358405

[IDEA] Read committed transaction with Accord

2021-10-13 Thread Henrik Ingo
just note that needing to re-execute all reads during the Accord phase (commit phase) would make the design more expensive, since the transaction is now executed twice. The goal of a simplistic light weight semantic is achieved by not doing so and claiming the weaker interpretation of read committe

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Henrik Ingo
On Tue, Oct 12, 2021 at 11:54 PM Henrik Ingo wrote: > Secondary indexes are supported without any additional work needed. > > Correction: The "transaction reads its own writes" feature would require to also store secondary index keys in the transaction state. These of cou

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Henrik Ingo
READ COMMITTED isolation level and to get the current isolation level. Future work: A motivation for the above proposal is that the same scheme could be extended to support SNAPSHOT ISOLATION transactions. This would require MVCC support from the storage engine. --- It would be interesting to hear

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
e in RDBMS land now, and the above is just a replication solution, there is no sharding anywhere. For the Serializeable paper, I struggle to even imagine how it could scale to multiple shards. For SI it's kind of easier as only write conflicts need to be checked. henrik -- Henrik Ing

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
ated state is > being updated from the same region, it might simply not be needed – at > least any time soon. > > For sure. I think we're all just trying to understand the landscape what we are talking about here, not trying to say everything should be implemented in v1. henrik --

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
On Fri, Oct 1, 2021 at 4:37 PM Henrik Ingo wrote: > A known optimization for the hot rows problem is to "hint" or manually > force clients to direct all updates to the hot row to the same node, > essentially making the system leader based. This allows the database to >

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
iver strict serializable interactive > transactions > > using Accord. These would simply corroborate that the participating keys > > had not modified their write timestamps during the final transaction. > These > > could even be undertaken with still only a single wide area rou

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-22 Thread Henrik Ingo
is still the optimal building block performance wise to do so, or whether users would then have lower consistency level but still pay the performance cost of a stricter consistency level. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.co

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-22 Thread Henrik Ingo
Clock I guess?) scheme like in Accord is more familiar to current Cassandra semantics. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.com/DataStaxEng> [image: Visit us on

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-21 Thread Henrik Ingo
somehow use the (timestamp, id) to query a node to verify whether such a transaction was recently committed. I'm unsure whether that's convenient for a user though. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image:

Re: [DISCUSS] CEP-7 Storage Attached Index

2021-09-16 Thread Henrik Ingo
> > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > only > > > > > > &

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-07 Thread Henrik Ingo
On Tue, Sep 7, 2021 at 5:06 PM bened...@apache.org wrote: > > I was thinking that a path similar to Calvin/FaunaDB is certainly > looming in the horizon at least. > > I’m not sure which aspect of these systems you are referring to. Unless I > have misunderstood, I consider them to be strictly

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-07 Thread Henrik Ingo
On Tue, Sep 7, 2021 at 12:26 PM bened...@apache.org wrote: > > whether I should just* think of this as "better and more efficient LWT” > > So, the LWT concept is a Cassandra one and doesn’t have an agreed-upon > definition. My understanding of a core feature/limitation of LWTs is that > they

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-07 Thread Henrik Ingo
On Tue, Sep 7, 2021 at 1:31 AM bened...@apache.org wrote: > > Of course, but we may have to be selective in our back-and-forth. We can > always take some discussion off-list to keep it manageable. > > I'll try to converge.Sorry if a few comments were a bit "editorial" in the first message. I

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-06 Thread Henrik Ingo
alone library for integration into > Cassandra. I hope the community sees the important value proposition of > this proposal, and will adopt the CEP after this discussion, so that the > library and its integration into Cassandra can be developed in parallel and > with the involvement of t