Re: Upgrade the jna version to 4.3.0

[Amitkumar Ghatwal]

Hi Jason,

I have created a JIRA ticket :
https://issues.apache.org/jira/browse/CASSANDRA-13300 for this upgrade
activity . Please do let me know when the jna version will be upgraded on
github -https://github.com/apache/cassandra/blob/trunk/lib/ .

Regards,
Amit




From:   Jason Brown 
To: dev@cassandra.apache.org
Date:   03/05/2017 11:29 PM
Subject:Re: Upgrade the jna version to 4.3.0



Hi Amit,

Can you open a Jira for that? Also we can figure out which branches to
target for the upgrade on the Jira.

Thanks,

Jason
On Sun, Mar 5, 2017 at 08:25 Amitkumar Ghatwal  wrote:

>
>
> Hi All,
>
> Could you please upgrade the jna version present in the github cassandra
> location :
> https://github.com/apache/cassandra/blob/trunk/lib/jna-4.0.0.jar
> to below latest version   - 4.3.0 -
>
>
http://repo1.maven.org/maven2/net/java/dev/jna/jna/4.3.0/jna-4.3.0-javadoc.jar

>  .
>
> Let me know the process to upgrade the same .
>
> Regards,
> Amit
>
>
>

Re: Upgrade the jna version to 4.3.0

[Jason Brown]

Hi Amit,

Can you open a Jira for that? Also we can figure out which branches to
target for the upgrade on the Jira.

Thanks,

Jason
On Sun, Mar 5, 2017 at 08:25 Amitkumar Ghatwal  wrote:

>
>
> Hi All,
>
> Could you please upgrade the jna version present in the github cassandra
> location :
> https://github.com/apache/cassandra/blob/trunk/lib/jna-4.0.0.jar
> to below latest version   - 4.3.0 -
>
> http://repo1.maven.org/maven2/net/java/dev/jna/jna/4.3.0/jna-4.3.0-javadoc.jar
>  .
>
> Let me know the process to upgrade the same .
>
> Regards,
> Amit
>
>
>

Upgrade the jna version to 4.3.0

[Amitkumar Ghatwal]

Hi All,

Could you please upgrade the jna version present in the github cassandra
location : https://github.com/apache/cassandra/blob/trunk/lib/jna-4.0.0.jar
to below latest version   - 4.3.0 -
http://repo1.maven.org/maven2/net/java/dev/jna/jna/4.3.0/jna-4.3.0-javadoc.jar
 .

Let me know the process to upgrade the same .

Regards,
Amit

Re: State of triggers

[benjamin roth]

There is a German saying:

Sometimes you don't see the woods because of the lots of trees.

Am 05.03.2017 09:25 schrieb "DuyHai Doan" :

> No problem, distributed systems are hard to reason about, I got caught many
> times in the past
>
> On Sun, Mar 5, 2017 at 9:23 AM, benjamin roth  wrote:
>
> > Sorry. Answer was to fast. Maybe you are right.
> >
> > Am 05.03.2017 09:21 schrieb "benjamin roth" :
> >
> > > No. You just change the partitioner. That's all
> > >
> > > Am 05.03.2017 09:15 schrieb "DuyHai Doan" :
> > >
> > >> "How can that be achieved? I haven't done "scientific researches" yet
> > but
> > >> I
> > >> guess a "MV partitioner" could do the trick. Instead of applying the
> > >> regular partitioner, an MV partitioner would calculate the PK of the
> > base
> > >> table (which is always possible) and then apply the regular
> > partitioner."
> > >>
> > >> The main purpose of MV is to avoid the drawbacks of 2nd index
> > >> architecture,
> > >> e.g. to scan a lot of nodes to fetch the results.
> > >>
> > >> With MV, since you give the partition key, the guarantee is that
> you'll
> > >> hit
> > >> a single node.
> > >>
> > >> Now if you put MV data on the same node as base table data, you're
> doing
> > >> more-or-less the same thing as 2nd index.
> > >>
> > >> Let's take a dead simple example
> > >>
> > >> CREATE TABLE user (user_id uuid PRIMARY KEY, email text);
> > >> CREATE MV user_by_email AS SELECT * FROM user WHERE user_id IS NOT
> NULL
> > >> AND
> > >> email IS NOT NULL PRIMARY KEY((email),user_id);
> > >>
> > >> SELECT * FROM user_by_email WHERE email = xxx;
> > >>
> > >> With this query, how can you find the user_id that corresponds to
> email
> > >> 'xxx' so that your MV partitioner idea can work ?
> > >>
> > >>
> > >>
> > >> On Sun, Mar 5, 2017 at 9:05 AM, benjamin roth 
> wrote:
> > >>
> > >> > While I was reading the MV paragraph in your post, an idea popped
> up:
> > >> >
> > >> > The problem with MV inconsistencies and inconsistent range movement
> is
> > >> that
> > >> > the "MV contract" is broken. This only happens because base data and
> > >> > replica data reside on different hosts. If base data + replicas
> would
> > >> stay
> > >> > on the same host then a rebuild/remove would always stream both
> > matching
> > >> > parts of a base table + mv.
> > >> >
> > >> > So my idea:
> > >> > Why not make a replica ALWAYS stay local regardless where the token
> of
> > >> a MV
> > >> > would point at. That would solve these problems:
> > >> > 1. Rebuild / remove node would not break MV contract
> > >> > 2. A write always stays local:
> > >> >
> > >> > a) That means replication happens sync. That means a quorum write to
> > the
> > >> > base table guarantees instant data availability with quorum read on
> a
> > >> view
> > >> >
> > >> > b) It saves network roundtrips + request/response handling and helps
> > to
> > >> > keep a cluster healthier in case of bulk operations (like repair
> > >> streams or
> > >> > rebuild stream). Write load stays local and is not spread across the
> > >> whole
> > >> > cluster. I think it makes the load in these situations more
> > predictable.
> > >> >
> > >> > How can that be achieved? I haven't done "scientific researches" yet
> > >> but I
> > >> > guess a "MV partitioner" could do the trick. Instead of applying the
> > >> > regular partitioner, an MV partitioner would calculate the PK of the
> > >> base
> > >> > table (which is always possible) and then apply the regular
> > partitioner.
> > >> >
> > >> > I'll create a proper Jira for it on monday. Currently it's sunday
> here
> > >> and
> > >> > my family wants me back so just a few thoughts on this right now.
> > >> >
> > >> > Any feedback is appreciated!
> > >> >
> > >> > 2017-03-05 6:34 GMT+01:00 Edward Capriolo :
> > >> >
> > >> > > On Sat, Mar 4, 2017 at 10:26 AM, Jeff Jirsa 
> > wrote:
> > >> > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > > On Mar 4, 2017, at 7:06 AM, Edward Capriolo <
> > >> edlinuxg...@gmail.com>
> > >> > > > wrote:
> > >> > > > >
> > >> > > > >> On Fri, Mar 3, 2017 at 12:04 PM, Jeff Jirsa <
> jji...@gmail.com>
> > >> > wrote:
> > >> > > > >>
> > >> > > > >> On Fri, Mar 3, 2017 at 5:40 AM, Edward Capriolo <
> > >> > > edlinuxg...@gmail.com>
> > >> > > > >> wrote:
> > >> > > > >>
> > >> > > > >>>
> > >> > > > >>> I used them. I built do it yourself secondary indexes with
> > them.
> > >> > They
> > >> > > > >> have
> > >> > > > >>> there gotchas, but so do all the secondary index
> > >> implementations.
> > >> > > Just
> > >> > > > >>> because datastax does not write about something. Lets see
> > like 5
> > >> > > years
> > >> > > > >> ago
> > >> > > > >>> there was this: https://github.com/hmsonline/
> > cassandra-triggers
> > >> > > > >>>
> > >> > > > >>>
> > >> > > > >> Still in use? How'd it work? Production ready? Would you
> 

Re: State of triggers

[benjamin roth]

Not maybe. You are absolutely right. Bad idea. Hmpf.

Am 05.03.2017 09:23 schrieb "benjamin roth" :

> Sorry. Answer was to fast. Maybe you are right.
>
> Am 05.03.2017 09:21 schrieb "benjamin roth" :
>
>> No. You just change the partitioner. That's all
>>
>> Am 05.03.2017 09:15 schrieb "DuyHai Doan" :
>>
>>> "How can that be achieved? I haven't done "scientific researches" yet
>>> but I
>>> guess a "MV partitioner" could do the trick. Instead of applying the
>>> regular partitioner, an MV partitioner would calculate the PK of the base
>>> table (which is always possible) and then apply the regular partitioner."
>>>
>>> The main purpose of MV is to avoid the drawbacks of 2nd index
>>> architecture,
>>> e.g. to scan a lot of nodes to fetch the results.
>>>
>>> With MV, since you give the partition key, the guarantee is that you'll
>>> hit
>>> a single node.
>>>
>>> Now if you put MV data on the same node as base table data, you're doing
>>> more-or-less the same thing as 2nd index.
>>>
>>> Let's take a dead simple example
>>>
>>> CREATE TABLE user (user_id uuid PRIMARY KEY, email text);
>>> CREATE MV user_by_email AS SELECT * FROM user WHERE user_id IS NOT NULL
>>> AND
>>> email IS NOT NULL PRIMARY KEY((email),user_id);
>>>
>>> SELECT * FROM user_by_email WHERE email = xxx;
>>>
>>> With this query, how can you find the user_id that corresponds to email
>>> 'xxx' so that your MV partitioner idea can work ?
>>>
>>>
>>>
>>> On Sun, Mar 5, 2017 at 9:05 AM, benjamin roth  wrote:
>>>
>>> > While I was reading the MV paragraph in your post, an idea popped up:
>>> >
>>> > The problem with MV inconsistencies and inconsistent range movement is
>>> that
>>> > the "MV contract" is broken. This only happens because base data and
>>> > replica data reside on different hosts. If base data + replicas would
>>> stay
>>> > on the same host then a rebuild/remove would always stream both
>>> matching
>>> > parts of a base table + mv.
>>> >
>>> > So my idea:
>>> > Why not make a replica ALWAYS stay local regardless where the token of
>>> a MV
>>> > would point at. That would solve these problems:
>>> > 1. Rebuild / remove node would not break MV contract
>>> > 2. A write always stays local:
>>> >
>>> > a) That means replication happens sync. That means a quorum write to
>>> the
>>> > base table guarantees instant data availability with quorum read on a
>>> view
>>> >
>>> > b) It saves network roundtrips + request/response handling and helps to
>>> > keep a cluster healthier in case of bulk operations (like repair
>>> streams or
>>> > rebuild stream). Write load stays local and is not spread across the
>>> whole
>>> > cluster. I think it makes the load in these situations more
>>> predictable.
>>> >
>>> > How can that be achieved? I haven't done "scientific researches" yet
>>> but I
>>> > guess a "MV partitioner" could do the trick. Instead of applying the
>>> > regular partitioner, an MV partitioner would calculate the PK of the
>>> base
>>> > table (which is always possible) and then apply the regular
>>> partitioner.
>>> >
>>> > I'll create a proper Jira for it on monday. Currently it's sunday here
>>> and
>>> > my family wants me back so just a few thoughts on this right now.
>>> >
>>> > Any feedback is appreciated!
>>> >
>>> > 2017-03-05 6:34 GMT+01:00 Edward Capriolo :
>>> >
>>> > > On Sat, Mar 4, 2017 at 10:26 AM, Jeff Jirsa 
>>> wrote:
>>> > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > > On Mar 4, 2017, at 7:06 AM, Edward Capriolo <
>>> edlinuxg...@gmail.com>
>>> > > > wrote:
>>> > > > >
>>> > > > >> On Fri, Mar 3, 2017 at 12:04 PM, Jeff Jirsa 
>>> > wrote:
>>> > > > >>
>>> > > > >> On Fri, Mar 3, 2017 at 5:40 AM, Edward Capriolo <
>>> > > edlinuxg...@gmail.com>
>>> > > > >> wrote:
>>> > > > >>
>>> > > > >>>
>>> > > > >>> I used them. I built do it yourself secondary indexes with
>>> them.
>>> > They
>>> > > > >> have
>>> > > > >>> there gotchas, but so do all the secondary index
>>> implementations.
>>> > > Just
>>> > > > >>> because datastax does not write about something. Lets see like
>>> 5
>>> > > years
>>> > > > >> ago
>>> > > > >>> there was this: https://github.com/hmsonline/c
>>> assandra-triggers
>>> > > > >>>
>>> > > > >>>
>>> > > > >> Still in use? How'd it work? Production ready? Would you still
>>> do it
>>> > > > that
>>> > > > >> way in 2017?
>>> > > > >>
>>> > > > >>
>>> > > > >>> There is a fairly large divergence to what actual users do and
>>> what
>>> > > > other
>>> > > > >>> groups 'say' actual users do in some cases.
>>> > > > >>>
>>> > > > >>
>>> > > > >> A lot of people don't share what they're doing (for business
>>> > reasons,
>>> > > or
>>> > > > >> because they don't think it's important, or because they don't
>>> know
>>> > > > >> how/where), and that's fine but it makes it hard for anyone to
>>> know
>>> > > what
>>> > > > >> features are used, or how well 

Re: State of triggers

[DuyHai Doan]

No problem, distributed systems are hard to reason about, I got caught many
times in the past

On Sun, Mar 5, 2017 at 9:23 AM, benjamin roth  wrote:

> Sorry. Answer was to fast. Maybe you are right.
>
> Am 05.03.2017 09:21 schrieb "benjamin roth" :
>
> > No. You just change the partitioner. That's all
> >
> > Am 05.03.2017 09:15 schrieb "DuyHai Doan" :
> >
> >> "How can that be achieved? I haven't done "scientific researches" yet
> but
> >> I
> >> guess a "MV partitioner" could do the trick. Instead of applying the
> >> regular partitioner, an MV partitioner would calculate the PK of the
> base
> >> table (which is always possible) and then apply the regular
> partitioner."
> >>
> >> The main purpose of MV is to avoid the drawbacks of 2nd index
> >> architecture,
> >> e.g. to scan a lot of nodes to fetch the results.
> >>
> >> With MV, since you give the partition key, the guarantee is that you'll
> >> hit
> >> a single node.
> >>
> >> Now if you put MV data on the same node as base table data, you're doing
> >> more-or-less the same thing as 2nd index.
> >>
> >> Let's take a dead simple example
> >>
> >> CREATE TABLE user (user_id uuid PRIMARY KEY, email text);
> >> CREATE MV user_by_email AS SELECT * FROM user WHERE user_id IS NOT NULL
> >> AND
> >> email IS NOT NULL PRIMARY KEY((email),user_id);
> >>
> >> SELECT * FROM user_by_email WHERE email = xxx;
> >>
> >> With this query, how can you find the user_id that corresponds to email
> >> 'xxx' so that your MV partitioner idea can work ?
> >>
> >>
> >>
> >> On Sun, Mar 5, 2017 at 9:05 AM, benjamin roth  wrote:
> >>
> >> > While I was reading the MV paragraph in your post, an idea popped up:
> >> >
> >> > The problem with MV inconsistencies and inconsistent range movement is
> >> that
> >> > the "MV contract" is broken. This only happens because base data and
> >> > replica data reside on different hosts. If base data + replicas would
> >> stay
> >> > on the same host then a rebuild/remove would always stream both
> matching
> >> > parts of a base table + mv.
> >> >
> >> > So my idea:
> >> > Why not make a replica ALWAYS stay local regardless where the token of
> >> a MV
> >> > would point at. That would solve these problems:
> >> > 1. Rebuild / remove node would not break MV contract
> >> > 2. A write always stays local:
> >> >
> >> > a) That means replication happens sync. That means a quorum write to
> the
> >> > base table guarantees instant data availability with quorum read on a
> >> view
> >> >
> >> > b) It saves network roundtrips + request/response handling and helps
> to
> >> > keep a cluster healthier in case of bulk operations (like repair
> >> streams or
> >> > rebuild stream). Write load stays local and is not spread across the
> >> whole
> >> > cluster. I think it makes the load in these situations more
> predictable.
> >> >
> >> > How can that be achieved? I haven't done "scientific researches" yet
> >> but I
> >> > guess a "MV partitioner" could do the trick. Instead of applying the
> >> > regular partitioner, an MV partitioner would calculate the PK of the
> >> base
> >> > table (which is always possible) and then apply the regular
> partitioner.
> >> >
> >> > I'll create a proper Jira for it on monday. Currently it's sunday here
> >> and
> >> > my family wants me back so just a few thoughts on this right now.
> >> >
> >> > Any feedback is appreciated!
> >> >
> >> > 2017-03-05 6:34 GMT+01:00 Edward Capriolo :
> >> >
> >> > > On Sat, Mar 4, 2017 at 10:26 AM, Jeff Jirsa 
> wrote:
> >> > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > > On Mar 4, 2017, at 7:06 AM, Edward Capriolo <
> >> edlinuxg...@gmail.com>
> >> > > > wrote:
> >> > > > >
> >> > > > >> On Fri, Mar 3, 2017 at 12:04 PM, Jeff Jirsa 
> >> > wrote:
> >> > > > >>
> >> > > > >> On Fri, Mar 3, 2017 at 5:40 AM, Edward Capriolo <
> >> > > edlinuxg...@gmail.com>
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >>>
> >> > > > >>> I used them. I built do it yourself secondary indexes with
> them.
> >> > They
> >> > > > >> have
> >> > > > >>> there gotchas, but so do all the secondary index
> >> implementations.
> >> > > Just
> >> > > > >>> because datastax does not write about something. Lets see
> like 5
> >> > > years
> >> > > > >> ago
> >> > > > >>> there was this: https://github.com/hmsonline/
> cassandra-triggers
> >> > > > >>>
> >> > > > >>>
> >> > > > >> Still in use? How'd it work? Production ready? Would you still
> >> do it
> >> > > > that
> >> > > > >> way in 2017?
> >> > > > >>
> >> > > > >>
> >> > > > >>> There is a fairly large divergence to what actual users do and
> >> what
> >> > > > other
> >> > > > >>> groups 'say' actual users do in some cases.
> >> > > > >>>
> >> > > > >>
> >> > > > >> A lot of people don't share what they're doing (for business
> >> > reasons,
> >> > > or
> >> > > > >> because they don't think it's important, or 

Re: State of triggers

[benjamin roth]

Sorry. Answer was to fast. Maybe you are right.

Am 05.03.2017 09:21 schrieb "benjamin roth" :

> No. You just change the partitioner. That's all
>
> Am 05.03.2017 09:15 schrieb "DuyHai Doan" :
>
>> "How can that be achieved? I haven't done "scientific researches" yet but
>> I
>> guess a "MV partitioner" could do the trick. Instead of applying the
>> regular partitioner, an MV partitioner would calculate the PK of the base
>> table (which is always possible) and then apply the regular partitioner."
>>
>> The main purpose of MV is to avoid the drawbacks of 2nd index
>> architecture,
>> e.g. to scan a lot of nodes to fetch the results.
>>
>> With MV, since you give the partition key, the guarantee is that you'll
>> hit
>> a single node.
>>
>> Now if you put MV data on the same node as base table data, you're doing
>> more-or-less the same thing as 2nd index.
>>
>> Let's take a dead simple example
>>
>> CREATE TABLE user (user_id uuid PRIMARY KEY, email text);
>> CREATE MV user_by_email AS SELECT * FROM user WHERE user_id IS NOT NULL
>> AND
>> email IS NOT NULL PRIMARY KEY((email),user_id);
>>
>> SELECT * FROM user_by_email WHERE email = xxx;
>>
>> With this query, how can you find the user_id that corresponds to email
>> 'xxx' so that your MV partitioner idea can work ?
>>
>>
>>
>> On Sun, Mar 5, 2017 at 9:05 AM, benjamin roth  wrote:
>>
>> > While I was reading the MV paragraph in your post, an idea popped up:
>> >
>> > The problem with MV inconsistencies and inconsistent range movement is
>> that
>> > the "MV contract" is broken. This only happens because base data and
>> > replica data reside on different hosts. If base data + replicas would
>> stay
>> > on the same host then a rebuild/remove would always stream both matching
>> > parts of a base table + mv.
>> >
>> > So my idea:
>> > Why not make a replica ALWAYS stay local regardless where the token of
>> a MV
>> > would point at. That would solve these problems:
>> > 1. Rebuild / remove node would not break MV contract
>> > 2. A write always stays local:
>> >
>> > a) That means replication happens sync. That means a quorum write to the
>> > base table guarantees instant data availability with quorum read on a
>> view
>> >
>> > b) It saves network roundtrips + request/response handling and helps to
>> > keep a cluster healthier in case of bulk operations (like repair
>> streams or
>> > rebuild stream). Write load stays local and is not spread across the
>> whole
>> > cluster. I think it makes the load in these situations more predictable.
>> >
>> > How can that be achieved? I haven't done "scientific researches" yet
>> but I
>> > guess a "MV partitioner" could do the trick. Instead of applying the
>> > regular partitioner, an MV partitioner would calculate the PK of the
>> base
>> > table (which is always possible) and then apply the regular partitioner.
>> >
>> > I'll create a proper Jira for it on monday. Currently it's sunday here
>> and
>> > my family wants me back so just a few thoughts on this right now.
>> >
>> > Any feedback is appreciated!
>> >
>> > 2017-03-05 6:34 GMT+01:00 Edward Capriolo :
>> >
>> > > On Sat, Mar 4, 2017 at 10:26 AM, Jeff Jirsa  wrote:
>> > >
>> > > >
>> > > >
>> > > >
>> > > > > On Mar 4, 2017, at 7:06 AM, Edward Capriolo <
>> edlinuxg...@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > >> On Fri, Mar 3, 2017 at 12:04 PM, Jeff Jirsa 
>> > wrote:
>> > > > >>
>> > > > >> On Fri, Mar 3, 2017 at 5:40 AM, Edward Capriolo <
>> > > edlinuxg...@gmail.com>
>> > > > >> wrote:
>> > > > >>
>> > > > >>>
>> > > > >>> I used them. I built do it yourself secondary indexes with them.
>> > They
>> > > > >> have
>> > > > >>> there gotchas, but so do all the secondary index
>> implementations.
>> > > Just
>> > > > >>> because datastax does not write about something. Lets see like 5
>> > > years
>> > > > >> ago
>> > > > >>> there was this: https://github.com/hmsonline/cassandra-triggers
>> > > > >>>
>> > > > >>>
>> > > > >> Still in use? How'd it work? Production ready? Would you still
>> do it
>> > > > that
>> > > > >> way in 2017?
>> > > > >>
>> > > > >>
>> > > > >>> There is a fairly large divergence to what actual users do and
>> what
>> > > > other
>> > > > >>> groups 'say' actual users do in some cases.
>> > > > >>>
>> > > > >>
>> > > > >> A lot of people don't share what they're doing (for business
>> > reasons,
>> > > or
>> > > > >> because they don't think it's important, or because they don't
>> know
>> > > > >> how/where), and that's fine but it makes it hard for anyone to
>> know
>> > > what
>> > > > >> features are used, or how well they're really working in
>> production.
>> > > > >>
>> > > > >> I've seen a handful of "how do we use triggers" questions in IRC,
>> > and
>> > > > they
>> > > > >> weren't unreasonable questions, but seemed like a lot of pain,
>> and
>> > > more
>> > > > >> than one of those people 

Re: State of triggers

[benjamin roth]

No. You just change the partitioner. That's all

Am 05.03.2017 09:15 schrieb "DuyHai Doan" :

> "How can that be achieved? I haven't done "scientific researches" yet but I
> guess a "MV partitioner" could do the trick. Instead of applying the
> regular partitioner, an MV partitioner would calculate the PK of the base
> table (which is always possible) and then apply the regular partitioner."
>
> The main purpose of MV is to avoid the drawbacks of 2nd index architecture,
> e.g. to scan a lot of nodes to fetch the results.
>
> With MV, since you give the partition key, the guarantee is that you'll hit
> a single node.
>
> Now if you put MV data on the same node as base table data, you're doing
> more-or-less the same thing as 2nd index.
>
> Let's take a dead simple example
>
> CREATE TABLE user (user_id uuid PRIMARY KEY, email text);
> CREATE MV user_by_email AS SELECT * FROM user WHERE user_id IS NOT NULL AND
> email IS NOT NULL PRIMARY KEY((email),user_id);
>
> SELECT * FROM user_by_email WHERE email = xxx;
>
> With this query, how can you find the user_id that corresponds to email
> 'xxx' so that your MV partitioner idea can work ?
>
>
>
> On Sun, Mar 5, 2017 at 9:05 AM, benjamin roth  wrote:
>
> > While I was reading the MV paragraph in your post, an idea popped up:
> >
> > The problem with MV inconsistencies and inconsistent range movement is
> that
> > the "MV contract" is broken. This only happens because base data and
> > replica data reside on different hosts. If base data + replicas would
> stay
> > on the same host then a rebuild/remove would always stream both matching
> > parts of a base table + mv.
> >
> > So my idea:
> > Why not make a replica ALWAYS stay local regardless where the token of a
> MV
> > would point at. That would solve these problems:
> > 1. Rebuild / remove node would not break MV contract
> > 2. A write always stays local:
> >
> > a) That means replication happens sync. That means a quorum write to the
> > base table guarantees instant data availability with quorum read on a
> view
> >
> > b) It saves network roundtrips + request/response handling and helps to
> > keep a cluster healthier in case of bulk operations (like repair streams
> or
> > rebuild stream). Write load stays local and is not spread across the
> whole
> > cluster. I think it makes the load in these situations more predictable.
> >
> > How can that be achieved? I haven't done "scientific researches" yet but
> I
> > guess a "MV partitioner" could do the trick. Instead of applying the
> > regular partitioner, an MV partitioner would calculate the PK of the base
> > table (which is always possible) and then apply the regular partitioner.
> >
> > I'll create a proper Jira for it on monday. Currently it's sunday here
> and
> > my family wants me back so just a few thoughts on this right now.
> >
> > Any feedback is appreciated!
> >
> > 2017-03-05 6:34 GMT+01:00 Edward Capriolo :
> >
> > > On Sat, Mar 4, 2017 at 10:26 AM, Jeff Jirsa  wrote:
> > >
> > > >
> > > >
> > > >
> > > > > On Mar 4, 2017, at 7:06 AM, Edward Capriolo  >
> > > > wrote:
> > > > >
> > > > >> On Fri, Mar 3, 2017 at 12:04 PM, Jeff Jirsa 
> > wrote:
> > > > >>
> > > > >> On Fri, Mar 3, 2017 at 5:40 AM, Edward Capriolo <
> > > edlinuxg...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>>
> > > > >>> I used them. I built do it yourself secondary indexes with them.
> > They
> > > > >> have
> > > > >>> there gotchas, but so do all the secondary index implementations.
> > > Just
> > > > >>> because datastax does not write about something. Lets see like 5
> > > years
> > > > >> ago
> > > > >>> there was this: https://github.com/hmsonline/cassandra-triggers
> > > > >>>
> > > > >>>
> > > > >> Still in use? How'd it work? Production ready? Would you still do
> it
> > > > that
> > > > >> way in 2017?
> > > > >>
> > > > >>
> > > > >>> There is a fairly large divergence to what actual users do and
> what
> > > > other
> > > > >>> groups 'say' actual users do in some cases.
> > > > >>>
> > > > >>
> > > > >> A lot of people don't share what they're doing (for business
> > reasons,
> > > or
> > > > >> because they don't think it's important, or because they don't
> know
> > > > >> how/where), and that's fine but it makes it hard for anyone to
> know
> > > what
> > > > >> features are used, or how well they're really working in
> production.
> > > > >>
> > > > >> I've seen a handful of "how do we use triggers" questions in IRC,
> > and
> > > > they
> > > > >> weren't unreasonable questions, but seemed like a lot of pain, and
> > > more
> > > > >> than one of those people ultimately came back and said they used
> > some
> > > > other
> > > > >> mechanism (and of course, some of them silently disappear, so we
> > have
> > > no
> > > > >> idea if it worked or not).
> > > > >>
> > > > >> If anyone's actively using triggers, please don't keep it 

Re: State of triggers

[DuyHai Doan]

"How can that be achieved? I haven't done "scientific researches" yet but I
guess a "MV partitioner" could do the trick. Instead of applying the
regular partitioner, an MV partitioner would calculate the PK of the base
table (which is always possible) and then apply the regular partitioner."

The main purpose of MV is to avoid the drawbacks of 2nd index architecture,
e.g. to scan a lot of nodes to fetch the results.

With MV, since you give the partition key, the guarantee is that you'll hit
a single node.

Now if you put MV data on the same node as base table data, you're doing
more-or-less the same thing as 2nd index.

Let's take a dead simple example

CREATE TABLE user (user_id uuid PRIMARY KEY, email text);
CREATE MV user_by_email AS SELECT * FROM user WHERE user_id IS NOT NULL AND
email IS NOT NULL PRIMARY KEY((email),user_id);

SELECT * FROM user_by_email WHERE email = xxx;

With this query, how can you find the user_id that corresponds to email
'xxx' so that your MV partitioner idea can work ?



On Sun, Mar 5, 2017 at 9:05 AM, benjamin roth  wrote:

> While I was reading the MV paragraph in your post, an idea popped up:
>
> The problem with MV inconsistencies and inconsistent range movement is that
> the "MV contract" is broken. This only happens because base data and
> replica data reside on different hosts. If base data + replicas would stay
> on the same host then a rebuild/remove would always stream both matching
> parts of a base table + mv.
>
> So my idea:
> Why not make a replica ALWAYS stay local regardless where the token of a MV
> would point at. That would solve these problems:
> 1. Rebuild / remove node would not break MV contract
> 2. A write always stays local:
>
> a) That means replication happens sync. That means a quorum write to the
> base table guarantees instant data availability with quorum read on a view
>
> b) It saves network roundtrips + request/response handling and helps to
> keep a cluster healthier in case of bulk operations (like repair streams or
> rebuild stream). Write load stays local and is not spread across the whole
> cluster. I think it makes the load in these situations more predictable.
>
> How can that be achieved? I haven't done "scientific researches" yet but I
> guess a "MV partitioner" could do the trick. Instead of applying the
> regular partitioner, an MV partitioner would calculate the PK of the base
> table (which is always possible) and then apply the regular partitioner.
>
> I'll create a proper Jira for it on monday. Currently it's sunday here and
> my family wants me back so just a few thoughts on this right now.
>
> Any feedback is appreciated!
>
> 2017-03-05 6:34 GMT+01:00 Edward Capriolo :
>
> > On Sat, Mar 4, 2017 at 10:26 AM, Jeff Jirsa  wrote:
> >
> > >
> > >
> > >
> > > > On Mar 4, 2017, at 7:06 AM, Edward Capriolo 
> > > wrote:
> > > >
> > > >> On Fri, Mar 3, 2017 at 12:04 PM, Jeff Jirsa 
> wrote:
> > > >>
> > > >> On Fri, Mar 3, 2017 at 5:40 AM, Edward Capriolo <
> > edlinuxg...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>>
> > > >>> I used them. I built do it yourself secondary indexes with them.
> They
> > > >> have
> > > >>> there gotchas, but so do all the secondary index implementations.
> > Just
> > > >>> because datastax does not write about something. Lets see like 5
> > years
> > > >> ago
> > > >>> there was this: https://github.com/hmsonline/cassandra-triggers
> > > >>>
> > > >>>
> > > >> Still in use? How'd it work? Production ready? Would you still do it
> > > that
> > > >> way in 2017?
> > > >>
> > > >>
> > > >>> There is a fairly large divergence to what actual users do and what
> > > other
> > > >>> groups 'say' actual users do in some cases.
> > > >>>
> > > >>
> > > >> A lot of people don't share what they're doing (for business
> reasons,
> > or
> > > >> because they don't think it's important, or because they don't know
> > > >> how/where), and that's fine but it makes it hard for anyone to know
> > what
> > > >> features are used, or how well they're really working in production.
> > > >>
> > > >> I've seen a handful of "how do we use triggers" questions in IRC,
> and
> > > they
> > > >> weren't unreasonable questions, but seemed like a lot of pain, and
> > more
> > > >> than one of those people ultimately came back and said they used
> some
> > > other
> > > >> mechanism (and of course, some of them silently disappear, so we
> have
> > no
> > > >> idea if it worked or not).
> > > >>
> > > >> If anyone's actively using triggers, please don't keep it a secret.
> > > Knowing
> > > >> that they're being used would be a great way to justify continuing
> to
> > > >> maintain them.
> > > >>
> > > >> - Jeff
> > > >>
> > > >
> > > > "Still in use? How'd it work? Production ready? Would you still do it
> > > that way in 2017?"
> > > >
> > > > I mean that is a loaded question. How long has cassandra had
> Secondary
> > > > 

Re: State of triggers

[benjamin roth]

While I was reading the MV paragraph in your post, an idea popped up:

The problem with MV inconsistencies and inconsistent range movement is that
the "MV contract" is broken. This only happens because base data and
replica data reside on different hosts. If base data + replicas would stay
on the same host then a rebuild/remove would always stream both matching
parts of a base table + mv.

So my idea:
Why not make a replica ALWAYS stay local regardless where the token of a MV
would point at. That would solve these problems:
1. Rebuild / remove node would not break MV contract
2. A write always stays local:

a) That means replication happens sync. That means a quorum write to the
base table guarantees instant data availability with quorum read on a view

b) It saves network roundtrips + request/response handling and helps to
keep a cluster healthier in case of bulk operations (like repair streams or
rebuild stream). Write load stays local and is not spread across the whole
cluster. I think it makes the load in these situations more predictable.

How can that be achieved? I haven't done "scientific researches" yet but I
guess a "MV partitioner" could do the trick. Instead of applying the
regular partitioner, an MV partitioner would calculate the PK of the base
table (which is always possible) and then apply the regular partitioner.

I'll create a proper Jira for it on monday. Currently it's sunday here and
my family wants me back so just a few thoughts on this right now.

Any feedback is appreciated!

2017-03-05 6:34 GMT+01:00 Edward Capriolo :

> On Sat, Mar 4, 2017 at 10:26 AM, Jeff Jirsa  wrote:
>
> >
> >
> >
> > > On Mar 4, 2017, at 7:06 AM, Edward Capriolo 
> > wrote:
> > >
> > >> On Fri, Mar 3, 2017 at 12:04 PM, Jeff Jirsa  wrote:
> > >>
> > >> On Fri, Mar 3, 2017 at 5:40 AM, Edward Capriolo <
> edlinuxg...@gmail.com>
> > >> wrote:
> > >>
> > >>>
> > >>> I used them. I built do it yourself secondary indexes with them. They
> > >> have
> > >>> there gotchas, but so do all the secondary index implementations.
> Just
> > >>> because datastax does not write about something. Lets see like 5
> years
> > >> ago
> > >>> there was this: https://github.com/hmsonline/cassandra-triggers
> > >>>
> > >>>
> > >> Still in use? How'd it work? Production ready? Would you still do it
> > that
> > >> way in 2017?
> > >>
> > >>
> > >>> There is a fairly large divergence to what actual users do and what
> > other
> > >>> groups 'say' actual users do in some cases.
> > >>>
> > >>
> > >> A lot of people don't share what they're doing (for business reasons,
> or
> > >> because they don't think it's important, or because they don't know
> > >> how/where), and that's fine but it makes it hard for anyone to know
> what
> > >> features are used, or how well they're really working in production.
> > >>
> > >> I've seen a handful of "how do we use triggers" questions in IRC, and
> > they
> > >> weren't unreasonable questions, but seemed like a lot of pain, and
> more
> > >> than one of those people ultimately came back and said they used some
> > other
> > >> mechanism (and of course, some of them silently disappear, so we have
> no
> > >> idea if it worked or not).
> > >>
> > >> If anyone's actively using triggers, please don't keep it a secret.
> > Knowing
> > >> that they're being used would be a great way to justify continuing to
> > >> maintain them.
> > >>
> > >> - Jeff
> > >>
> > >
> > > "Still in use? How'd it work? Production ready? Would you still do it
> > that way in 2017?"
> > >
> > > I mean that is a loaded question. How long has cassandra had Secondary
> > > Indexes? Did they work well? Would you use them? How many times were
> > they re-written?
> >
> > It wasn't really meant to be a loaded question; I was being sincere
> >
> > But I'll answer: secondary indexes suck for many use cases, but they're
> > invaluable for their actual intended purpose, and I have no idea how many
> > times they've been rewritten but they're production ready for their
> narrow
> > use case (defined by cardinality).
> >
> > Is there a real triggers use case still? Alternative to MVs? Alternative
> > to CDC? I've never implemented triggers - since you have, what's the
> level
> > of surprise for the developer?
>
>
> :) You mention alternatives/: Lets break them down.
>
> MV:
> They seem to have a lot pf promise. IE you can use them for things other
> then equality searches, and I do think the CQL example with the top N high
> scores is pretty useful. Then again our buddy Mr Roth has a thread named
> "Rebuild / remove node with MV is inconsistent". I actually think a lot of
> the use case for mv falls into the category of "something you should
> actually be doing with storm". I can vibe with the concept of not needing a
> streaming platform, but i KNOW storm would do this correctly. I don't want
> to land on something like 2x index v1 v2 where there was