Re: Cassandra Mutation object decoding

[J. D. Jordan]

You may also want to look at the triggers interface.

> On Nov 22, 2016, at 7:45 PM, Chris Lohfink  wrote:
> 
> There are different kinds of tombstones, a partition tombstone is held in
> the MutableDeletionInfo of the PartitionUpdate that you can get from
> deletionInfo() method which returns the private deletionInfo field from the
> holder. There are also row and cell deletions so you have to handle each of
> those. It can be a little non-trivial to work backwards but all the
> information is there in the Mutation (if you have full access to cfmetadata
> etc), may be easier to go directly to whatever output your looking for.
> 
> A lot of the metadata is driven from system though, you need to be either
> be on a Cassandra node or rebuild them with the same cfids. In sstable-tools
>  we rebuild the tables from the
> source cluster to parse the Mutations for replaying/viewing hints and
> commitlogs. But that had to be a bit (massively) hackier since its for 3.7
> before CASSANDRA-8844 .
> Its definitely possible but not easy (which is probably why it hasn't been
> added in yet).
> 
> Chris
> 
> On Tue, Nov 22, 2016 at 6:59 PM, Sanal Vasudevan 
> wrote:
> 
>> Hi Bejamin,
>> 
>> Nice to hear from you.
>> 
>> My goal is to reconstruct the CQL operation from the Mutation object.
>> So that I can trigger the same action on another NoSQL target like MongoDB.
>> 
>> Please let me know know if you have ideas?
>> 
>> Many thanks.
>> Sanal
>> 
>> 
>> On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer <
>> benjamin.le...@datastax.com
>>> wrote:
>> 
>>> Hi Sanal,
>>> 
>>> What you want to do is not an easy stuff and it might break with new
>> major
>>> releases.
>>> 
>>> My question would be: why do you want to do that? There might be another
>>> way to reach the same goal.
>>> 
>>> Benjamin
>>> 
>>> On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
>>> wrote:
>>> 
 Thank you Vladimir.
 Anyone else has any other ideas as to how this can be done?
 
 
 Many thanks,
 Sanal
 
 
 On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin <
>> vla...@winguzone.com>
 wrote:
 
> Hi Sanal,
> 
> 
> 
> do we have metadata inside Mutation object to decode whether the
>>> CQL
> was an INSERT or UPDATE operation?
> 
> I'm not sure it's possible to distinguish them - both of them just
>> add
> data to SSTable.
> 
> 
> 
> 
> 
> Best regards, Vladimir Yudovin,
> 
> Winguzone - Hosted Cloud Cassandra
> Launch your cluster in minutes.
> 
> 
> 
> 
> 
>  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> lt;get2sa...@gmail.com wrote 
> 
> 
> 
> 
> Hi there,
> 
> 
> 
> I am trying to read the Commit logs to decode the original CQL which
 used.
> 
> I get to the point an implemention of CommitLogReadHandler is able to
 push
> 
> back Mutation objects from the Commit logs.
> 
> 
> 
> Questions:
> 
> 1) CQL: delete from myks.mytable where key1 = 1;
> 
> For the above CQL, the Mutation object has zero objects of
> 
> org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
> 
> Is this the only way to detect a DELETE operation? or we have any
>> other
> 
> metadata to indicate a DELETE operation?
> 
> mutation.getPartitionUpdates().forEach(rows - {
>>> if(rows.isEmpty())
> 
> System.out.println("May be a DELETE operation") });
> 
> 2) Likewise do we have metadata inside Mutation object to decode
>>> whether
> 
> the CQL was an INSERT or UPDATE operation?
> 
> 
> 
> Josh Mckenzie indicated that PartitionUpdate.deletionInfo
> 
> (MutableDeletionInfo) may have some information but deletionInfo is
> private.
> 
> 
> 
> Basically, I am looking for help to find a way to classify Mutation
 object
> 
> to INSERT/UPDATE/DELETE with related column and key information.
> 
> 
> 
> Many thanks.
> 
> --
> 
> Sanal
> 
> 
> 
> 
> 
> 
> 
 
 
 --
 Sanal Vasudevan Nair
 
>>> 
>> 
>> 
>> 
>> --
>> Sanal Vasudevan Nair
>> 

Re: Wiki Contributer

[Dave Brosius]

If your wiki-name was ThomasBrown, then you are added, if not let me 
know what it is.



On 11/22/2016 06:33 PM, Thomas Brown wrote:

Hi,

Could I please be added as a Wiki Contributor

Thanks

Re: Cassandra Mutation object decoding

[Nate McCall]

Have you explored the QueryProcessor interface? That is a legitimate
extension point and may be a more suitable layer at which to
integrate.

On Wed, Nov 23, 2016 at 1:59 PM, Sanal Vasudevan  wrote:
> Hi Bejamin,
>
> Nice to hear from you.
>
> My goal is to reconstruct the CQL operation from the Mutation object.
> So that I can trigger the same action on another NoSQL target like MongoDB.
>
> Please let me know know if you have ideas?
>
> Many thanks.
> Sanal
>
>
> On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer > wrote:
>
>> Hi Sanal,
>>
>> What you want to do is not an easy stuff and it might break with new major
>> releases.
>>
>> My question would be: why do you want to do that? There might be another
>> way to reach the same goal.
>>
>> Benjamin
>>
>> On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
>> wrote:
>>
>> > Thank you Vladimir.
>> > Anyone else has any other ideas as to how this can be done?
>> >
>> >
>> > Many thanks,
>> > Sanal
>> >
>> >
>> > On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin 
>> > wrote:
>> >
>> > > Hi Sanal,
>> > >
>> > >
>> > >
>> > > do we have metadata inside Mutation object to decode whether the
>> CQL
>> > > was an INSERT or UPDATE operation?
>> > >
>> > > I'm not sure it's possible to distinguish them - both of them just add
>> > > data to SSTable.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > Best regards, Vladimir Yudovin,
>> > >
>> > > Winguzone - Hosted Cloud Cassandra
>> > > Launch your cluster in minutes.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
>> > > lt;get2sa...@gmail.com wrote 
>> > >
>> > >
>> > >
>> > >
>> > > Hi there,
>> > >
>> > >
>> > >
>> > > I am trying to read the Commit logs to decode the original CQL which
>> > used.
>> > >
>> > > I get to the point an implemention of CommitLogReadHandler is able to
>> > push
>> > >
>> > > back Mutation objects from the Commit logs.
>> > >
>> > >
>> > >
>> > > Questions:
>> > >
>> > > 1) CQL: delete from myks.mytable where key1 = 1;
>> > >
>> > > For the above CQL, the Mutation object has zero objects of
>> > >
>> > > org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
>> > >
>> > > Is this the only way to detect a DELETE operation? or we have any other
>> > >
>> > > metadata to indicate a DELETE operation?
>> > >
>> > >  mutation.getPartitionUpdates().forEach(rows - {
>> if(rows.isEmpty())
>> > >
>> > > System.out.println("May be a DELETE operation") });
>> > >
>> > > 2) Likewise do we have metadata inside Mutation object to decode
>> whether
>> > >
>> > > the CQL was an INSERT or UPDATE operation?
>> > >
>> > >
>> > >
>> > > Josh Mckenzie indicated that PartitionUpdate.deletionInfo
>> > >
>> > > (MutableDeletionInfo) may have some information but deletionInfo is
>> > > private.
>> > >
>> > >
>> > >
>> > > Basically, I am looking for help to find a way to classify Mutation
>> > object
>> > >
>> > > to INSERT/UPDATE/DELETE with related column and key information.
>> > >
>> > >
>> > >
>> > > Many thanks.
>> > >
>> > > --
>> > >
>> > > Sanal
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> >
>> >
>> > --
>> > Sanal Vasudevan Nair
>> >
>>
>
>
>
> --
> Sanal Vasudevan Nair

Re: Cassandra Mutation object decoding

[Chris Lohfink]

There are different kinds of tombstones, a partition tombstone is held in
the MutableDeletionInfo of the PartitionUpdate that you can get from
deletionInfo() method which returns the private deletionInfo field from the
holder. There are also row and cell deletions so you have to handle each of
those. It can be a little non-trivial to work backwards but all the
information is there in the Mutation (if you have full access to cfmetadata
etc), may be easier to go directly to whatever output your looking for.

A lot of the metadata is driven from system though, you need to be either
be on a Cassandra node or rebuild them with the same cfids. In sstable-tools
 we rebuild the tables from the
source cluster to parse the Mutations for replaying/viewing hints and
commitlogs. But that had to be a bit (massively) hackier since its for 3.7
before CASSANDRA-8844 .
Its definitely possible but not easy (which is probably why it hasn't been
added in yet).

Chris

On Tue, Nov 22, 2016 at 6:59 PM, Sanal Vasudevan 
wrote:

> Hi Bejamin,
>
> Nice to hear from you.
>
> My goal is to reconstruct the CQL operation from the Mutation object.
> So that I can trigger the same action on another NoSQL target like MongoDB.
>
> Please let me know know if you have ideas?
>
> Many thanks.
> Sanal
>
>
> On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer <
> benjamin.le...@datastax.com
> > wrote:
>
> > Hi Sanal,
> >
> > What you want to do is not an easy stuff and it might break with new
> major
> > releases.
> >
> > My question would be: why do you want to do that? There might be another
> > way to reach the same goal.
> >
> > Benjamin
> >
> > On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
> > wrote:
> >
> > > Thank you Vladimir.
> > > Anyone else has any other ideas as to how this can be done?
> > >
> > >
> > > Many thanks,
> > > Sanal
> > >
> > >
> > > On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin <
> vla...@winguzone.com>
> > > wrote:
> > >
> > > > Hi Sanal,
> > > >
> > > >
> > > >
> > > > do we have metadata inside Mutation object to decode whether the
> > CQL
> > > > was an INSERT or UPDATE operation?
> > > >
> > > > I'm not sure it's possible to distinguish them - both of them just
> add
> > > > data to SSTable.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Best regards, Vladimir Yudovin,
> > > >
> > > > Winguzone - Hosted Cloud Cassandra
> > > > Launch your cluster in minutes.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> > > > lt;get2sa...@gmail.com wrote 
> > > >
> > > >
> > > >
> > > >
> > > > Hi there,
> > > >
> > > >
> > > >
> > > > I am trying to read the Commit logs to decode the original CQL which
> > > used.
> > > >
> > > > I get to the point an implemention of CommitLogReadHandler is able to
> > > push
> > > >
> > > > back Mutation objects from the Commit logs.
> > > >
> > > >
> > > >
> > > > Questions:
> > > >
> > > > 1) CQL: delete from myks.mytable where key1 = 1;
> > > >
> > > > For the above CQL, the Mutation object has zero objects of
> > > >
> > > > org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
> > > >
> > > > Is this the only way to detect a DELETE operation? or we have any
> other
> > > >
> > > > metadata to indicate a DELETE operation?
> > > >
> > > >  mutation.getPartitionUpdates().forEach(rows - {
> > if(rows.isEmpty())
> > > >
> > > > System.out.println("May be a DELETE operation") });
> > > >
> > > > 2) Likewise do we have metadata inside Mutation object to decode
> > whether
> > > >
> > > > the CQL was an INSERT or UPDATE operation?
> > > >
> > > >
> > > >
> > > > Josh Mckenzie indicated that PartitionUpdate.deletionInfo
> > > >
> > > > (MutableDeletionInfo) may have some information but deletionInfo is
> > > > private.
> > > >
> > > >
> > > >
> > > > Basically, I am looking for help to find a way to classify Mutation
> > > object
> > > >
> > > > to INSERT/UPDATE/DELETE with related column and key information.
> > > >
> > > >
> > > >
> > > > Many thanks.
> > > >
> > > > --
> > > >
> > > > Sanal
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Sanal Vasudevan Nair
> > >
> >
>
>
>
> --
> Sanal Vasudevan Nair
>

Re: Cassandra Mutation object decoding

[Sanal Vasudevan]

Hi Bejamin,

Nice to hear from you.

My goal is to reconstruct the CQL operation from the Mutation object.
So that I can trigger the same action on another NoSQL target like MongoDB.

Please let me know know if you have ideas?

Many thanks.
Sanal


On Tue, Nov 22, 2016 at 7:28 PM, Benjamin Lerer  wrote:

> Hi Sanal,
>
> What you want to do is not an easy stuff and it might break with new major
> releases.
>
> My question would be: why do you want to do that? There might be another
> way to reach the same goal.
>
> Benjamin
>
> On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
> wrote:
>
> > Thank you Vladimir.
> > Anyone else has any other ideas as to how this can be done?
> >
> >
> > Many thanks,
> > Sanal
> >
> >
> > On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin 
> > wrote:
> >
> > > Hi Sanal,
> > >
> > >
> > >
> > > do we have metadata inside Mutation object to decode whether the
> CQL
> > > was an INSERT or UPDATE operation?
> > >
> > > I'm not sure it's possible to distinguish them - both of them just add
> > > data to SSTable.
> > >
> > >
> > >
> > >
> > >
> > > Best regards, Vladimir Yudovin,
> > >
> > > Winguzone - Hosted Cloud Cassandra
> > > Launch your cluster in minutes.
> > >
> > >
> > >
> > >
> > >
> > >  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> > > lt;get2sa...@gmail.com wrote 
> > >
> > >
> > >
> > >
> > > Hi there,
> > >
> > >
> > >
> > > I am trying to read the Commit logs to decode the original CQL which
> > used.
> > >
> > > I get to the point an implemention of CommitLogReadHandler is able to
> > push
> > >
> > > back Mutation objects from the Commit logs.
> > >
> > >
> > >
> > > Questions:
> > >
> > > 1) CQL: delete from myks.mytable where key1 = 1;
> > >
> > > For the above CQL, the Mutation object has zero objects of
> > >
> > > org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
> > >
> > > Is this the only way to detect a DELETE operation? or we have any other
> > >
> > > metadata to indicate a DELETE operation?
> > >
> > >  mutation.getPartitionUpdates().forEach(rows - {
> if(rows.isEmpty())
> > >
> > > System.out.println("May be a DELETE operation") });
> > >
> > > 2) Likewise do we have metadata inside Mutation object to decode
> whether
> > >
> > > the CQL was an INSERT or UPDATE operation?
> > >
> > >
> > >
> > > Josh Mckenzie indicated that PartitionUpdate.deletionInfo
> > >
> > > (MutableDeletionInfo) may have some information but deletionInfo is
> > > private.
> > >
> > >
> > >
> > > Basically, I am looking for help to find a way to classify Mutation
> > object
> > >
> > > to INSERT/UPDATE/DELETE with related column and key information.
> > >
> > >
> > >
> > > Many thanks.
> > >
> > > --
> > >
> > > Sanal
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Sanal Vasudevan Nair
> >
>



-- 
Sanal Vasudevan Nair

Wiki Contributer

[Thomas Brown]

Hi,

Could I please be added as a Wiki Contributor

Thanks

-- 
*Thomas Brown*
*Digital Marketing Associate *
m: +61427864972




MANAGED.  SUPPORT.
 CONSULTING.


   

Read our latest technical blog posts here
.

This email has been sent on behalf of Instaclustr Limited (Australia)
and Instaclustr Inc (USA).

This email and any attachments may contain confidential and legally
privileged information.  If you are not the intended recipient, do not copy
or disclose its content, but please reply to this email immediately and
highlight the error to the sender and then immediately delete the message.

[GitHub] cassandra issue #85: sasi: AND optimizations

[blerer]

Github user blerer commented on the issue:

https://github.com/apache/cassandra/pull/85
  
Thanks. :-)
People are usually tracking the JIRA tickets and not the pull requests. So 
it is best to keep it in sync with your developments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cassandra issue #85: sasi: AND optimizations

[iksaif]

Github user iksaif commented on the issue:

https://github.com/apache/cassandra/pull/85
  
Done, I'm not yet very familiar with the workflow.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cassandra issue #85: sasi: AND optimizations

[blerer]

Github user blerer commented on the issue:

https://github.com/apache/cassandra/pull/85
  
I guess it is CASSANDRA-12915. Could you mark it as patch available?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cassandra issue #85: sasi: AND optimizations

[blerer]

Github user blerer commented on the issue:

https://github.com/apache/cassandra/pull/85
  
Is there a JIRA ticket associated to your patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cassandra pull request #85: sasi: AND optimizations

[iksaif]

GitHub user iksaif opened a pull request:

https://github.com/apache/cassandra/pull/85

sasi: AND optimizations

when op is AND:
- do not use indexes that return way more results that the others
- do not use more indexes if one of them returns no results
- try to use something smarter to get the primary index (which is
  important because it limits the searches done on other indexes)

Cosmetic changes such as changing logger.debug() to logger.trace() will be 
done later, I'd like a first opinion on the concept.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/iksaif/cassandra cassandra-3.X

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cassandra/pull/85.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #85


commit bc54f7ae176e9314190c49d1780fb87e26b62728
Author: Corentin Chary 
Date:   2016-11-21T16:04:42Z

sasi: AND optimizations

when op is AND:
- do not use indexes that return way more results that the others
- do not use more indexes if one of them returns no results
- try to use something smarter to get the primary index (which is
  important because it limits the searches done on other indexes)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Cassandra Mutation object decoding

[Benjamin Lerer]

Hi Sanal,

What you want to do is not an easy stuff and it might break with new major
releases.

My question would be: why do you want to do that? There might be another
way to reach the same goal.

Benjamin

On Mon, Nov 21, 2016 at 7:14 PM, Sanal Vasudevan 
wrote:

> Thank you Vladimir.
> Anyone else has any other ideas as to how this can be done?
>
>
> Many thanks,
> Sanal
>
>
> On Sun, Nov 20, 2016 at 4:46 AM, Vladimir Yudovin 
> wrote:
>
> > Hi Sanal,
> >
> >
> >
> > do we have metadata inside Mutation object to decode whether the CQL
> > was an INSERT or UPDATE operation?
> >
> > I'm not sure it's possible to distinguish them - both of them just add
> > data to SSTable.
> >
> >
> >
> >
> >
> > Best regards, Vladimir Yudovin,
> >
> > Winguzone - Hosted Cloud Cassandra
> > Launch your cluster in minutes.
> >
> >
> >
> >
> >
> >  On Fri, 18 Nov 2016 15:55:00 -0500Sanal Vasudevan &
> > lt;get2sa...@gmail.com wrote 
> >
> >
> >
> >
> > Hi there,
> >
> >
> >
> > I am trying to read the Commit logs to decode the original CQL which
> used.
> >
> > I get to the point an implemention of CommitLogReadHandler is able to
> push
> >
> > back Mutation objects from the Commit logs.
> >
> >
> >
> > Questions:
> >
> > 1) CQL: delete from myks.mytable where key1 = 1;
> >
> > For the above CQL, the Mutation object has zero objects of
> >
> > org.apache.cassandra.db.rows.Row inside ParitionUpdate object.
> >
> > Is this the only way to detect a DELETE operation? or we have any other
> >
> > metadata to indicate a DELETE operation?
> >
> >  mutation.getPartitionUpdates().forEach(rows - { if(rows.isEmpty())
> >
> > System.out.println("May be a DELETE operation") });
> >
> > 2) Likewise do we have metadata inside Mutation object to decode whether
> >
> > the CQL was an INSERT or UPDATE operation?
> >
> >
> >
> > Josh Mckenzie indicated that PartitionUpdate.deletionInfo
> >
> > (MutableDeletionInfo) may have some information but deletionInfo is
> > private.
> >
> >
> >
> > Basically, I am looking for help to find a way to classify Mutation
> object
> >
> > to INSERT/UPDATE/DELETE with related column and key information.
> >
> >
> >
> > Many thanks.
> >
> > --
> >
> > Sanal
> >
> >
> >
> >
> >
> >
> >
>
>
> --
> Sanal Vasudevan Nair
>