Re: A question to 'paging' support in DataStax java driver

[Sebastian Estevez]

I didn't read the whole thread last time around, please disregard my
comment about the java driver jira.

One other thought (hopefully relevant this time). Once we have
https://issues.apache.org/jira/browse/CASSANDRA-10783, you could write a
write a (*start*, *rows*) style paging UDF which would allow you to read
just page 4 for example. Granted you will still have to *scan* the data
from 0 to start at the server and throw it away, but might get you closer
to what you are looking for.




All the best,


[image: datastax_logo.png] <http://www.datastax.com/>

Sebastián Estévez

Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com

[image: linkedin.png] <https://www.linkedin.com/company/datastax> [image:
facebook.png] <https://www.facebook.com/datastax> [image: twitter.png]
<https://twitter.com/datastax> [image: g+.png]
<https://plus.google.com/+Datastax/about>
<http://feeds.feedburner.com/datastax>
<http://goog_410786983>


<http://www.datastax.com/gartner-magic-quadrant-odbms>

DataStax is the fastest, most scalable distributed database technology,
delivering Apache Cassandra to the world’s most innovative enterprises.
Datastax is built to be agile, always-on, and predictably scalable to any
size. With more than 500 customers in 45 countries, DataStax is the
database technology and transactional backbone of choice for the worlds
most innovative companies such as Netflix, Adobe, Intuit, and eBay.

On Tue, May 10, 2016 at 9:23 AM, Sebastian Estevez <
sebastian.este...@datastax.com> wrote:

> I think this request belongs in the java driver jira not the Cassandra
> jira.
>
> https://datastax-oss.atlassian.net/projects/JAVA/
>
> all the best,
>
> Sebastián
> On May 10, 2016 1:09 AM, "Lu, Boying" <boying...@emc.com> wrote:
>
>> I filed a JIRA https://issues.apache.org/jira/browse/CASSANDRA-11741 to
>> track this.
>>
>>
>>
>> *From:* DuyHai Doan [mailto:doanduy...@gmail.com]
>> *Sent:* 2016年5月10日 12:47
>> *To:* user@cassandra.apache.org
>> *Subject:* Re: A question to 'paging' support in DataStax java driver
>>
>>
>>
>> I guess it's technically possible but then we'll need to update the
>> binary protocol. Just create a JIRA and ask for this feature
>>
>>
>>
>> On Tue, May 10, 2016 at 5:00 AM, Lu, Boying <boying...@emc.com> wrote:
>>
>> Thanks very much.
>>
>>
>>
>> I understand that the data needs to be read from the DB to get the next
>> ‘PagingState’.
>>
>>
>>
>> But is it possible not to return those data to the client side, just
>> returning the ‘PagingState’?
>>
>> I.e. the data is read on the server side, but not return to client side,
>> this can save some bandwidth
>>
>> between client and server.
>>
>>
>>
>>
>>
>> *From:* DuyHai Doan [mailto:doanduy...@gmail.com]
>> *Sent:* 2016年5月9日 21:06
>> *To:* user@cassandra.apache.org
>> *Subject:* Re: A question to 'paging' support in DataStax java driver
>>
>>
>>
>> In a truly consistent world (should I say "snapshot isolation" world
>> instead), re-reading the same page should yield the same results no matter
>> how many new inserts have occurred since the last page read.
>>
>>
>>
>> Caching previous page at app level can be a solution but not viable if
>> the amount of data is huge, also you'll need a cache layer and deal with
>> cache invalidation etc ...
>>
>>
>>
>> The point is, providing snapshot isolation in a distributed system is
>> hard without some sort of synchronous coordination e.g. global lock (read
>> http://www.bailis.org/papers/hat-vldb2014.pdf)
>>
>>
>>
>>
>>
>> On Mon, May 9, 2016 at 2:17 PM, Bhuvan Rawal <bhu1ra...@gmail.com> wrote:
>>
>> Hi Doan,
>>
>>
>>
>> What does it have to do being eventual consistency? Lets assume a
>> scenario with complete consistency and we are at page X, and at the same
>> time some inserts/updates happened at page X-2 and we jumped to that.
>>
>> User will see inconsistent page in that case as well, right? Also in such
>> cases how would you design a user facing application (Cache previous pages
>> at app level?)
>>
>>
>>
>> Regards,
>>
>> Bhuvan
>>
>>
>>
>> On Mon, May 9, 2016 at 4:18 PM, DuyHai Doan <doanduy...@gmail.com> wrote:
>>
>> "Is it possible to just return PagingState object without returning
>> data?" --> No
>>
>>
>>
>> Simply because before reading the actual data for

RE: A question to 'paging' support in DataStax java driver

[Sebastian Estevez]

I think this request belongs in the java driver jira not the Cassandra jira.

https://datastax-oss.atlassian.net/projects/JAVA/

all the best,

Sebastián
On May 10, 2016 1:09 AM, "Lu, Boying" <boying...@emc.com> wrote:

> I filed a JIRA https://issues.apache.org/jira/browse/CASSANDRA-11741 to
> track this.
>
>
>
> *From:* DuyHai Doan [mailto:doanduy...@gmail.com]
> *Sent:* 2016年5月10日 12:47
> *To:* user@cassandra.apache.org
> *Subject:* Re: A question to 'paging' support in DataStax java driver
>
>
>
> I guess it's technically possible but then we'll need to update the binary
> protocol. Just create a JIRA and ask for this feature
>
>
>
> On Tue, May 10, 2016 at 5:00 AM, Lu, Boying <boying...@emc.com> wrote:
>
> Thanks very much.
>
>
>
> I understand that the data needs to be read from the DB to get the next
> ‘PagingState’.
>
>
>
> But is it possible not to return those data to the client side, just
> returning the ‘PagingState’?
>
> I.e. the data is read on the server side, but not return to client side,
> this can save some bandwidth
>
> between client and server.
>
>
>
>
>
> *From:* DuyHai Doan [mailto:doanduy...@gmail.com]
> *Sent:* 2016年5月9日 21:06
> *To:* user@cassandra.apache.org
> *Subject:* Re: A question to 'paging' support in DataStax java driver
>
>
>
> In a truly consistent world (should I say "snapshot isolation" world
> instead), re-reading the same page should yield the same results no matter
> how many new inserts have occurred since the last page read.
>
>
>
> Caching previous page at app level can be a solution but not viable if the
> amount of data is huge, also you'll need a cache layer and deal with cache
> invalidation etc ...
>
>
>
> The point is, providing snapshot isolation in a distributed system is hard
> without some sort of synchronous coordination e.g. global lock (read
> http://www.bailis.org/papers/hat-vldb2014.pdf)
>
>
>
>
>
> On Mon, May 9, 2016 at 2:17 PM, Bhuvan Rawal <bhu1ra...@gmail.com> wrote:
>
> Hi Doan,
>
>
>
> What does it have to do being eventual consistency? Lets assume a scenario
> with complete consistency and we are at page X, and at the same time some
> inserts/updates happened at page X-2 and we jumped to that.
>
> User will see inconsistent page in that case as well, right? Also in such
> cases how would you design a user facing application (Cache previous pages
> at app level?)
>
>
>
> Regards,
>
> Bhuvan
>
>
>
> On Mon, May 9, 2016 at 4:18 PM, DuyHai Doan <doanduy...@gmail.com> wrote:
>
> "Is it possible to just return PagingState object without returning
> data?" --> No
>
>
>
> Simply because before reading the actual data for each page of N rows, you
> cannot know at which token value a page of data starts...
>
>
>
> And it is worst than that, with paging you don't have any isolation. Let's
> suppose you keep in your application/web front-end the paging states for
> page 1, 2 and 3. Since there are concurrent inserts on the cluster at the
> same time, when you re-use the paging state 2 for example, you may not get
> the same results as the previous read.
>
>
>
> And it is inevitable in an eventual consistent distributed DB world
>
>
>
> On Mon, May 9, 2016 at 12:25 PM, Lu, Boying <boying...@emc.com> wrote:
>
> dHi, All,
>
>
>
> We are considering to use DataStax java driver in our codes. One important
> feature provided by the driver we want to use is ‘paging’.
>
> But according to the
> https://datastax.github.io/java-driver/3.0.0/manual/paging/, it seems
> that we can’t jump between pages.
>
>
>
> Is it possible to just return PagingState object without returning data?
> e.g.  If I want to jump to the page 5 from the page 1,
>
> I need to go through each page from page 1 to page 5,  Is it possible to
> just return the PagingState object of page 1, 2, 3 and 4 without
>
> actual data of each page? This can save some bandwidth at least.
>
>
>
> Thanks in advance.
>
>
>
> Boying
>
>
>
>
>
>
>
>
>
>
>
>
>

RE: A question to 'paging' support in DataStax java driver

[Lu, Boying]

I filed a JIRA https://issues.apache.org/jira/browse/CASSANDRA-11741 to track 
this.

From: DuyHai Doan [mailto:doanduy...@gmail.com]
Sent: 2016年5月10日 12:47
To: user@cassandra.apache.org
Subject: Re: A question to 'paging' support in DataStax java driver

I guess it's technically possible but then we'll need to update the binary 
protocol. Just create a JIRA and ask for this feature

On Tue, May 10, 2016 at 5:00 AM, Lu, Boying 
<boying...@emc.com<mailto:boying...@emc.com>> wrote:
Thanks very much.

I understand that the data needs to be read from the DB to get the next 
‘PagingState’.

But is it possible not to return those data to the client side, just returning 
the ‘PagingState’?
I.e. the data is read on the server side, but not return to client side, this 
can save some bandwidth
between client and server.


From: DuyHai Doan [mailto:doanduy...@gmail.com<mailto:doanduy...@gmail.com>]
Sent: 2016年5月9日 21:06
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: A question to 'paging' support in DataStax java driver

In a truly consistent world (should I say "snapshot isolation" world instead), 
re-reading the same page should yield the same results no matter how many new 
inserts have occurred since the last page read.

Caching previous page at app level can be a solution but not viable if the 
amount of data is huge, also you'll need a cache layer and deal with cache 
invalidation etc ...

The point is, providing snapshot isolation in a distributed system is hard 
without some sort of synchronous coordination e.g. global lock (read 
http://www.bailis.org/papers/hat-vldb2014.pdf)


On Mon, May 9, 2016 at 2:17 PM, Bhuvan Rawal 
<bhu1ra...@gmail.com<mailto:bhu1ra...@gmail.com>> wrote:
Hi Doan,

What does it have to do being eventual consistency? Lets assume a scenario with 
complete consistency and we are at page X, and at the same time some 
inserts/updates happened at page X-2 and we jumped to that.
User will see inconsistent page in that case as well, right? Also in such cases 
how would you design a user facing application (Cache previous pages at app 
level?)

Regards,
Bhuvan

On Mon, May 9, 2016 at 4:18 PM, DuyHai Doan 
<doanduy...@gmail.com<mailto:doanduy...@gmail.com>> wrote:
"Is it possible to just return PagingState object without returning data?" --> 
No

Simply because before reading the actual data for each page of N rows, you 
cannot know at which token value a page of data starts...

And it is worst than that, with paging you don't have any isolation. Let's 
suppose you keep in your application/web front-end the paging states for page 
1, 2 and 3. Since there are concurrent inserts on the cluster at the same time, 
when you re-use the paging state 2 for example, you may not get the same 
results as the previous read.

And it is inevitable in an eventual consistent distributed DB world

On Mon, May 9, 2016 at 12:25 PM, Lu, Boying 
<boying...@emc.com<mailto:boying...@emc.com>> wrote:
dHi, All,

We are considering to use DataStax java driver in our codes. One important 
feature provided by the driver we want to use is ‘paging’.
But according to the 
https://datastax.github.io/java-driver/3.0.0/manual/paging/, it seems that we 
can’t jump between pages.

Is it possible to just return PagingState object without returning data? e.g.  
If I want to jump to the page 5 from the page 1,
I need to go through each page from page 1 to page 5,  Is it possible to just 
return the PagingState object of page 1, 2, 3 and 4 without
actual data of each page? This can save some bandwidth at least.

Thanks in advance.

Boying

Re: A question to 'paging' support in DataStax java driver

[DuyHai Doan]

I guess it's technically possible but then we'll need to update the binary
protocol. Just create a JIRA and ask for this feature

On Tue, May 10, 2016 at 5:00 AM, Lu, Boying <boying...@emc.com> wrote:

> Thanks very much.
>
>
>
> I understand that the data needs to be read from the DB to get the next
> ‘PagingState’.
>
>
>
> But is it possible not to return those data to the client side, just
> returning the ‘PagingState’?
>
> I.e. the data is read on the server side, but not return to client side,
> this can save some bandwidth
>
> between client and server.
>
>
>
>
>
> *From:* DuyHai Doan [mailto:doanduy...@gmail.com]
> *Sent:* 2016年5月9日 21:06
> *To:* user@cassandra.apache.org
> *Subject:* Re: A question to 'paging' support in DataStax java driver
>
>
>
> In a truly consistent world (should I say "snapshot isolation" world
> instead), re-reading the same page should yield the same results no matter
> how many new inserts have occurred since the last page read.
>
>
>
> Caching previous page at app level can be a solution but not viable if the
> amount of data is huge, also you'll need a cache layer and deal with cache
> invalidation etc ...
>
>
>
> The point is, providing snapshot isolation in a distributed system is hard
> without some sort of synchronous coordination e.g. global lock (read
> http://www.bailis.org/papers/hat-vldb2014.pdf)
>
>
>
>
>
> On Mon, May 9, 2016 at 2:17 PM, Bhuvan Rawal <bhu1ra...@gmail.com> wrote:
>
> Hi Doan,
>
>
>
> What does it have to do being eventual consistency? Lets assume a scenario
> with complete consistency and we are at page X, and at the same time some
> inserts/updates happened at page X-2 and we jumped to that.
>
> User will see inconsistent page in that case as well, right? Also in such
> cases how would you design a user facing application (Cache previous pages
> at app level?)
>
>
>
> Regards,
>
> Bhuvan
>
>
>
> On Mon, May 9, 2016 at 4:18 PM, DuyHai Doan <doanduy...@gmail.com> wrote:
>
> "Is it possible to just return PagingState object without returning
> data?" --> No
>
>
>
> Simply because before reading the actual data for each page of N rows, you
> cannot know at which token value a page of data starts...
>
>
>
> And it is worst than that, with paging you don't have any isolation. Let's
> suppose you keep in your application/web front-end the paging states for
> page 1, 2 and 3. Since there are concurrent inserts on the cluster at the
> same time, when you re-use the paging state 2 for example, you may not get
> the same results as the previous read.
>
>
>
> And it is inevitable in an eventual consistent distributed DB world
>
>
>
> On Mon, May 9, 2016 at 12:25 PM, Lu, Boying <boying...@emc.com> wrote:
>
> dHi, All,
>
>
>
> We are considering to use DataStax java driver in our codes. One important
> feature provided by the driver we want to use is ‘paging’.
>
> But according to the
> https://datastax.github.io/java-driver/3.0.0/manual/paging/, it seems
> that we can’t jump between pages.
>
>
>
> Is it possible to just return PagingState object without returning data?
> e.g.  If I want to jump to the page 5 from the page 1,
>
> I need to go through each page from page 1 to page 5,  Is it possible to
> just return the PagingState object of page 1, 2, 3 and 4 without
>
> actual data of each page? This can save some bandwidth at least.
>
>
>
> Thanks in advance.
>
>
>
> Boying
>
>
>
>
>
>
>
>
>
>
>

RE: A question to 'paging' support in DataStax java driver

[Lu, Boying]

Thanks very much.

I understand that the data needs to be read from the DB to get the next 
‘PagingState’.

But is it possible not to return those data to the client side, just returning 
the ‘PagingState’?
I.e. the data is read on the server side, but not return to client side, this 
can save some bandwidth
between client and server.


From: DuyHai Doan [mailto:doanduy...@gmail.com]
Sent: 2016年5月9日 21:06
To: user@cassandra.apache.org
Subject: Re: A question to 'paging' support in DataStax java driver

In a truly consistent world (should I say "snapshot isolation" world instead), 
re-reading the same page should yield the same results no matter how many new 
inserts have occurred since the last page read.

Caching previous page at app level can be a solution but not viable if the 
amount of data is huge, also you'll need a cache layer and deal with cache 
invalidation etc ...

The point is, providing snapshot isolation in a distributed system is hard 
without some sort of synchronous coordination e.g. global lock (read 
http://www.bailis.org/papers/hat-vldb2014.pdf)


On Mon, May 9, 2016 at 2:17 PM, Bhuvan Rawal 
<bhu1ra...@gmail.com<mailto:bhu1ra...@gmail.com>> wrote:
Hi Doan,

What does it have to do being eventual consistency? Lets assume a scenario with 
complete consistency and we are at page X, and at the same time some 
inserts/updates happened at page X-2 and we jumped to that.
User will see inconsistent page in that case as well, right? Also in such cases 
how would you design a user facing application (Cache previous pages at app 
level?)

Regards,
Bhuvan

On Mon, May 9, 2016 at 4:18 PM, DuyHai Doan 
<doanduy...@gmail.com<mailto:doanduy...@gmail.com>> wrote:
"Is it possible to just return PagingState object without returning data?" --> 
No

Simply because before reading the actual data for each page of N rows, you 
cannot know at which token value a page of data starts...

And it is worst than that, with paging you don't have any isolation. Let's 
suppose you keep in your application/web front-end the paging states for page 
1, 2 and 3. Since there are concurrent inserts on the cluster at the same time, 
when you re-use the paging state 2 for example, you may not get the same 
results as the previous read.

And it is inevitable in an eventual consistent distributed DB world

On Mon, May 9, 2016 at 12:25 PM, Lu, Boying 
<boying...@emc.com<mailto:boying...@emc.com>> wrote:
dHi, All,

We are considering to use DataStax java driver in our codes. One important 
feature provided by the driver we want to use is ‘paging’.
But according to the 
https://datastax.github.io/java-driver/3.0.0/manual/paging/, it seems that we 
can’t jump between pages.

Is it possible to just return PagingState object without returning data? e.g.  
If I want to jump to the page 5 from the page 1,
I need to go through each page from page 1 to page 5,  Is it possible to just 
return the PagingState object of page 1, 2, 3 and 4 without
actual data of each page? This can save some bandwidth at least.

Thanks in advance.

Boying

Re: A question to 'paging' support in DataStax java driver

[DuyHai Doan]

In a truly consistent world (should I say "snapshot isolation" world
instead), re-reading the same page should yield the same results no matter
how many new inserts have occurred since the last page read.

Caching previous page at app level can be a solution but not viable if the
amount of data is huge, also you'll need a cache layer and deal with cache
invalidation etc ...

The point is, providing snapshot isolation in a distributed system is hard
without some sort of synchronous coordination e.g. global lock (read
http://www.bailis.org/papers/hat-vldb2014.pdf)


On Mon, May 9, 2016 at 2:17 PM, Bhuvan Rawal  wrote:

> Hi Doan,
>
> What does it have to do being eventual consistency? Lets assume a scenario
> with complete consistency and we are at page X, and at the same time some
> inserts/updates happened at page X-2 and we jumped to that.
> User will see inconsistent page in that case as well, right? Also in such
> cases how would you design a user facing application (Cache previous pages
> at app level?)
>
> Regards,
> Bhuvan
>
> On Mon, May 9, 2016 at 4:18 PM, DuyHai Doan  wrote:
>
>> "Is it possible to just return PagingState object without returning
>> data?" --> No
>>
>> Simply because before reading the actual data for each page of N rows,
>> you cannot know at which token value a page of data starts...
>>
>> And it is worst than that, with paging you don't have any isolation.
>> Let's suppose you keep in your application/web front-end the paging states
>> for page 1, 2 and 3. Since there are concurrent inserts on the cluster at
>> the same time, when you re-use the paging state 2 for example, you may not
>> get the same results as the previous read.
>>
>> And it is inevitable in an eventual consistent distributed DB world
>>
>> On Mon, May 9, 2016 at 12:25 PM, Lu, Boying  wrote:
>>
>>> dHi, All,
>>>
>>>
>>>
>>> We are considering to use DataStax java driver in our codes. One
>>> important feature provided by the driver we want to use is ‘paging’.
>>>
>>> But according to the
>>> https://datastax.github.io/java-driver/3.0.0/manual/paging/, it seems
>>> that we can’t jump between pages.
>>>
>>>
>>>
>>> Is it possible to just return PagingState object without returning data?
>>> e.g.  If I want to jump to the page 5 from the page 1,
>>>
>>> I need to go through each page from page 1 to page 5,  Is it possible to
>>> just return the PagingState object of page 1, 2, 3 and 4 without
>>>
>>> actual data of each page? This can save some bandwidth at least.
>>>
>>>
>>>
>>> Thanks in advance.
>>>
>>>
>>>
>>> Boying
>>>
>>>
>>>
>>>
>>>
>>
>>
>

Re: A question to 'paging' support in DataStax java driver

[Bhuvan Rawal]

Hi Doan,

What does it have to do being eventual consistency? Lets assume a scenario
with complete consistency and we are at page X, and at the same time some
inserts/updates happened at page X-2 and we jumped to that.
User will see inconsistent page in that case as well, right? Also in such
cases how would you design a user facing application (Cache previous pages
at app level?)

Regards,
Bhuvan

On Mon, May 9, 2016 at 4:18 PM, DuyHai Doan  wrote:

> "Is it possible to just return PagingState object without returning
> data?" --> No
>
> Simply because before reading the actual data for each page of N rows, you
> cannot know at which token value a page of data starts...
>
> And it is worst than that, with paging you don't have any isolation. Let's
> suppose you keep in your application/web front-end the paging states for
> page 1, 2 and 3. Since there are concurrent inserts on the cluster at the
> same time, when you re-use the paging state 2 for example, you may not get
> the same results as the previous read.
>
> And it is inevitable in an eventual consistent distributed DB world
>
> On Mon, May 9, 2016 at 12:25 PM, Lu, Boying  wrote:
>
>> dHi, All,
>>
>>
>>
>> We are considering to use DataStax java driver in our codes. One
>> important feature provided by the driver we want to use is ‘paging’.
>>
>> But according to the
>> https://datastax.github.io/java-driver/3.0.0/manual/paging/, it seems
>> that we can’t jump between pages.
>>
>>
>>
>> Is it possible to just return PagingState object without returning data?
>> e.g.  If I want to jump to the page 5 from the page 1,
>>
>> I need to go through each page from page 1 to page 5,  Is it possible to
>> just return the PagingState object of page 1, 2, 3 and 4 without
>>
>> actual data of each page? This can save some bandwidth at least.
>>
>>
>>
>> Thanks in advance.
>>
>>
>>
>> Boying
>>
>>
>>
>>
>>
>
>

Re: A question to 'paging' support in DataStax java driver

[DuyHai Doan]

"Is it possible to just return PagingState object without returning data?"
--> No

Simply because before reading the actual data for each page of N rows, you
cannot know at which token value a page of data starts...

And it is worst than that, with paging you don't have any isolation. Let's
suppose you keep in your application/web front-end the paging states for
page 1, 2 and 3. Since there are concurrent inserts on the cluster at the
same time, when you re-use the paging state 2 for example, you may not get
the same results as the previous read.

And it is inevitable in an eventual consistent distributed DB world

On Mon, May 9, 2016 at 12:25 PM, Lu, Boying  wrote:

> dHi, All,
>
>
>
> We are considering to use DataStax java driver in our codes. One important
> feature provided by the driver we want to use is ‘paging’.
>
> But according to the
> https://datastax.github.io/java-driver/3.0.0/manual/paging/, it seems
> that we can’t jump between pages.
>
>
>
> Is it possible to just return PagingState object without returning data?
> e.g.  If I want to jump to the page 5 from the page 1,
>
> I need to go through each page from page 1 to page 5,  Is it possible to
> just return the PagingState object of page 1, 2, 3 and 4 without
>
> actual data of each page? This can save some bandwidth at least.
>
>
>
> Thanks in advance.
>
>
>
> Boying
>
>
>
>
>

A question to 'paging' support in DataStax java driver

[Lu, Boying]

dHi, All,

We are considering to use DataStax java driver in our codes. One important 
feature provided by the driver we want to use is 'paging'.
But according to the 
https://datastax.github.io/java-driver/3.0.0/manual/paging/, it seems that we 
can't jump between pages.

Is it possible to just return PagingState object without returning data? e.g.  
If I want to jump to the page 5 from the page 1,
I need to go through each page from page 1 to page 5,  Is it possible to just 
return the PagingState object of page 1, 2, 3 and 4 without
actual data of each page? This can save some bandwidth at least.

Thanks in advance.

Boying