Re: Cassandra store questions

[Valentin Kulichenko]

Hi Igor,

Thanks for response!

1. It's a bit inconsistent with other store implementations we have in the
product and actually I find this counterintuitive. Why don't we just load
all the data available in the table? Explicit query is useful when you want
to customize this and load subset of data based on some criteria. If this
is not possible for some reason, then I would at least throw an exception
in case query is not specified.

2. Is it possible to automatically split the data in bulks and load them in
parallel? We do this in the JDBC store, for example.

-Val

On Thu, Oct 6, 2016 at 11:00 PM, Igor Rudyak  wrote:

> Hi Val,
>
> 1) If you'll call loadCache(null) it will do nothing. You need to provide
> at least one CQL query.
>
> 2) It depends. If you'll provide more than one CQL query, it will use
> separate thread for each of the queries (max number of threads limited to
> the number of CPU cores). But for each provided CQL query it will use only
> one thread to load all the data returned by the query. Also it will run the
> same CQL query from ALL Ignite nodes to load the same data, which is bad.
> That's because loadCache method will be executed on each Ignite node. As
> you see, it's not very efficient way to load data from Cassandra just by
> specifying CQL query. The ticket I created, is all about how to load data
> from one table (or from multiple tables as well) in parallel by
> partitioning it. Such a way each Ignite node will be responsible to load
> data from the specific partition range of Cassandra table, which is much
> more efficient. To support such kind of cache warm-up you should design
> your Cassandra table specific way - there should be some mapping from
> Ignite partition to the set of Cassandra partitions. Yes I have plans to
> implement this.
>
> Igor Rudyak
>
>
> On Thu, Oct 6, 2016 at 10:19 AM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
>> Hi Igor,
>>
>> I've got couple of quick questions about the Cassandra store.
>>
>>1. In [1] you suggested to provide an explicit query as a parameter
>>for loadCache() method, because otherwise user was always getting empty
>>result. Is this a requirement to provide the query? What if I just call
>>loadCache(null)?
>>2. There is a ticket [2] about parallel load in Cassandra store. Does
>>it mean that currently it loads only in a single threaded fashion? If so,
>>do you have any plans to implement this improvement?
>>
>> [1] http://apache-ignite-users.70518.x6.nabble.com/Cannot-
>> query-on-a-cache-using-Cassandra-as-a-persistent-store-td7870.html
>> [2] https://gridgain.freshdesk.com/helpdesk/tickets/2180
>>
>> Thanks,
>> Val
>>
>
>

Re: Where and how to store metrics for queries.

[Dmitriy Setrakyan]

To my knowledge we already use MongoDB in web console for storing user
profile data and other settings. How about you store all the metrics in
MongoDB as well?

On Thu, Oct 6, 2016 at 6:26 PM, Alexey Kuznetsov 
wrote:

> Dima & Val,
>
> My first intention was to store metrics locally and collect them on demand.
>
> But after that I start to think about following scenarios:
>
> What if we have a grid with clients that connected to grid, execute some
> query and *disconnected*?
> In this case we would have no option to collect metrics about what queries
> were executed on grid.
>
> Also, very often clients may resided in a *slow* network segment and
> sending a task for collecting metrics
>  from *fast server nodes* and *slow clients* may take a lot of time in case
> of large grid.
>
> Could you suggest me how I could deal with two mentioned above cases?
>
> Or we could ignore them for now?
>
> --
> Alexey Kuznetsov
>

[GitHub] ignite pull request #1112: IGNITE-3963 FlinkIgniteSinkSelfTest fails with ex...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1112


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-4047) CQ skips events if autoUnsubscribe flag was set to false

[Nikolay Tikhonov (JIRA)]

Nikolay Tikhonov created IGNITE-4047:


 Summary: CQ skips events if autoUnsubscribe flag was set to false
 Key: IGNITE-4047
 URL: https://issues.apache.org/jira/browse/IGNITE-4047
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.7
Reporter: Nikolay Tikhonov


CQ skips events after topology change, if {{autoUnsubscribe}} flag was set to 
{{false}}. Test attached.

See 
http://apache-ignite-users.70518.x6.nabble.com/CacheContinuousQuery-did-not-work-after-the-second-server-node-joinned-into-the-topology-td8011.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1152: Ignite 11581

[DmitriyGovorukhin]

Github user DmitriyGovorukhin closed the pull request at:

https://github.com/apache/ignite/pull/1152


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Cassandra store questions

[Igor Rudyak]

Hi Val,

1) If you'll call loadCache(null) it will do nothing. You need to provide
at least one CQL query.

2) It depends. If you'll provide more than one CQL query, it will use
separate thread for each of the queries (max number of threads limited to
the number of CPU cores). But for each provided CQL query it will use only
one thread to load all the data returned by the query. Also it will run the
same CQL query from ALL Ignite nodes to load the same data, which is bad.
That's because loadCache method will be executed on each Ignite node. As
you see, it's not very efficient way to load data from Cassandra just by
specifying CQL query. The ticket I created, is all about how to load data
from one table (or from multiple tables as well) in parallel by
partitioning it. Such a way each Ignite node will be responsible to load
data from the specific partition range of Cassandra table, which is much
more efficient. To support such kind of cache warm-up you should design
your Cassandra table specific way - there should be some mapping from
Ignite partition to the set of Cassandra partitions. Yes I have plans to
implement this.

Igor Rudyak


On Thu, Oct 6, 2016 at 10:19 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Hi Igor,
>
> I've got couple of quick questions about the Cassandra store.
>
>1. In [1] you suggested to provide an explicit query as a parameter
>for loadCache() method, because otherwise user was always getting empty
>result. Is this a requirement to provide the query? What if I just call
>loadCache(null)?
>2. There is a ticket [2] about parallel load in Cassandra store. Does
>it mean that currently it loads only in a single threaded fashion? If so,
>do you have any plans to implement this improvement?
>
> [1] http://apache-ignite-users.70518.x6.nabble.com/
> Cannot-query-on-a-cache-using-Cassandra-as-a-persistent-store-td7870.html
> [2] https://gridgain.freshdesk.com/helpdesk/tickets/2180
>
> Thanks,
> Val
>