Ok, thanks. Igor
On Oct 13, 2016 4:37 PM, "Valentin Kulichenko" < valentin.kuliche...@gmail.com> wrote: > Here is the ticket: https://issues.apache.org/jira/browse/IGNITE-4075 > > -Val > > On Wed, Oct 12, 2016 at 6:45 PM, Igor Rudyak <irud...@gmail.com> wrote: > >> Hi Val, >> >> I don't have any objections - please create a ticket and link it to the >> root ticket https://issues.apache.org/jira/browse/IGNITE-1371 >> >> Igor >> >> On Wed, Oct 12, 2016 at 4:10 PM, Valentin Kulichenko < >> valentin.kuliche...@gmail.com> wrote: >> >>> Hi Igor, >>> >>> 1. I still think we should do this. Loading nothing is very >>> counterintuitive and prevents a newbie user from quick start. For large >>> tables, when only part of the dataset is needed, user will explicitly >>> specify the query, of course. Do you have objections? If no, I will create >>> a ticket. >>> >>> 2. Got it, thanks. >>> >>> -Val >>> >>> On Mon, Oct 10, 2016 at 12:12 AM, Igor Rudyak <irud...@gmail.com> wrote: >>> >>>> Hi Val, >>>> >>>> 1) Well, it's not a problem to implement such default behavior, but >>>> there is one concern. In most cases, when you are using Cassandra as a >>>> persistent store you are going to store large amount of data, which is >>>> significantly bigger that amount of RAM in your Ignite cluster. In the such >>>> case it doesn't make sense to launch CQL query like "select * from >>>> my_table" cause: >>>> a) You still will not be able to keep all data from Cassandra table >>>> in Ignite cache >>>> b) All the data will be pulled from Cassandra table using only one >>>> thread - which is very slow >>>> >>>> 2) Unfortunately it's not possible in Cassandra. For JDBC you are >>>> splitting table into chunks of 512 rows each, using sub-queries and >>>> ordering by primary keys. Such kind of things are not supported in >>>> Cassandra. Probably the only way to load data from Cassandra table in >>>> parallel, is to load it from some specified partitions (in parallel for >>>> each partition). >>>> >>>> >>>> Igor Rudyak >>>> >>>> On Fri, Oct 7, 2016 at 1:45 PM, Valentin Kulichenko < >>>> valentin.kuliche...@gmail.com> wrote: >>>> >>>>> Hi Igor, >>>>> >>>>> Thanks for response! >>>>> >>>>> 1. It's a bit inconsistent with other store implementations we have in >>>>> the product and actually I find this counterintuitive. Why don't we just >>>>> load all the data available in the table? Explicit query is useful when >>>>> you >>>>> want to customize this and load subset of data based on some criteria. If >>>>> this is not possible for some reason, then I would at least throw an >>>>> exception in case query is not specified. >>>>> >>>>> 2. Is it possible to automatically split the data in bulks and load >>>>> them in parallel? We do this in the JDBC store, for example. >>>>> >>>>> -Val >>>>> >>>>> On Thu, Oct 6, 2016 at 11:00 PM, Igor Rudyak <irud...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi Val, >>>>>> >>>>>> 1) If you'll call loadCache(null) it will do nothing. You need to >>>>>> provide at least one CQL query. >>>>>> >>>>>> 2) It depends. If you'll provide more than one CQL query, it will use >>>>>> separate thread for each of the queries (max number of threads limited to >>>>>> the number of CPU cores). But for each provided CQL query it will use >>>>>> only >>>>>> one thread to load all the data returned by the query. Also it will run >>>>>> the >>>>>> same CQL query from ALL Ignite nodes to load the same data, which is bad. >>>>>> That's because loadCache method will be executed on each Ignite node. As >>>>>> you see, it's not very efficient way to load data from Cassandra just by >>>>>> specifying CQL query. The ticket I created, is all about how to load data >>>>>> from one table (or from multiple tables as well) in parallel by >>>>>> partitioning it. Such a way each Ignite node will be responsible to load >>>>>> data from the specific partition range of Cassandra table, which is much >>>>>> more efficient. To support such kind of cache warm-up you should design >>>>>> your Cassandra table specific way - there should be some mapping from >>>>>> Ignite partition to the set of Cassandra partitions. Yes I have plans to >>>>>> implement this. >>>>>> >>>>>> Igor Rudyak >>>>>> >>>>>> >>>>>> On Thu, Oct 6, 2016 at 10:19 AM, Valentin Kulichenko < >>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>> >>>>>>> Hi Igor, >>>>>>> >>>>>>> I've got couple of quick questions about the Cassandra store. >>>>>>> >>>>>>> 1. In [1] you suggested to provide an explicit query as a >>>>>>> parameter for loadCache() method, because otherwise user was always >>>>>>> getting >>>>>>> empty result. Is this a requirement to provide the query? What if I >>>>>>> just >>>>>>> call loadCache(null)? >>>>>>> 2. There is a ticket [2] about parallel load in Cassandra store. >>>>>>> Does it mean that currently it loads only in a single threaded >>>>>>> fashion? If >>>>>>> so, do you have any plans to implement this improvement? >>>>>>> >>>>>>> [1] http://apache-ignite-users.70518.x6.nabble.com/Cannot-qu >>>>>>> ery-on-a-cache-using-Cassandra-as-a-persistent-store-td7870.html >>>>>>> [2] https://gridgain.freshdesk.com/helpdesk/tickets/2180 >>>>>>> >>>>>>> Thanks, >>>>>>> Val >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
