I like the new method on IgniteCache API: <R> QueryCursor<R> query(Query<E> qry, Transformer<E, R> transformer);
I do agree that it makes little sense for the SqlFieldsQuery, but we can simply throw an unsupported exception here. For all other queries it does make sense. Val, can you put together all the thoughts expressed in this thread into a ticket? D. On Thu, Feb 4, 2016 at 4:14 PM, Andrey Gura <[email protected]> wrote: > Val, > > From my point of view, special query class that allows transforming is > confusing. Two points about it: > > 1. ScanTransformQuery will duplicate ScanQuery code with all drawbacks that > we have. Moreover, any fix for ScanQuery should be repeated for > ScanTransformQuery. It will lead to bugs. DRY. > > 2. If some users want to do transformations for SqlQuery we will introduce > SqlTransformQuery. Right? At this point see previous item. > > Transformation logic is some kind of strategy. IMHO, the most convenient > API should get transformation logic as some function provided by user. > > On Thu, Feb 4, 2016 at 11:05 PM, Valentin Kulichenko < > [email protected]> wrote: > > > Agree with Sergi. Mixing SQL with Java code transformers is confusing. In > > rare case when it's really required, user can implement a custom > function. > > > > I copy-pasted the API to the ticket [1]. Please provide any additional > > comments there. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-2546 > > > > -Val > > > > On Thu, Feb 4, 2016 at 10:08 AM, Andrey Gura <[email protected]> wrote: > > > > > Sergi, > > > > > > > > > > What you are going to transform remotely here? > > > > > > > > > I'm not going. I believe :) > > > > > > Just hypothetical use case: You have one SqlFieldsQuery but different > > > requirements for returned values. For one case you have to return some > > > string fields as is and for another case you have to trim string to 32 > > > characters. Of course you still can trim strings locally but > transformers > > > allow you do it remotely. > > > > > > Anyway, this solution can be usefull for rest query types. > > > > > > On Thu, Feb 4, 2016 at 8:54 PM, Sergi Vladykin < > [email protected] > > > > > > wrote: > > > > > > > The whole point of Transformer is to do remote transform, how will > this > > > > work with SqlFieldsQuery? What you are going to transform remotely > > here? > > > I > > > > believe all the transformations must happen at SQL level here. > > > > > > > > Sergi > > > > > > > > > > > > > > > > 2016-02-04 20:10 GMT+03:00 Andrey Gura <[email protected]>: > > > > > > > > > SqlQuery, TextQuery and SpiQuery are similar to ScanQuery because > all > > > of > > > > > them also defined as Query<Cache.Entry<>>. > > > > > > > > > > It can be usefull to have one query SqlQuery that can return > > different > > > > > result that will be produced from cache entry by transformer. > > > > > > > > > > Actually only SqlFieldsQuery has different definition. So > > transformers > > > > can > > > > > be applied to any type of query (including SqlFieldsQuery, I > > believe). > > > > > > > > > > On Thu, Feb 4, 2016 at 7:42 PM, Sergi Vladykin < > > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > I don't like the idea of having additional method *query(Query<E> > > > qry, > > > > > > Transformer<E, R> transfomer); *because I don't see how these > > > > > transformers > > > > > > will work for example with SQL, but this API makes you think that > > > > > > transformers are supported for all the query types. > > > > > > > > > > > > Sergi > > > > > > > > > > > > 2016-02-04 16:46 GMT+03:00 Andrey Gura <[email protected]>: > > > > > > > > > > > > > Val, > > > > > > > > > > > > > > can we introduce new method into IgnteCache API? > > > > > > > > > > > > > > Now we have method: public <R> QueryCursor<R> query(Query<R> > > qry); > > > > > > > > > > > > > > New method will be something like this: <R> QueryCursor<R> > > > > > query(Query<E> > > > > > > > qry, Transformer<E, R> transfomer); > > > > > > > > > > > > > > It allows provide transformers for all query types and chnages > > will > > > > be > > > > > > > related only with query cursor functionality. > > > > > > > > > > > > > > Will it work? > > > > > > > > > > > > > > On Thu, Feb 4, 2016 at 11:13 AM, Andrey Kornev < > > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Another perhaps bigger problem with running queries > (including > > > scan > > > > > > > > queries) using closures was discussed at length on the @dev > not > > > so > > > > > long > > > > > > > > ago. It has to do with partitions migration due to cluster > > > topology > > > > > > > changes > > > > > > > > which may result in the query returning incomplete result. > And > > > > while > > > > > it > > > > > > > is > > > > > > > > possible to solve this problem for the scan queries by using > > some > > > > > > clever > > > > > > > > tricks, all bets are off with the SQL queries.Andrey > > > > > > > > _____________________________ > > > > > > > > From: Valentin Kulichenko <[email protected]> > > > > > > > > Sent: Thursday, February 4, 2016 6:29 AM > > > > > > > > Subject: Re: Transformers in SCAN queries > > > > > > > > To: <[email protected]> > > > > > > > > > > > > > > > > > > > > > > > > Dmitry, > > > > > > > > > > > > > > > > The main difference in my view is that you lose pagination > > when > > > > > > sending > > > > > > > > results from servers to client. What if one wants to iterate > > > > through > > > > > > all > > > > > > > > entries in cache? > > > > > > > > > > > > > > > > On Wed, Feb 3, 2016 at 9:47 PM, Dmitriy Setrakyan < > > > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Valentin, > > > > > > > > > > > > > > > > > > Wouldn’t the same effect be achieved by broadcasting a > > closure > > > > to > > > > > > the > > > > > > > > > cluster and executing scan-query on every node locally? > > > > > > > > > > > > > > > > > > D. > > > > > > > > > > > > > > > > > > On Wed, Feb 3, 2016 at 9:17 PM, Valentin Kulichenko < > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > I keep getting requests from our users to add optional > > > > > > transformers > > > > > > > to > > > > > > > > > SCAN > > > > > > > > > > queries. This will allow to iterate through cache, but > do > > > not > > > > > > > transfer > > > > > > > > > > whole key-value pairs across networks (e.g., get only > > keys). > > > > The > > > > > > > > feature > > > > > > > > > > looks useful and I created a ticket [1]. > > > > > > > > > > > > > > > > > > > > I am struggling with the design now. The problem is > that I > > > > > wanted > > > > > > to > > > > > > > > > extend > > > > > > > > > > existing ScanQuery object for this, but this seems to be > > > > > > impossible > > > > > > > > > because > > > > > > > > > > it already extends Query<Cache.Entry<K, V>> and thus can > > > > iterate > > > > > > > only > > > > > > > > > > through entries. > > > > > > > > > > > > > > > > > > > > The only option I see now is to create a separate query > > > type, > > > > > > > > copy-paste > > > > > > > > > > everything from ScanQuery and add *mandatory* > transformer. > > > > > > Something > > > > > > > > like > > > > > > > > > > this: > > > > > > > > > > > > > > > > > > > > ScanTransformQuery<K, V, R> extends Query<R> { > > > > > > > > > > IgniteBiPredicate<K, V> filter; > > > > > > > > > > IgniteClosure<Cache.Entry<K, V>, R> transformer; > > > > > > > > > > int part; > > > > > > > > > > ... > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > Thoughts? Does anyone has other ideas? > > > > > > > > > > > > > > > > > > > > [1] > https://issues.apache.org/jira/browse/IGNITE-2546 > > > > > > > > > > > > > > > > > > > > -Val > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Andrey Gura > > > > > > > GridGain Systems, Inc. > > > > > > > www.gridgain.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Andrey Gura > > > > > GridGain Systems, Inc. > > > > > www.gridgain.com > > > > > > > > > > > > > > > > > > > > > -- > > > Andrey Gura > > > GridGain Systems, Inc. > > > www.gridgain.com > > > > > > > > > -- > Andrey Gura > GridGain Systems, Inc. > www.gridgain.com >
