Re: ContinuousQueryWithTransformer implementation questions - 2

Hello, guys. Anton, Yakov, can you, please, share your wisdom, and make final review of IGNITE-425 Task: https://issues.apache.org/jira/browse/IGNITE-425 PR: https://github.com/apache/ignite/pull/2372 2017-09-18 18:33 GMT+03:00 Николай Ижиков : > So, resuming: > > 1) My

Re: ContinuousQueryWithTransformer implementation questions - 2

So, resuming: 1) My solution reduces network communication. As far as I know, a lot of people want to have this feature at Ignite 2.x. It's impossible to gain perfect API right now, it will take months to gain it. My solution ready right now!, let's merge it and refactor whole Continuous Query

Re: ContinuousQueryWithTransformer implementation questions - 2

Vladimir, Here is a short summary what is wrong with continuous query OK. I agree - this is problems of current API. How we can fix it by not merging ContinuousQueryWIthTransformer? How we can quickly design, discuss and implement new API? Because at the moment there is no any ticket to

Re: ContinuousQueryWithTransformer implementation questions - 2

Nikolay, Here is a short summary what is wrong with continuous query: 1) It should not have "initialQuery" 2) It should not be called through IgniteCache.query() method, as it has nothing in common with other "query" types 3) Main thing: our listeners are *ALWAYS* executed on a node which

Re: ContinuousQueryWithTransformer implementation questions - 2

Igniters, I discussed API of ContinuousQuery and ContinuousQueryWithTransformer with Anton Vinogradov one more time. Since users who use regular ContinuousQuery already knows pros. and cons of using initialQuery and to not to complicate API more and more until 3.0 we agreed that best choice is

Re: ContinuousQueryWithTransformer implementation questions - 2

Vladimir, If the API is so bad then it might take much more time to make up and roll out the new. Plus, there should be a community member who is ready to take it over. My suggestion would be to accept this contribution and initiate an activity towards the new API if you like. Personally, I

Re: ContinuousQueryWithTransformer implementation questions - 2

Hello, Vova. Can you, please, share your knowledge: Why we have abandon ContinuousQuery? Guys. I'm new in the project so can someone more experienced tell me: What's is wrong with the current implementation of ContinuousQuery? 1. initialQuery is useless - OK, understood. What else is

Re: ContinuousQueryWithTransformer implementation questions - 2

Guys. I'm new in the project so can someone more experienced tell me: What's is wrong with the current implementation of ContinuousQuery? 1. initialQuery is useless - OK, understood. What else is wrong? 2017-09-12 20:02 GMT+03:00 Николай Ижиков : > Vova, > > > Public

Re: ContinuousQueryWithTransformer implementation questions - 2

I meant "It is much better *than* extend*ing* confusing behavior *of *already confusing and overengineered API." On Tue, Sep 12, 2017 at 7:35 PM, Vladimir Ozerov wrote: > Nikolay, > > Public API is the face of our product. We cannot and do not want to change > it in a

Re: ContinuousQueryWithTransformer implementation questions - 2

Nikolay, Public API is the face of our product. We cannot and do not want to change it in a rush. This ticket was in a backlog for more than 2 years. It is not a big problem if we spend additional week or month for API design. It is much better to extend confusing behavior on already confusing

Re: ContinuousQueryWithTransformer implementation questions - 2

Dima, We definitely can have factories if we want. On Tue, Sep 12, 2017 at 5:55 PM, Dmitriy Setrakyan wrote: > Vladimir, are their factories for the proposed listeners? > > On Tue, Sep 12, 2017 at 7:52 AM, Alexey Goncharuk < > alexey.goncha...@gmail.com> wrote: > > >

Re: ContinuousQueryWithTransformer implementation questions - 2

Alex, The source of our confusion in current API is that we called "filter" what actually should be "listener". I propose to set has the simplest API possible. Note that CacheEntryListener is a part of jcache API. Call of this method will set passed listener on remote nodes. Nothing more. UUID

Re: ContinuousQueryWithTransformer implementation questions - 2

Vova > I propose to deprecate current continuous queries and develop new API. > This should not break anything. If the community agrees that *whole* continuous query API is bad - it OK. Let's develop new API. But developing new public API and implementing it is a very long process. One can see

Re: ContinuousQueryWithTransformer implementation questions - 2

Vladimir, are their factories for the proposed listeners? On Tue, Sep 12, 2017 at 7:52 AM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > Vladimir, > > Can you please clarify how the proposed API will work? > > My opinion is that our query API is big piece of ... you know, especially >

Re: ContinuousQueryWithTransformer implementation questions - 2

Vladimir, Can you please clarify how the proposed API will work? My opinion is that our query API is big piece of ... you know, especially > ContinuousQuery. A lot of concepts and features are mixed in a single > entity, what makes it hard to understand and use. Let's finally deprecate >

Re: ContinuousQueryWithTransformer implementation questions - 2

Yakov, I propose to deprecate current continuous queries and develop new API. This should not break anything. Adding transformers on top of current API will make it total mess. On Tue, Sep 12, 2017 at 5:39 PM, Николай Ижиков wrote: > Yakov. > > > 2. In a new class we

Re: ContinuousQueryWithTransformer implementation questions - 2

Yakov. > 2. In a new class we still have initial query which uses types which is questionable. After some discussion with a colleague, I think I got your point. If I understand your question right: Why does someone want to execute an initial query that returns ? If one use

Re: ContinuousQueryWithTransformer implementation questions - 2

Agree with Vladimir. However, this will break API compatibility. So, at this point this is impossible. --Yakov

Re: ContinuousQueryWithTransformer implementation questions - 2

Hello, Vladimir. class ContinuousQueryCacheEntryListener implements CacheEntryListener { ContinuousQueryRemoteFilter rmtFilter; ContinuousQueryRemoteTransformer rmtTransformer; ContinuousQueryLocalCallback locCb; } I think class you proposed should be separated in two because of

Re: ContinuousQueryWithTransformer implementation questions - 2

My opinion is that our query API is big piece of ... you know, especially ContinuousQuery. A lot of concepts and features are mixed in a single entity, what makes it hard to understand and use. Let's finally deprecate ContinuousQuery and design nice and consistent API. E.g.: interface IgniteCache

Re: ContinuousQueryWithTransformer implementation questions - 2

Dmitry, can you please take a look at public API change. Ticket - https://issues.apache.org/jira/browse/IGNITE-425 PR - https://github.com/apache/ignite/pull/2372 Issues: 1. Do you see any other option other than creating separate class? As for me I don't. 2. In a new class we still have initial

Re: ContinuousQueryWithTransformer implementation questions - 2

Hello, Nikolay. I think we can't use this method: We need to set: * transformer * *local listener* - differs from regular ContinuousQuery listener * other continuous query params. So at least we have to add local listener param. If we use ContinuousQuery instance as first param to get all

Re: ContinuousQueryWithTransformer implementation questions - 2

Yakov, Anton, thank you! 2017-09-04 15:16 GMT+03:00 Yakov Zhdanov : > Well, let's leave it as is for now, since I don't see any easy way to > consistently solve this. > > --Yakov > -- Nikolay Izhikov nizhikov@gmail.com

Re: ContinuousQueryWithTransformer implementation questions - 2

Well, let's leave it as is for now, since I don't see any easy way to consistently solve this. --Yakov

Re: ContinuousQueryWithTransformer implementation questions - 2

+1 to keep current behavior and start new thread to solve notification issue. On Mon, Sep 4, 2017 at 2:05 PM, Николай Ижиков wrote: > Hello, Yakov. > > I made a bit of investigation about your proposal of handling filter and > transformer exceptions: > > 1. If we cancel

Re: ContinuousQueryWithTransformer implementation questions - 2

Hello, Yakov. I made a bit of investigation about your proposal of handling filter and transformer exceptions: 1. If we cancel continuous query from remote node user can't know it. There is no public API to check "Is continuous query still alive?". The only consequence of canceling query

Re: ContinuousQueryWithTransformer implementation questions - 2

I would postpone review until we come to a clear decision on what should be done if filter or transformer fails. I don't think cancelling query is too much. From my standpoint this is a kind of heuristic exception and may break some sensitive logic. Thanks! -- Yakov Zhdanov, Director R *GridGain

Re: ContinuousQueryWithTransformer implementation questions - 2

Nikolay, You can also use: 1) Global Exception Registry ctx.kernalContext().exceptionRegistry().onException(shortMsg, ex); 2) Statistic final boolean statsEnabled = cctx.config().isStatisticsEnabled(); if (statsEnabled) metrics.incrementOnFails(); P.s. Please make sure that code still

Re: ContinuousQueryWithTransformer implementation questions - 2

Hello, Yakov. The new class is OK - got it. Thanks! > Should we extract a super class? Yes, we should. I already have done it. See my last commit in PR - https://github.com/apache/ignite/pull/2372/commits/af1ed2e4dbef4ba5999f8566198cb75ad922f93b > We can put hard requirement that filter

Re: ContinuousQueryWithTransformer implementation questions - 2

I think I have already agreed on a separate class since it seems to be the only option due to generics issue. Should we extract a super class? We can put hard requirement that filter and transformer cannot throw exception (same as cache interceptor). If exception is thrown then we cancel the

Re: ContinuousQueryWithTransformer implementation questions - 2

Semyon, Yakov, Here is ticket - https://issues.apache.org/jira/browse/IGNITE-6221 Thank you. Can we now return to my questions about IGNITE-425? 1. I introduce new query class - ContinuousQueryWithTransformer [3]. Reasons: * ContinuousQuery is final so user can't extends it. I don't want to

Re: ContinuousQueryWithTransformer implementation questions - 2

Hi, I had an impression that current behavior is required by jcache, but now I can not find anything about this neither in spec nor in jcache tck tests. So I think we can change current behavior. Thanks On Tue, Aug 29, 2017 at 9:48 PM, Nikolay Izhikov wrote: > Yakov. >

Re: ContinuousQueryWithTransformer implementation questions - 2

Igniters, Does anyone else see potential issues on user side with current approach? Sam, is this JCache requirement? --Yakov 2017-08-29 15:16 GMT+03:00 Nikolay Izhikov : > Yakov. > > I think exception equals `true` is intended behavior. > > Filter evaluation

Re: ContinuousQueryWithTransformer implementation questions - 2

Yakov. I think exception equals `true` is intended behavior. Filter evaluation implementation from master - [1] Test from master to check filter exception(without explicit asserts checking listeners call) - [2] Here is my quick test with asserts on listener call after filter exception: ```

Re: ContinuousQueryWithTransformer implementation questions - 2

> If filter throws exception entry would be passed to listener. this is strange. Imagine a filter that very rarely throws some runtime exception due to external or environmental reasons, but in case of normal execution filter evaluates to false. In case of error entry is passed to a local

Re: ContinuousQueryWithTransformer implementation questions - 2

Hi, Yakov. If filter throws exception entry would be passed to listener. > Nikolay, I would also suggest you extract some super class for continuous > query. It will help to avoid code duplicates. Yes, I will do this after reaching consensus on API changes. 29.08.2017 14:04, Yakov Zhdanov

Re: ContinuousQueryWithTransformer implementation questions - 2

I don't like the idea of having separate class, but it seems to be the only way as there are too many API and generics differences. Nikolay, I would also suggest you extract some super class for continuous query. It will help to avoid code duplicates. As far as remote transformer failure - we

Re: ContinuousQueryWithTransformer implementation questions - 2

Nikolay, > 2. What behavior is expected if transformer throws exception for some event? I see following options: Client should be notified, I vote for > Introduce special callback. onTransformError? On Tue, Aug 29, 2017 at 1:36 PM, Nikolay Izhikov wrote: > Hello,

ContinuousQueryWithTransformer implementation questions - 2

cc0b22b0e3e [4] http://apache-ignite-developers.2346864.n4.nabble.com/ContinuousQueryWithTransformer-implementation-questions-td20078.html

Re: ContinuousQueryWithTransformer implementation questions

Yeah, unfortunately current ContinuousQuery object can be used for querying with transformer. That's actually not good, because adding transformers to continuous queries and scan queries will be very inconsistent. AFAIK, there are plans to completely rework query API since we added a lot of stuff

Re: ContinuousQueryWithTransformer implementation questions

Nikolay, We already have the following method for queries with transformer. It currently throws exception for ContinuousQuery. QueryCursor query(Query qry, IgniteClosure transformer) Would it be possible to utilize it instead of creating new API? -Val On Wed, Jul 26, 2017 at 5:26

ContinuousQueryWithTransformer implementation questions

Hello, Igniters. I'm working on IGNITE-425 [1] issue. I made a couple of changes in my branch [2] so I want to confirm that changes with community before moving forward: Text of issue: ``` Currently if updated entry passes the filter, it is sent to node initiated the query entirely. It would be