AggregateUnionTransposeRule fails when some inputs have unique grouping key

2021-05-19 Thread Vladimir Ozerov
Hi, The AggregateUnionTransposeRule attempts to push the Aggregate below the Union. Before: Aggregate[group=$0, agg=SUM($1] Union[all] Input1 Input2 After: Aggregate[group=$0, agg=SUM($1] Union[all] Aggregate[group=$0, agg=SUM($1] Input1 Aggregate[group=$0, agg=SUM($1]

Re: AggregateUnionTransposeRule fails when some inputs have unique grouping key

2021-05-19 Thread Vladimir Ozerov
Hi Pavel, Yes, I missed the list, sorry. ср, 19 мая 2021 г. в 14:40, Pavel Tupitsyn : > Hi Vladimir, > > Looks like this message is for d...@calcite.apache.org, not > dev@ignite.apache.org, > or am I mistaken? > > On Wed, May 19, 2021 at 2:25 PM Vladimir Ozerov > wrot

Re: [DISCUSS] PMC Chair rotation time

2019-10-24 Thread Vladimir Ozerov
global communications. > > - Roman Shtykh. > Rocketmq and Ignite committer. > Highly involved to Ignite development and popularization. > > - Vladimir Ozerov > Ignite and Hazelcast committer. > Man with really strong leadership skills. > > - Yakov Zhdanov > Igni

Re: Adding support for Ignite secondary indexes to Apache Calcite planner

2019-12-10 Thread Vladimir Ozerov
Hi Roman, Why do you think that Drill-style will not let you exploit collation? Collation should be propagated from the index scan in the same way as in other sorted operators, such as merge join or streaming aggregate. Provided that you use converter-hack (or any alternative solution to trigger p

Re: Adding support for Ignite secondary indexes to Apache Calcite planner

2019-12-11 Thread Vladimir Ozerov
lying DbScanSortRemovalRule we have: > > Project > IndexScan > > while for Phoenix approach we would have two equivalent subsets in our > planner: > > Project > Sort > TableScan > > and > > Project > IndexScan > > and most likely the last plan

Re: Adding support for Ignite secondary indexes to Apache Calcite planner

2019-12-11 Thread Vladimir Ozerov
materialized_views.html > [2] https://calcite.apache.org/docs/lattice.html > > -- > Kind Regards > Roman Kondakov > > > On 11.12.2019 17:11, Vladimir Ozerov wrote: > > Roman, > > > > What is the advantage of Phoenix approach then? BTW, it looks like > Pho

JSR 381 (Visual Recognition in Java) and Apache Ignite

2020-03-30 Thread Vladimir Ozerov
Hi Alexey, Igniters, Let me introduce you Heather, Zoran, and Frank. Heather is the Chair of the JCP. Zoran and Frank are JSR 381 spec leads. They are interested in discussing the upcoming visual recognition specification with the Apache Ignite community, to understand whether the community has an

Re: [ANNOUNCE] New Committer: Taras Ledkov

2020-05-12 Thread Vladimir Ozerov
Congratulations, Taras! Well deserved! вт, 12 мая 2020 г. в 20:27, Ivan Rakov : > Taras, > > Congratulations and welcome! > > On Tue, May 12, 2020 at 8:26 PM Denis Magda wrote: > > > Taras, > > > > Welcome, that was long overdue on our part! Hope to see you soon among > the > > PMC group. > > >

Re: Continuous queries and duplicates

2019-01-09 Thread Vladimir Ozerov
is > > > > possible to see V2 in initial query and in event. With MVCC it is now > > > > technically possible to query data on certain snapshot and then > receive > > > > only events happened after this snapshot. So that we never see V2 > > twice. >

Re: Time to remove automated messages from the devlist?

2019-01-10 Thread Vladimir Ozerov
s keeping on dev list): > > >> > >> > TC bot + > > >> > >> > Jira - > > >> > >> > GitHub - > > >> > >> > пт, 16 нояб. 2018 г. в 22:25, Dmitriy Pavlov < > dpav...@apache.org > > >: > > >&

Re: Asynchronous index rebuild

2019-01-10 Thread Vladimir Ozerov
simple, the implementation looks sort-of simple as well > (although it adds another bit of complexity to PME). > > Why do you prefer to go with the join-leave-rejoin approach? > > Thanks, > Stan > > From: Vladimir Ozerov > Sent: 30 ноября 2018 г. 13:09 > To: dev >

Re: Best Effort Affinity for thin clients

2019-01-15 Thread Vladimir Ozerov
et me know what you think. > > [1] - > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients > > Best Regards, > Igor > > > On Tue, Jun 19, 2018 at 11:54 AM Vladimir Ozerov > wrote: > > > Denis, > > >

Re: proposed realization KILL QUERY command

2019-01-15 Thread Vladimir Ozerov
L QUERY command. But it hard to use. > > 5) Any other approach you can think of > > If be honestly I prefer first variant, it looks simple to use by user (it > require read a docs, but any case it required for any use cases). > > Lets discuss it again and chose better approach t

Re: proposed realization KILL QUERY command

2019-01-15 Thread Vladimir Ozerov
a > symbols for such queries to avoid fault of user and kill all queries by > mistake, like KILL QUERY 7B* > > Complex: > > KILL QUERY WHERE queryId=7BX3C86 - for killing concrete query. > > KILL QUERY WHERE nodeId=37d7afd8-b87d-4aa1-b3d1-c1c03380 - for kill > all run

Re: CompactFooter for ClientBinaryMarshaller

2019-01-18 Thread Vladimir Ozerov
"Compact footer" is optimization which saves a lot of space. Object serialized in this form do not have the full information required for deserialization. Metadata necessary for deserialization (aka "schema") is located on cluster nodes. For this client it could be requested through special command

Re: CompactFooter for ClientBinaryMarshaller

2019-01-23 Thread Vladimir Ozerov
try to retrieve and getting null. > > Therefore, I suppose switching client to server defaults is what has to > be > > done. If the user decides to switch to full schema mode, at least he/she > > will be aware of it. And for deserialization, the schema will be > retri

Re: Continuous queries and duplicates

2019-01-23 Thread Vladimir Ozerov
ven a partition. So I think it wouldn’t require any > cross-node ordering. > > Thank you, > > Piotr > > śr., 9 sty 2019, 21:11: Vladimir Ozerov napisał(a): > > > Hi, > > > > MVCC caches have the same ordering guarantees as non-MVCC caches, i.e. > two > &

Re: SQL View with list of existing indexes

2019-01-24 Thread Vladimir Ozerov
Hi Yuriy, Please note that MySQL link is about SHOW command, which is a different beast. In general I think that PG approach is better as it allows user to get quick overview of index content without complex JOINs. I would start with plain single view and add columns view later if we found it usef

Re: Baseline auto-adjust`s discuss

2019-01-24 Thread Vladimir Ozerov
Hi Anton, This is great feature, but I am a bit confused about automatic disabling of a feature during manual baseline adjustment. This may lead to unpleasant situations when a user enabled auto-adjustment, then re-adjusted it manually somehow (e.g. from some previously created script) so that aut

Re: Baseline auto-adjust`s discuss

2019-01-25 Thread Vladimir Ozerov
n (ex. -1 equal to endless waiting) and so on, but > in our case we want to disable whole functionality rather than change > parameter value. > > -- > Best regards, > Anton Kalashnikov > > > 24.01.2019, 22:03, "Vladimir Ozerov" : > > Hi Anton, > > &

Re: Distributed MetaStorage discussion

2019-01-25 Thread Vladimir Ozerov
Ivan, The change you describe is extremely valuable thing as it allows to detect changes into global configuration which is of great importance for SQL. Will topology and affinity changes be reflected in metastore history as well? From SQL perspective it is important for us to be able to understan

Re: Distributed MetaStorage discussion

2019-01-25 Thread Vladimir Ozerov
gy and affinity versions, but motivation here > is not that clear for me to be honest. > > I think that the most common approach with single incrementing version > is much simpler then several counters and I would prefer to leave it that > way. > > > пт, 25 янв. 2019 г. в 16

Re: SQL View with list of existing indexes

2019-01-28 Thread Vladimir Ozerov
h column order from the columns view. For > example you want to know which columns are indexed already. In case we will > have plain comma-separated form it can't be achieved. > > > > > > чт, 24 янв. 2019 г. в 18:09, Vladimir Ozerov : > > > Hi Yuriy, > > >

Re: H2 license and vulnerabilities

2019-01-28 Thread Vladimir Ozerov
Hi Steve, H2 cannot be removed from Ignite easily as it is integrated pretty deep into indexing module. Good news is that our usage of H2 is pretty limited - we only use it's parser, planner and execution pipeline. We do not use H2 as data storage. Please let me know if you need any additional cla

Re: proposed realization KILL QUERY command

2019-01-30 Thread Vladimir Ozerov
ounds like the node is experiencing big troubles and it might be > > better just to shut it down completely. > > > > - > > Denis > > > > > > On Tue, Jan 15, 2019 at 8:00 AM Юрий > wrote: > > > >> Denis and other Ignit

Re: Best Effort Affinity for thin clients

2019-02-04 Thread Vladimir Ozerov
add flag to every response, that shows > whether > > >> the > > >> > > Affinity Topology Version of the cluster has changed since the > last > > >> > request > > >> > > from the client. > > >> > > I propose to keep this

Re: Best Effort Affinity for thin clients

2019-02-04 Thread Vladimir Ozerov
is a public thing [1]. > > [1] - https://apacheignite.readme.io/docs/cache-groups > > Best Regards, > Igor > > > On Mon, Feb 4, 2019 at 11:47 AM Vladimir Ozerov > wrote: > > > Pavel, Igor, > > > > This is not very accurate to say that this will not save

Re: Best Effort Affinity for thin clients

2019-02-04 Thread Vladimir Ozerov
Best Regards, > Igor > > > On Mon, Feb 4, 2019 at 1:13 PM Vladimir Ozerov > wrote: > > > Igor, > > > > Yes, cache groups are public API. However, we try to avoid new APIs > > depending on them. > > The main point from my side is that “similar cache

Re: Binary clients: fallback to the previous versions of the protocol

2019-02-13 Thread Vladimir Ozerov
Hi Dmitriy, It is very common practice to keep client protocol compatible with multiple versions of the server. We constantly face this in practice. I do not see any reason to drop or complicate this functional: user just connects to the server and we automatically negotiate on the best feature se

Re: SQL: INSERT with hidden columns _key, _val and check the type of input objects

2019-02-22 Thread Vladimir Ozerov
Denis, SQL is a language with strict schema what was one of significant factors of it's worldwide success. I doubt we will ever have SQL without configuration/definiton, because otherwise it will be not SQL, but something else (e.g. document-oriented, JSON, whatever). On Fri, Feb 22, 2019 at 1:52

Re: SQL: INSERT with hidden columns _key, _val and check the type of input objects

2019-02-23 Thread Vladimir Ozerov
ties. > > Basically, my idea is that all of the objects and their fields stored in > the caches should be visible to SQL w/o extra settings. If someone wants to > create indexes then use DDL which was designed for this. > > > - > Denis > > > On Fri, Feb 22,

Re: SQL: INSERT with hidden columns _key, _val and check the type of input objects

2019-02-27 Thread Vladimir Ozerov
t; Ok, agreed, let's not boil the ocean...at least for now ;) > > -- > Denis Magda > > > On Sat, Feb 23, 2019 at 12:50 AM Vladimir Ozerov > wrote: > > > Denis, > > > > Yes, this is what my answer was about - you cannot have SQL without > > defi

Re: SQL: INSERT with hidden columns _key, _val and check the type of input objects

2019-02-27 Thread Vladimir Ozerov
nds on that and > it isn't fixable easily. > > For example, they can have objects of which only subset of fields is > indexed, the rest is not. Then they are inserting them via SQL as shown. > > Regards, > -- > Ilya Kasnacheev > > > ср, 27 февр. 2019 г. в 12:10

Re: Please re-commit 3 last changes in the master

2019-03-04 Thread Vladimir Ozerov
Looks like everything is good now - all three commits were returned. On Mon, Mar 4, 2019 at 2:04 PM Dmitriy Pavlov wrote: > Thanks, Ivan, these commits are in sync in GitHub & GitBox. Only one commit > remained, Vladimir O., please chime in > > пн, 4 мар. 2019 г. в 14:03, Ivan Rakov : > > > Than

Re: Storing short/empty strings in Ignite

2019-03-05 Thread Vladimir Ozerov
Hi Val, I would say that we do not need string length at all, because it can be derived from object footer (next field offset MINUS current field offset). It is not very good idea to implement proposed change in Apache Ignite 2.x because it is breaking and will add unnecessary complexity to alread

Re: Batch updates in Ignite B+ tree.

2019-03-05 Thread Vladimir Ozerov
Hi Pavel, As far as I know batch tree updates already being developed. Alex, could you please elaborate? On Tue, Mar 5, 2019 at 5:05 PM Pavel Pereslegin wrote: > Hi Igniters! > > I am working on implementing batch updates in PageMemory [1] to > improve the performance of preloader, datastreamer

Re: Ignite 2.8 Release: Time & Scope & Release manager

2019-03-07 Thread Vladimir Ozerov
Igniters, Making release from master is not an option. We have a lot of not-yet-ready and not-yet-tested features. From SQL side this is partition pruning and SQL views with KILL command. So if we do not want to release a mess, then there are only two options: release Java 11 fixes on top of 2.7,

Re: Ignite 2.8 Release: Time & Scope & Release manager

2019-03-07 Thread Vladimir Ozerov
ge, it says the master is release ready branch. > > Otherwise feature is developed in its own branch. > > So my vote goes for master-based release. > > чт, 7 мар. 2019 г. в 20:28, Vladimir Ozerov : > > > Igniters, > > > > Making release from master is not an opt

Re: Peer review: Victory over Patch Available debt

2019-03-18 Thread Vladimir Ozerov
Hi, This is tough question, and first of all I'd like to ask participants to keep cold head. This is a public question and can be discussed on the dev list safely. On the one hand, it is true that a number of patches are not reviewed for a long time, what negatively affects community development.

Re: Thin client: transactions support

2019-03-27 Thread Vladimir Ozerov
> > > > > operation we have the next options: > > > > > > * Serialize full set of arguments and use some value for missing > > > > > > arguments. For example -1 for int/long types and null for string > > > type. > > > > We > &g

Re: Thin client: transactions support

2019-03-27 Thread Vladimir Ozerov
As far as SUSPEND/RESUME/SAVEPOINT - we do not support them yet, and adding them in future should not conflict with simple START/END infrastructure. On Wed, Mar 27, 2019 at 11:00 AM Vladimir Ozerov wrote: > Hi Alex, > > I am not sure we need 5 commands. Wouldn't it be enough to

Re: Thin client: transactions support

2019-03-27 Thread Vladimir Ozerov
end the old transaction > implicitly (rollback) or throw an exception to the client? In my opinion, > the first option is better. For example, if we got a previously used > connection from the connection pool, we should not worry about any > uncompleted transaction started by the previ

Re: Thin client: transactions support

2019-04-04 Thread Vladimir Ozerov
> >> > > > > can start with thread-per-connection approach (we only can > support > >> > one > >> > > > > active transaction per connection, see below, so we need one > >> > additional > >> > > > > dedicated th

Re: Allow distributed SQL query execution over explicit set of partitions

2017-01-18 Thread Vladimir Ozerov
Several side notes about API. 1) I would avoid ranges even in this form.for the sake of simplicity. Ignite do not have any notion of "partition range" in affinity API, so I do not understand how users are going to work on ranges unless they have some very special custom affinity function, which is

Re: Make async API great again

2017-01-20 Thread Vladimir Ozerov
Very cool! Clean and straightforward. P.S.: If someone doesn't understand the fix - please look at new methods with "async" suffix [1]. [1] https://github.com/gridgain/apache-ignite/blob/b81621bf2e8a 35b20989f95ff52c0f6d91dd75d6/modules/core/src/main/java/ org/apache/ignite/IgniteCompute.java On

Re: Make async API great again

2017-01-20 Thread Vladimir Ozerov
IgniteCompute was designed this way initially. See IgniteCompute.future() override. May be it makes sense to leave this type only for execute(...) methods. On Fri, Jan 20, 2017 at 5:45 PM, Semyon Boikov wrote: > Hi Taras, > > Why 'async' methods return ComputeTaskFuture, not just IgniteFuture? I

Re: Abandon of the non-throwing version of C++ client API

2017-01-23 Thread Vladimir Ozerov
+1 to Igor's idea. Ignite is relatively high-level product and we do not expect ultra-optimized users who cannot allow exceptions to be enabled. Macros should be a good workaround for them, though. On Sat, Jan 21, 2017 at 6:47 PM, Denis Magda wrote: > Hi Igor, > > My C++ experience is based only

Re: Abandon of the non-throwing version of C++ client API

2017-01-23 Thread Vladimir Ozerov
Why have you > changed you mind proposing to make the throwing version default one? Any > new tendency in C++ community? > > — > Denis > > On Jan 23, 2017, at 2:56 AM, Vladimir Ozerov wrote: > > +1 to Igor's idea. Ignite is relatively high-level product and we do no

Read this if you want to integrate Ignite with other platforms (Python, R, etc.)

2017-01-25 Thread Vladimir Ozerov
Igniters, I see growing community interest to implementation of so-called "clients" to other platforms, such as R, Python, etc.. Let me briefly describe architecture of our existing integrations - C++ and .NET. *1) Not a client!* We do not have clients in classical sense. Our C++ and .NET integra

Should we take care of Java 9 in Ignite 2.0 scope?

2017-01-25 Thread Vladimir Ozerov
Igniters, Please see this article [1] from Kotlin guys. They had to re-pack public API because Java 9 doesn't allow several modules to share the same public package. Looks like this limitation could impact us at some point, so that we will not be able to support Java 9 without breaking API changes

Re: Read this if you want to integrate Ignite with other platforms (Python, R, etc.)

2017-01-25 Thread Vladimir Ozerov
Shashank, Dmitriy, In principle it should be possible to use 3rd party marshalling frameworks, but with some limitations. Java side expects objects in certain format. At the very high level it looks like [HEADER][BODY][FOOTER]. This is enough for Java to process the object. While you *MUST* marsh

Re: DDL implementation details

2017-01-27 Thread Vladimir Ozerov
16, 2017 at 11:06 PM, Sergi Vladykin < > >> sergi.vlady...@gmail.com > >>>> > >>> wrote: > >>> > >>>> Dima, > >>>> > >>>> I agree that cache==table is definitely a wrong choice, but as far

Binary object hash code resolution in Apache Ignite 2.0

2017-01-27 Thread Vladimir Ozerov
Folks, Historically we have fundamental flaw in how we deal with hashCode/equals for BinaryObjects: 1) hashCode() is taken from original deserialized object. 2) But equals() is performed through binary array comparison. We recently introduced BinaryIdentityResolver [1] as a part of DML feature, b

Do not use H2 parser for DDL

2017-01-27 Thread Vladimir Ozerov
Folks, H2 has nice and powerful SQL parser, and we pass all incoming SQL commands through it. However it doesn't support custom SQL syntax, which we are likely to need in DDL at some point in future. As DDL statements are relatively easy to parse comparing to SELECTs and they do not involve optim

Re: Binary object hash code resolution in Apache Ignite 2.0

2017-01-27 Thread Vladimir Ozerov
quals check > is done by comparing byte arrays? Wouldn't it be faster? > > On Fri, Jan 27, 2017 at 4:38 AM, Pavel Tupitsyn > wrote: > > > Makes sense to me. > > > > Ticket for reference: https://issues.apache.org/jira/browse/IGNITE-4558 > > > > On

Re: Binary object hash code resolution in Apache Ignite 2.0

2017-01-27 Thread Vladimir Ozerov
; Denis > > > On Jan 27, 2017, at 2:37 AM, Vladimir Ozerov > wrote: > > > > Folks, > > > > Historically we have fundamental flaw in how we deal with hashCode/equals > > for BinaryObjects: > > 1) hashCode() is taken from original deserialized obje

Re: Do not use H2 parser for DDL

2017-01-27 Thread Vladimir Ozerov
My point was that we can avoid dependency on 3rd party developers for this relatively simple logic. On Fri, Jan 27, 2017 at 8:07 PM, Dmitriy Setrakyan wrote: > On Fri, Jan 27, 2017 at 5:45 AM, Sergi Vladykin > wrote: > > > H2 to some extent supports syntax (and quirks) from other databases. For

Re: How to improve SQL testing

2017-01-31 Thread Vladimir Ozerov
Evgeniy, AFAIK YCSB is used for NoSQL database benchmark, right? This is not exactly our case, because we are going to test pure SQL. On Tue, Jan 31, 2017 at 10:55 AM, Evgeniy Stanilovskiy < stanilov...@gmail.com> wrote: > Sergey, did you take a look into https://en.wikipedia.org/wiki/YCSB ? > h

Re: Unexpected behavior of DiscoveryCustomMessage acks

2017-02-02 Thread Vladimir Ozerov
Sounds very nasty. On Thu, Feb 2, 2017 at 3:50 PM, Sergey Chugunov wrote: > Hello folks, > > Working on IGNITE-4302 > I developed a protocol for delivering metadata updates to all nodes in > cluster. > > This protocol relies on a guarantee of *

Re: Ensure that builder approach is used for all setters in public API

2017-02-03 Thread Vladimir Ozerov
Andrey, This is very important change from usability standpoint. But my main concern is changes to SPI *interfaces*. If we do so users who implemented custom SPIs will have broken compatibility. On the other hand, I doubt there will be too much affected users, and we break compilation in AI 2.0 an

Re: IGNITE-3244

2017-02-03 Thread Vladimir Ozerov
This is very important thing because we need same thing in .NET/CPP platforms. I think we should extend the protocol and add type IDs to container's header: Provided that Java do not have normal generics, it will be a no-op for Collection and Map. But it will be used extensively in .NET and CPP. D

Re: Do not use H2 parser for DDL

2017-02-03 Thread Vladimir Ozerov
t; > > >> > > > Alex, > > > > > > My preference would be to keep it simple, without introducing any > custom > > > grammar, at least for the "CREATE INDEX" command. I understand the need > > for > > > decoupling cache from

Re: IGNITE-3244

2017-02-06 Thread Vladimir Ozerov
Denis, We need to ensure that type IDs are attached for both arrays and collections/maps. May be we have it for arrays already, but I doubt it is so for generic containers. On Fri, Feb 3, 2017 at 8:59 PM, Denis Magda wrote: > If a value can be serialized to BinaryObject then it will be serializ

Re: Ignite-2.0 - Make near readers update async by default

2017-02-06 Thread Vladimir Ozerov
Yakov, I think forcing reads in TX or from primary is not what users expect by default. So i would have this mode disabled by default. On Mon, Feb 6, 2017 at 10:40 AM, Yakov Zhdanov wrote: > >It still seems that outdated reads will be *more* outdated with async > mode. > >Also, is there a guara

Re: Ensure that builder approach is used for all setters in public API

2017-02-06 Thread Vladimir Ozerov
> > >> Vladimir, > > >> > > >> Ok. I'll go ahead with changing SPI interfaces and run TC test. > > >> I think, it would be better to have this branch merged to master as 2 > > >> separate commits at least. > > >> And ma

Re: Do not use H2 parser for DDL

2017-02-06 Thread Vladimir Ozerov
-funcs.html > > Sergi > > 2017-02-03 17:41 GMT+03:00 Vladimir Ozerov : > > > Sergi, > > > > Does H2 support naming for function parameters? E.g. NEW_CACHE(backups: > 1, > > mode: PARTITIONED) > > > > On Fri, Feb 3, 2017 at 4:28 PM, Sergi Vladykin

Re: jdbc vs jdbc2 packages

2017-02-06 Thread Vladimir Ozerov
Big +1 to Val, not only about JDBC, but about our overall approach to clients. Starting a node with "client=true" is: + Very reach feature set, which is cool - Tons of dependencies - Tons of threads It would be very cool if we have a true thin client with small single JAR. It should have: - Failov

Rethink native SQL API in Apache Ignite 2.0

2017-02-07 Thread Vladimir Ozerov
Igniters, Our SQL engine becomes more and more sophisticated. Initially we had only SELECTs, now we have DML, in AI 2.x we will have DDL. This is very cool, but it seems that we completely forgot about extending our native SQL API (IgniteCache.query, SqlQuery, SqlFieldsQuery) in response to these

Re: Ensure that builder approach is used for all setters in public API

2017-02-07 Thread Vladimir Ozerov
; > > > Can you list all setters that we have on MBeans? > > > > -Val > > > > On Mon, Feb 6, 2017 at 2:21 PM, Andrey Mashenkov < > > andrey.mashen...@gmail.com > > > wrote: > > > > > Folks, > > > > > > Changing MB

Re: DML data streaming

2017-02-10 Thread Vladimir Ozerov
I propose to ship streaming with INSERT support only for now. This is enough for multitude cases and will add value to Ignite 1.9 immediately. We can think about correct streaming UPDATE/DELETE architecture separately .It is much more difficult thing, we cannot support it in a clean way right now d

Custom thread pools for compute tasks

2017-02-11 Thread Vladimir Ozerov
Igniters, We have one well-known limitation: one distributed opeartion cannot be executed from within another if they both end up in the same pool, because it will lead to starvation and hang. Most notable example - running data streamer from compute task. Users with complex logic will have the s

Re: IGNITE-13

2017-02-12 Thread Vladimir Ozerov
Hi, It is hard to say whether it makes sense or not. No doubt, it could speed up marshalling process at the cost of 2x memory required for strings. From my previous experience with marshalling micro-optimizations, we will hardly ever notice speedup in distributed environment. But, there is anothe

Re: general question

2017-02-12 Thread Vladimir Ozerov
Alex, FieldAccessor stores field positions for the given schema to allow for fast field value lookup. On Fri, Feb 10, 2017 at 7:06 PM, ALEKSEY KUZNETSOV wrote: > Guys, i have a question.Why do we store fields position information in > FieldAccessor before hashcode evaluation? Test example is in

Re: general question

2017-02-13 Thread Vladimir Ozerov
[A, C, B], [B, A, C], [B, C, A]. [C, A, B], [C, B, A]. We use schemas to achieve O(1) field lookup time. On Mon, Feb 13, 2017 at 11:05 AM, ALEKSEY KUZNETSOV < alkuznetsov...@gmail.com> wrote: > What is schema in your context? > > пн, 13 февр. 2017 г. в 10:50, Vladi

Re: IGNITE-4158 - ready for review

2017-02-13 Thread Vladimir Ozerov
Merged to 2.0. On Sun, Feb 12, 2017 at 10:13 PM, Denis Magda wrote: > Hi Vyacheslav, > > Val, Vovan, please assist with the review > > — > Denis > > > On Feb 10, 2017, at 3:53 AM, Vyacheslav Daradur > wrote: > > > > Hello. I did it. > > > > Please, review. > > > > https://issues.apache.org/jira

Re: Apache Ignite 1.9

2017-02-14 Thread Vladimir Ozerov
Can we push to 1.9 and then merge/rebase to master? It should be more reliable than cherry-picking. On Tue, Feb 14, 2017 at 12:15 PM, Anton Vinogradov wrote: > Denis, > > Merge to the master first and then cherry-pick to 1.9 from there is a > *correct > *case. > > On Tue, Feb 14, 2017 at 5:24 AM

Re: DML data streaming

2017-02-14 Thread Vladimir Ozerov
m public API and some kind of mapping is introduced. AFAIK this should be handled as a part of IGNITE-3487 ]1]. [1] https://issues.apache.org/jira/browse/IGNITE-3487 On Sat, Feb 11, 2017 at 3:36 AM, Dmitriy Setrakyan wrote: > On Fri, Feb 10, 2017 at 3:36 AM, Vladimir Ozerov > wrote: > &g

Re: Custom thread pools for compute tasks

2017-02-14 Thread Vladimir Ozerov
Gents, I created the ticket for custom executors: IGNITE-4699 [1] Alexander, I am not sure it makes sense since we are talking about distributed operations. CallerRunsPolicy is not suitable where user intentionally asks for job execution on remote node. [1] https://issues.apache.org/jira/browse/I

Re: Rethink native SQL API in Apache Ignite 2.0

2017-02-14 Thread Vladimir Ozerov
ause adding new API is not a breaking change. > > > > Sergi > > > > 2017-02-10 4:55 GMT+03:00 Dmitriy Setrakyan : > > > > > Vladimir, > > > > > > Will these changes be backward compatible? > > > > > > Sergi,

Re: Inaccurate documentation about transactions

2017-02-15 Thread Vladimir Ozerov
Alexandr, If you PUT some value within transaction, this is absolutely normal that you will see it on successive GET in the _same_ transaction. DIRTY_READ is a different thing - it allows reads of not-yet-committed changes from _another_ transaction. On Wed, Feb 15, 2017 at 9:41 AM, Alexandr Kura

Re: DML data streaming

2017-02-15 Thread Vladimir Ozerov
PM, Dmitriy Setrakyan wrote: > Vova, > > Agree about the primitive types. However, it is not clear to me how the > mapping from a primitive type to a column name will be supported. Do you > have a design in mind? > > D. > > On Tue, Feb 14, 2017 at 6:16 AM, Vladimir

Re: Questions of the Ignite Configurations

2017-02-15 Thread Vladimir Ozerov
Hi Vyacheslav, Answering your questions: 1. Yes 2. Yes 3. Only binary infrastructure 4. Yes On Wed, Feb 15, 2017 at 12:32 PM, Vyacheslav Daradur wrote: > Hello everybody. > > I have some questions about classes of configuration in the Ignite. > > 1. IgniteConfiguration - is it only one instanc

Re: DML data streaming

2017-02-15 Thread Vladimir Ozerov
ee any improvement here. Usability will only suffer with this > change. > > I'd suggest to just add mapping for system columns like _key, _val , _ver. > > Sergi > > 2017-02-15 13:18 GMT+03:00 Vladimir Ozerov : > > > I think the whole QueryEntity class require re

Re: DML data streaming

2017-02-16 Thread Vladimir Ozerov
*pk_id* BIGINT PRIMARY KEY, val1 BIGINT, val2 VARCHAR ) keyFieldName = "*pk_id*", valFieldName = null (because value is complex and is composed of two attributes). Vladimir. On Wed, Feb 15, 2017 at 11:42 PM, Dmitriy Setrakyan wrote: > On Wed, Feb 15, 2017 at 4:28 AM

Re: IGNITE-602

2017-02-16 Thread Vladimir Ozerov
Hi Dmitriy, We should print all objects, because this is the whole purpose of GridToStringBuilder class - to print as much as possible in pretty format. For cyclic references, we should introduce smarter approach, ensuring that certain object is printed only once, I would use a kind of stack or Id

Re: IGNITE-602

2017-02-16 Thread Vladimir Ozerov
llo, Vladimir. Do you mean something like "ArrayList [size=1, > name=ArrayList [size=1, ArrayList {position 0}]]"? > > 2017-02-16 12:32 GMT+03:00 Vladimir Ozerov : > > > Hi Dmitriy, > > > > We should print all objects, because this is the whole purpose of >

DML usability: routing single column to several object fields

2017-02-16 Thread Vladimir Ozerov
Igniters, Consider the following pattern which is typically used by users when working through cache API and quering data through SQL at the same time: class Person { @QuerySqlField long id; @QuerySqlField String name; } cache.put(*id*, new Person(*id*, name)); SELECT *id*, nam

Re: IGNITE-2626 is cloned?

2017-02-16 Thread Vladimir Ozerov
Yes, IGNITE-3191 resolves the problem mentioned in IGNITE-2626. No configuration parameter needed. We have system property to control this, but only for compatibility reasons - without it users might have lost ability to migrate from [1.x] to 1.[x+1] if they persist objects outside of Ignite. On F

Re: DML data streaming

2017-02-16 Thread Vladimir Ozerov
val BIGINT } On Fri, Feb 17, 2017 at 12:40 AM, Dmitriy Setrakyan wrote: > Vladimir, I am not sure I understand your point. The value type name should > be the table name, no? > > On Thu, Feb 16, 2017 at 12:13 AM, Vladimir Ozerov > wrote: > > > Dima, > > > >

Re: DML usability: routing single column to several object fields

2017-02-17 Thread Vladimir Ozerov
> will fit into the current design. Am I wrong? > > D. > > On Thu, Feb 16, 2017 at 4:33 AM, Vladimir Ozerov > wrote: > > > Igniters, > > > > Consider the following pattern which is typically used by users when > > working through cache

Re: Apache Ignite 1.9

2017-02-17 Thread Vladimir Ozerov
; >>> an > >> > >>> >> outdated Spark version. > >> > >>> >> > >> > >>> >> Anton, could you finalize it this week? As far as I remember, > the > >> > only > >> > >>> >> thing

Re: Apache Ignite 1.9

2017-02-20 Thread Vladimir Ozerov
Igniters, IGNITE-4169 [1] (streaming in DML) has been merged to 1.9 branch. [1] https://issues.apache.org/jira/browse/IGNITE-4169 On Fri, Feb 17, 2017 at 4:01 PM, Anton Vinogradov wrote: > https://issues.apache.org/jira/browse/IGNITE-3862 > already at 1.9, fixed "fixVersion". > > On Fri, Feb

Re: IGNITE-3422 - ready for review

2017-02-20 Thread Vladimir Ozerov
Hi Vyacheslav, Thank you for contribution. I reviewed implementation again and now I am in doubts whether our product would really benefit from it or not. See my comments in the ticket. I'de prefer Denis Magda to chime in and give his feedback first. Vladimir. On Wed, Feb 15, 2017 at 5:00 PM, Vy

Re: Entry filter in IgniteCache#loadCache

2017-02-20 Thread Vladimir Ozerov
+1 On Sat, Feb 18, 2017 at 11:51 PM, Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Igniters, > > Currently IgniteCache.loadCache has optional entry filter that can be > passed as an argument. It seems to be redundant because: > >- Filtering that is done by this filter can be a

Re: Locking of partition with affinityRun/affinityCall

2017-03-01 Thread Vladimir Ozerov
I do not like the change because it is too verbose and adds no useful information for users. Moreover, normally users have no idea what "partition eviction" means. Let's simply write "Partition's data is guaranteed to be available on the node while job is being executed". This should be enough. On

Re: JIRA ticket

2017-03-01 Thread Vladimir Ozerov
Igor, Please consider throwing away *startTime() *and *duration() *from future interface. These two methods are not need in 99% cases, but consume memory. They should not belong to future. On Wed, Mar 1, 2017 at 2:22 PM, Yakov Zhdanov wrote: > Igor, please go ahead. > > --Yakov > > 2017-03-01 1

Re: Additional SQL metrics

2017-03-02 Thread Vladimir Ozerov
Denis, Query execution is complex process involving different stages which are not very easy to match with each other. Especially provided that any node can leave topology at any time. Another problem is that engine evolves and metrics like "did a query do broadcast or unicast" may easily become u

Re: Additional SQL metrics

2017-03-02 Thread Vladimir Ozerov
e phase, > join phase) into an execution plan produced by H2? Are there any > implementation hooks? > > — > Denis > > > > On Mar 2, 2017, at 12:02 PM, Dmitriy Setrakyan > wrote: > > > > I think some of the metrics specified by Denis also make sense, so I > would &

Re: [VOTE] Apache Ignite 1.9.0 RC2

2017-03-05 Thread Vladimir Ozerov
+1 (binding) On Fri, Mar 3, 2017 at 9:59 PM, Seliverstov Igor wrote: > +1 > > 2 мар. 2017 г. 11:45 ПП пользователь "Valentin Kulichenko" < > valentin.kuliche...@gmail.com> написал: > > > +1 (binding) > > > > -Val > > > > On Thu, Mar 2, 2017 at 11:33 AM, Denis Magda wrote: > > > > > +1 (binding)

Re: Continuations for services

2017-03-08 Thread Vladimir Ozerov
Val, Don't panic! :) I created a ticket for custom thread pools for compute tasks recently. Probably it can help us with services as well. 08 марта 2017 г. 22:15 пользователь "Dmitriy Setrakyan" < dsetrak...@apache.org> написал: How about deploying services in their own thread pools? On Wed, M

  1   2   3   4   5   6   7   8   9   10   >