Re: Unused variables in code

[Yakov Zhdanov]

Thanks, Alexander! I looked this through and it seems fine to me.

Anton V, can you also review and merge if it is OK to you?

--Yakov

2017-01-31 12:49 GMT+03:00 Александр Меньшиков :

> And result of testing:
> http://ci.ignite.apache.org/project.html?projectId=IgniteTests=
> projectOverview_IgniteTests=pull/1466/head
>
> 2017-01-30 18:39 GMT+03:00 Александр Меньшиков :
>
> > Done.
> >
> > My PR:
> > https://github.com/apache/ignite/pull/1466/files
> > 
> >
> >
> >
> >
> > 2017-01-25 13:11 GMT+03:00 Александр Меньшиков :
> >
> >> Okay, I created issue:
> >> https://issues.apache.org/jira/browse/IGNITE-4612
> >>
> >>
> >> 2017-01-24 18:19 GMT+03:00 Alexey Kuznetsov :
> >>
> >>> I think it is better to have such issue in JIRA "Minor code cleanup":
> >>>
> >>> 1) Cleanup unused imports.
> >>> 2) Rename e -> ignored
> >>> 3) Remove unused variables
> >>> 4)...
> >>> 5) PROFIT :)
> >>>
> >>> On Tue, Jan 24, 2017 at 9:46 PM, Александр Меньшиков <
> >>> sharple...@gmail.com>
> >>> wrote:
> >>>
> >>> > Hello, everyone!
> >>> >
> >>> > I found some unused variables in code. For example:
> >>> >
> >>> > Unnecessary argument in method
> >>> > MiniFuture#onResult(ClusterTopologyCheckedException e) in inner
> class
> >>> of
> >>> > GridDhtLockFuture.
> >>> >
> >>> > There isn't another onResult() method (in super class including) with
> >>> > another behavior. So argument can be safe delete.
> >>> >
> >>> > Also I found some ignored exceptions wasn't named "ignored". For
> >>> example
> >>> > unused exception was named "e0" in method:
> >>> >
> >>> > GridDhtTransactionalCacheAdapter#onForceKeysError(UUID,
> >>> > GridDhtLockRequest,
> >>> > IgniteCheckedException)
> >>> >
> >>> >
> >>> > So I think i can go through code and cleanup thing like that.
> >>> >
> >>> > Need I create new issue in JIRA for cleanup or some issue for cleanup
> >>> > already exists? If that please give me a link.
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Alexey Kuznetsov
> >>>
> >>
> >>
> >
>

[jira] [Created] (IGNITE-4632) AffinityFunction unchecked exception handling (unassigned backup)

[Alexandr Kuramshin (JIRA)]

Alexandr Kuramshin created IGNITE-4632:
--

 Summary: AffinityFunction unchecked exception handling (unassigned 
backup)
 Key: IGNITE-4632
 URL: https://issues.apache.org/jira/browse/IGNITE-4632
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 1.8
Reporter: Alexandr Kuramshin
Priority: Minor


{{AffinityFunction}} implementation may throw unchecked exception upon 
assignment. In some cases additional processing should be performed when 
affinity function method invocation throws an exception.

Special case when the cache with backups is running, and a node with a primary 
partition will left. Then we get the primary partition unassigned if 
{{AffinityFunction.partition(Object)}} throws an exception. My suggestion is to 
shutdown the node in such the case (like SEGMENTED), because the cluster could 
not work normally without the primary partition assigned.

{noformat}
Failed processing message [senderId=8a1ab9a3-786e-4601-ba22-efd380849d99, 
msg=GridDhtPartitionSupplyMessageV2 [updateSeq=16069, 
topVer=AffinityTopologyVersion [topVer=7, minorTopVer=0], missed=[16, 17, 33, 
22, 56, 10], clean=[0, 1, 2, 34, 3, 5, 7, 9, 45, 46, 49, 18, 50, 55, 25, 26, 
58, 29, 61], msgSize=0, size=19, parts=[0, 1, 2, 34, 3, 5, 7, 9, 45, 46, 49, 
18, 50, 55, 25, 26, 58, 29, 61], super=GridCacheMessage [msgId=70098615, 
depInfo=null, err=null, skipPrepare=false, cacheId=-148990687, 
cacheId=-148990687]]]
com.sbt.persistence.exceptions.DPLException: ParticleKeyMapper не может 
обратывать никаких других объектов кроме ОУ. Системная ошибка - обратитесь в 
службу технической поддержки DPL
 at 
com.sbt.dpl.gridgain.ParticleAffinityFunction.partition(ParticleAffinityFunction.java:67)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:219)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:194)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.localNode(GridCacheAffinityManager.java:382)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.handleSupplyMessage(GridDhtPartitionDemander.java:680)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleSupplyMessage(GridDhtPreloader.java:390)
 at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:395)
 at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:385)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:758)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Python Platform for Ignite

[Shashank Gandham]

Okay, I will start look into the C++ shared libraries. 

> On 31-Jan-2017, at 1:44 AM, Denis Magda  wrote:
> 
> Hi Shashank,
> 
> I’ve prepared tickets related to this activity:
> https://issues.apache.org/jira/browse/IGNITE-4600 
> 
> 
> I would agree that if we want to be as generic as possible then we need to 
> write our own implementation rather relying on SWIG.
> So, let’s try to base Ignite.Python on existing Ignite.C++ shared libs that 
> Vladimir mentioned. Could you investigate this part? The community will 
> assist if something is unclear or required more details.
> 
> —
> Denis
> 
>> On Jan 28, 2017, at 10:22 AM, Shashank Gandham  
>> wrote:
>> 
>> *As asked by Denis (Correction from Dmitry)
>> 
>>> On 28-Jan-2017, at 11:17 PM, Shashank Gandham  
>>> wrote:
>>> 
>>> As I mentioned earlier I would like to work on developing the Python 
>>> platform [1]
>>> 
>>> Vladimir made things clear that we will have to develop a Python platform 
>>> from the ground up if we have to have all the functionalities of Ignite. I 
>>> think it would be worth the effort to build things from the ground up, 
>>> rather than using the REST API or any other workarounds [2]
>>> 
>>> As asked by Dmitry, I am starting a new thread to have a broader discussion 
>>> on how should we work on this. This will be my first contribution to a 
>>> project as big as Ignite so I may take some time to adjust to all the 
>>> standards and procedures, I assure I will pick up the pace as fast as 
>>> possible.
>>> 
>>> 
>>> [1] 
>>> http://apache-ignite-developers.2346864.n4.nabble.com/GSOC-2017-td13970.html
>>> [2] 
>>> http://apache-ignite-developers.2346864.n4.nabble.com/Read-this-if-you-want-to-integrate-Ignite-with-other-platforms-Python-R-etc-td14006.html
> 

Re: Review for IGNITE-4436 API to collect list of running SQL queries

[Denis Magda]

> On Jan 31, 2017, at 4:24 PM, Alexey Kuznetsov  wrote:
> 
> Denis,
> 
>>> I would make it transparent for the user giving ability to work with
> LOCAL queries as well. How difficult is to support this?
> I think it not should be very hard, but I will need some help from our SQL
> experts.
> 
>>> Isn’t the goal of this task to support all SQL queries? Don’t get why
> you ask about SqlQuery.
> I implemented support for SqlFieldsQuery, but in Ignite we have othre
> queries: SqlQuery, ScanQuery, TextQuery….
> 

Shouldn’t the implementation for SqlQueries be the same as for 
SqlFieldsQueries? Does it requires additional efforts?
I had a feeling that both types of SQL queries will share a single 
implementation in terms of the APIs you’ve contributed.

 As for ScanQuery I would do it as a next step.
> Well I think we could go this way. If it will be too hard to do it at once.
> 
> By the way, how about to include this feature in Ignite1.9?

Absolutely, if you can complete it by the time of 1.9 code freeze.

—
Denis

> 
> 
> On Wed, Feb 1, 2017 at 1:27 AM, Denis Magda  wrote:
> 
>> Hi Alex,
>> 
 1) Current implementation will collect info and cancel only "TwoStep"
 queries.
   Should we also support LOCAL queries?
 
>> 
>> I would make it transparent for the user giving ability to work with LOCAL
>> queries as well. How difficult is to support this?
>> 
 2) Should we also support SqlQuery / ScanQuery ?
>> 
>> Isn’t the goal of this task to support all SQL queries? Don’t get why you
>> ask about SqlQuery.
>> As for ScanQuery I would do it as a next step.
>> 
>> The review of the code should be done by Sergi. Let’s give him some time
>> to do that.
>> 
>> —
>> Denis
>> 
>>> On Jan 31, 2017, at 5:13 AM, Alexey Kuznetsov 
>> wrote:
>>> 
>>> Any feedback, please?
>>> 
>>> On Mon, Jan 30, 2017 at 9:38 PM, Alexey Kuznetsov >> 
>>> wrote:
>>> 
 Hi All!
 
 I prepared first version https://issues.apache.org/
>> jira/browse/IGNITE-4436
 (SQL: create API to collect list of running SQL queries and cancel
>> them).
 
 Please review it and give a feedback: http://reviews.ignit
 e.apache.org/ignite/review/IGNT-CR-84
 
 But also I have following questions:
 1) Current implementation will collect info and cancel only "TwoStep"
 queries.
   Should we also support LOCAL queries?
 
 2) Should we also support SqlQuery / ScanQuery ?
 
 
>>> 
>>> 
>>> --
>>> Alexey Kuznetsov
>> 
>> 
> 
> 
> -- 
> Alexey Kuznetsov

Re: Review for IGNITE-4436 API to collect list of running SQL queries

[Alexey Kuznetsov]

Denis,

>> I would make it transparent for the user giving ability to work with
LOCAL queries as well. How difficult is to support this?
I think it not should be very hard, but I will need some help from our SQL
experts.

>> Isn’t the goal of this task to support all SQL queries? Don’t get why
you ask about SqlQuery.
I implemented support for SqlFieldsQuery, but in Ignite we have othre
queries: SqlQuery, ScanQuery, TextQuery

>>>As for ScanQuery I would do it as a next step.
Well I think we could go this way. If it will be too hard to do it at once.

By the way, how about to include this feature in Ignite1.9?


On Wed, Feb 1, 2017 at 1:27 AM, Denis Magda  wrote:

> Hi Alex,
>
> >> 1) Current implementation will collect info and cancel only "TwoStep"
> >> queries.
> >>Should we also support LOCAL queries?
> >>
>
> I would make it transparent for the user giving ability to work with LOCAL
> queries as well. How difficult is to support this?
>
> >> 2) Should we also support SqlQuery / ScanQuery ?
>
> Isn’t the goal of this task to support all SQL queries? Don’t get why you
> ask about SqlQuery.
> As for ScanQuery I would do it as a next step.
>
> The review of the code should be done by Sergi. Let’s give him some time
> to do that.
>
> —
> Denis
>
> > On Jan 31, 2017, at 5:13 AM, Alexey Kuznetsov 
> wrote:
> >
> > Any feedback, please?
> >
> > On Mon, Jan 30, 2017 at 9:38 PM, Alexey Kuznetsov  >
> > wrote:
> >
> >> Hi All!
> >>
> >> I prepared first version https://issues.apache.org/
> jira/browse/IGNITE-4436
> >> (SQL: create API to collect list of running SQL queries and cancel
> them).
> >>
> >> Please review it and give a feedback: http://reviews.ignit
> >> e.apache.org/ignite/review/IGNT-CR-84
> >>
> >> But also I have following questions:
> >> 1) Current implementation will collect info and cancel only "TwoStep"
> >> queries.
> >>Should we also support LOCAL queries?
> >>
> >> 2) Should we also support SqlQuery / ScanQuery ?
> >>
> >>
> >
> >
> > --
> > Alexey Kuznetsov
>
>


-- 
Alexey Kuznetsov

Re: moving to geronimo JCache jar

[Alexander Fedotov]

Denis, it is my mistake to leave the header unchanged.
It should be fixed because from now on the generation of license notes for
dependencies under Apache Software License is enabled according to the
point 3 in JIRA .
I'll fix it and your notes in Upsource and update the PR.



On Tue, Jan 31, 2017 at 10:30 PM, Denis Magda  wrote:

> Alexander, provided review notes in the Upsource.
>
> However, I’m still a bit concerned about the content of
> ignite-core-licenses.txt (see attached). The file says that it contains
> licenses different from the Apache Software license but in fact lists
> shmem, Intellij IDEA annotations and JSR 107 all of which are available
> under Apache 2.0.
>
> Why is this so? Can someone explain? Dmitriy, probable you know the reason.
>
>
> —
> Denis
>
> > On Jan 30, 2017, at 12:19 PM, Denis Magda  wrote:
> >
> > Alexander, thanks!
> >
> > I’ll review it in the nearest couple of days.
> >
> > —
> > Denis
> >
> >> On Jan 30, 2017, at 5:10 AM, Alexander Fedotov <
> alexander.fedot...@gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> Created Upsource review for the subject:
> >> http://reviews.ignite.apache.org/ignite/review/IGNT-CR-82
> >>
> >> On Mon, Jan 30, 2017 at 11:52 AM, Alexander Fedotov <
> >> alexander.fedot...@gmail.com> wrote:
> >>
> >>> Hi all,
> >>>
> >>> https://issues.apache.org/jira/browse/IGNITE-3793 is completed.
> >>> Kindly take a look at the corresponding PR https://github.com/apache/i
> >>> gnite/pull/1475 .
> >>>
> >>> On Wed, Jan 25, 2017 at 8:04 PM, Denis Magda 
> wrote:
> >>>
>  We need to replace content of ignite-core-licenses.txt file which is
> the
>  following at the moment
> 
>  // --
>  // List of ignite-core module's dependencies provided as a part of
> this
>  distribution
>  // which licenses differ from Apache Software License.
>  // --
> 
>  
>  ==
>  For JSR107 API and SPI (https://github.com/jsr107/jsr107spec)
>  javax.cache:cache-api:jar:1.0.0
>  
>  ==
>  This product bundles JSR107 API and SPI which is available under a:
>  JSR-000107 JCACHE 2.9 Public Review - Updated Specification License.
> For
>  details, see https://raw.github.com/jsr107/
> jsr107spec/master/LICENSE.txt.
> 
> 
>  Updated this ticket description: https://issues.apache.org/jira
>  /browse/IGNITE-3793
> 
>  —
>  Denis
> > On Jan 24, 2017, at 8:24 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
>  wrote:
> >
> > Awesome, you are right. I just checked and the license is indeed
> Apache
> > 2.0. Is there anything we need to do at all right now?
> >
> > On Tue, Jan 24, 2017 at 8:17 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> >> This change was incorporated in this ticket: https://issues.apache.
> >> org/jira/browse/IGNITE-3793. We can't do it before 2.0 for
>  compatibility
> >> reasons.
> >>
> >> However, my point is that they changed the license to Apache 2.0, so
>  I'm
> >> not sure that licensing issue still exists.
> >>
> >> -Val
> >>
> >> On Tue, Jan 24, 2017 at 7:04 PM, Dmitriy Setrakyan <
>  dsetrak...@apache.org>
> >> wrote:
> >>
> >>> Any reason why we need to wait for 2.0? Sorry if this has already
> been
> >>> discussed.
> >>>
> >>> On Tue, Jan 24, 2017 at 7:02 PM, Denis Magda 
>  wrote:
> >>>
>  Yes, we planned to do that in 2.0. Val, the ticket is closed
>  https://issues.apache.org/jira/browse/IGNITE-2949 <
>  https://issues.apache.org/jira/browse/IGNITE-2949>
> 
>  Do we need to reopen it making sure that geronimo jar is added to
>  2.0?
> 
>  —
>  Denis
> 
> > On Jan 24, 2017, at 6:36 PM, Dmitriy Setrakyan <
> >> dsetrak...@apache.org>
>  wrote:
> >
> > We absolutely need to upgrade to the geronimo jcache library in
> the
> >>> next
> > release.
> >
> > On Tue, Jan 24, 2017 at 3:45 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> >> Guys,
> >>
> >> I noticed that the JCache license was updated to Apache 2.0
> several
>  months
> >> ago [1]. However, there was no release with the new license and
> >> 1.0.0
>  still
> >> has the old license name in the POM file [2] (the link is
> pointing
> >> to
>  the
> >> new one though).
> >>
> >> Is this enough from 

Re: moving to geronimo JCache jar

[Denis Magda]

Alexander, provided review notes in the Upsource.

However, I’m still a bit concerned about the content of 
ignite-core-licenses.txt (see attached). The file says that it contains 
licenses different from the Apache Software license but in fact lists shmem, 
Intellij IDEA annotations and JSR 107 all of which are available under Apache 
2.0.

Why is this so? Can someone explain? Dmitriy, probable you know the reason.
// --
// List of ignite-core module's dependencies provided as a part of this 
distribution
// which licenses differ from Apache Software License.
// --

==
For JSR107 API and SPI (https://github.com/jsr107/jsr107spec) 
javax.cache:cache-api:jar:1.0.0
==
This product bundles JSR107 API and SPI which is available under a:
The Apache Software License, Version 2.0. For details, see 
https://raw.github.com/jsr107/jsr107spec/master/LICENSE.txt.

==
For ignite-shmem (http://www.gridgain.com) org.gridgain:ignite-shmem:jar:1.0.0
==
This product bundles ignite-shmem which is available under a:
The Apache Software License, Version 2.0. For details, see 
http://www.apache.org/licenses/LICENSE-2.0.txt.

==
For IntelliJ IDEA Annotations (http://www.jetbrains.org) 
org.jetbrains:annotations:jar:13.0
==
This product bundles IntelliJ IDEA Annotations which is available under a:
The Apache Software License, Version 2.0. For details, see 
http://www.apache.org/licenses/LICENSE-2.0.txt.




—
Denis

> On Jan 30, 2017, at 12:19 PM, Denis Magda  wrote:
> 
> Alexander, thanks!
> 
> I’ll review it in the nearest couple of days.
> 
> —
> Denis
> 
>> On Jan 30, 2017, at 5:10 AM, Alexander Fedotov 
>>  wrote:
>> 
>> Hi,
>> 
>> Created Upsource review for the subject:
>> http://reviews.ignite.apache.org/ignite/review/IGNT-CR-82
>> 
>> On Mon, Jan 30, 2017 at 11:52 AM, Alexander Fedotov <
>> alexander.fedot...@gmail.com> wrote:
>> 
>>> Hi all,
>>> 
>>> https://issues.apache.org/jira/browse/IGNITE-3793 is completed.
>>> Kindly take a look at the corresponding PR https://github.com/apache/i
>>> gnite/pull/1475 .
>>> 
>>> On Wed, Jan 25, 2017 at 8:04 PM, Denis Magda  wrote:
>>> 
 We need to replace content of ignite-core-licenses.txt file which is the
 following at the moment
 
 // --
 // List of ignite-core module's dependencies provided as a part of this
 distribution
 // which licenses differ from Apache Software License.
 // --
 
 
 ==
 For JSR107 API and SPI (https://github.com/jsr107/jsr107spec)
 javax.cache:cache-api:jar:1.0.0
 
 ==
 This product bundles JSR107 API and SPI which is available under a:
 JSR-000107 JCACHE 2.9 Public Review - Updated Specification License. For
 details, see https://raw.github.com/jsr107/jsr107spec/master/LICENSE.txt.
 
 
 Updated this ticket description: https://issues.apache.org/jira
 /browse/IGNITE-3793
 
 —
 Denis
> On Jan 24, 2017, at 8:24 PM, Dmitriy Setrakyan 
 wrote:
> 
> Awesome, you are right. I just checked and the license is indeed Apache
> 2.0. Is there anything we need to do at all right now?
> 
> On Tue, Jan 24, 2017 at 8:17 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> 
>> This change was incorporated in this ticket: https://issues.apache.
>> org/jira/browse/IGNITE-3793. We can't do it before 2.0 for
 compatibility
>> reasons.
>> 
>> However, my point is that they changed the license to Apache 2.0, so
 I'm
>> not sure that licensing issue still exists.
>> 
>> -Val
>> 
>> On Tue, Jan 24, 2017 at 7:04 PM, Dmitriy Setrakyan <
 dsetrak...@apache.org>
>> wrote:
>> 
>>> Any reason why we need to wait for 2.0? Sorry if this has already been
>>> discussed.
>>> 
>>> On Tue, Jan 24, 2017 at 7:02 PM, Denis Magda 
 wrote:
>>> 
 Yes, we planned to do that in 2.0. Val, the ticket is closed
 https://issues.apache.org/jira/browse/IGNITE-2949 <
 

Re: Integration with RocketMQ

[Denis Magda]

Roman,

Do you have any progress on this? It will be perfect to have the feature in 
1.9. Please join the discussion:
http://apache-ignite-developers.2346864.n4.nabble.com/Apache-Ignite-1-9-td14170.html
 


—
Denis

> On Jan 11, 2017, at 5:09 PM, Roman Shtykh  wrote:
> 
> Denis,
> Yes, I plan to complete it by that time.
> #Btw, RocketMQ first release (4.0.0) under Apache license is planned for this 
> month. Roman 
> 
>On Thursday, January 12, 2017 7:30 AM, Denis Magda  
> wrote:
> 
> 
> Roman,
> 
> Makes perfect sense to me. Do you think we can release this feature in the 
> nearest AI release (1.9 or 2.0) around February/March?
> 
> —
> Denis
> 
>> On Jan 10, 2017, at 7:57 PM, Roman Shtykh  wrote:
>> 
>> Denis,
>> I think our Kafka integration is the closest to what may be required for 
>> RockerMQ. But also I wouldn't exclude expanding to Camel's approach in 
>> future, thanks a lot for pointing to it!
>> Let's have it with other streamers as Yakov suggested, and let's discuss 
>> again if something fancy is requested ;)
>> I will create a JIRA issue for that and ask RocketMQ developers for reviews 
>> and suggestions from their side. Roman 
>> 
>> On Wednesday, January 11, 2017 12:12 PM, Denis Magda  
>> wrote:
>> 
>> 
>> Roman,
>> 
>> Does it make sense to have corresponding integration modules in both 
>> projects?
>> 
>> For instance, there is Camel and Ignite integration that is hosted in both 
>> projects.
>> https://github.com/apache/ignite/tree/master/modules/camel
>> https://github.com/apache/camel/tree/master/components/camel-ignite 
>> 
>> 
>> I do believe that the modules are not 100% identical and the reason it’s 
>> done this way might be because the modules use different Camel or Ignite 
>> interfaces and concepts. Is this approach somehow related to RocketMQ and 
>> Ignite?
>> 
>> —
>> Denis
>> 
>>> On Jan 9, 2017, at 11:06 PM, Roman Shtykh  wrote:
>>> 
>>> Igniters,
>>> Recently I am looking into a new pub/sub messaging Apache incubator project 
>>> called RocketMQ [1], originally developed at Alibaba. And I am thinking 
>>> about implementing a streamer for Ignite.
>>> Do you think it has to be placed under modules/ of Ignite project, or 
>>> better to have it together with other RocketMQ Community Projects [2]? I 
>>> talked to the project's lead -- any approach seems to be ok.
>>> I anticipate quite frequent releases of the project this year (much more 
>>> frequent than Ignite's), and I think it will be easier to release such 
>>> integration updates with the 2nd approach, as soon as a new RocketMQ's (or 
>>> Ignite's) version is available.
>>> Any thoughts?
>>> [1] http://rocketmq.apache.org/[2] https://github.com/rocketmq
>>> Roman
>> 
>> 
> 
> 

Apache Ignite 1.9

[Denis Magda]

Igniters,

Looks like 2.0 will be slightly delayed since the community requires more time 
to make the new page memory architecture faster. At the same time we have a 
plenty of bug fixes and performance improvements that were merged or close to 
be merged into the master and ready to be released.

I propose to release all the changes we have in 1.9 close to the end of 
February.

According to the dev list discussions, we will be ready to release performance 
optimizations done by Yakov and Sam close to the end of 2016 and many SQL 
improvements driven by multiple folks.

In addition, this is what we might expect in 1.9:
- SQL execution over specific partitions 
(https://issues.apache.org/jira/browse/IGNITE-4523 
). Alexey Scherbakov.
- Spark version upgrade (https://issues.apache.org/jira/browse/IGNITE-3710 
). Anton V.
- Spark RDD Examples (https://issues.apache.org/jira/browse/IGNITE-4526 
). Manish Mishra.
- Rocket MQ data streamer (https://issues.apache.org/jira/browse/IGNITE-4539 
). Roman Shtykh.
- Hibernate 5 support (https://issues.apache.org/jira/browse/IGNITE-1794 
). Mykola Pereyma.
- .NET DML (https://issues.apache.org/jira/browse/IGNITE-4045 
). Pavel Tupitsyn.
- .NET Transaction Scope API (https://issues.apache.org/jira/browse/IGNITE-3430 
). Pavel Tupitsyn.
- C++ Continuous queries (https://issues.apache.org/jira/browse/IGNITE-1443 
). Igor Sapego.
- Kubernetes Support (https://issues.apache.org/jira/browse/IGNITE-4159 
). Denis Magda.
- JDBC batch mode for DML (https://issues.apache.org/jira/browse/IGNITE-4269 
). Alexander Paschenko.
- Benchmarks automation (https://issues.apache.org/jira/browse/IGNITE-4212 
). Oleg Ostanin.

Guys, please confirm that you will be able to complete listed tasks by the 
middle or by 20th of February. Anything else that is not in the list?

BTW, is there anyone who can take a role of the release manager for 1.9?

—
Denis

Re: Review for IGNITE-4436 API to collect list of running SQL queries

[Denis Magda]

Hi Alex,

>> 1) Current implementation will collect info and cancel only "TwoStep"
>> queries.
>>Should we also support LOCAL queries?
>> 

I would make it transparent for the user giving ability to work with LOCAL 
queries as well. How difficult is to support this?

>> 2) Should we also support SqlQuery / ScanQuery ?

Isn’t the goal of this task to support all SQL queries? Don’t get why you ask 
about SqlQuery. 
As for ScanQuery I would do it as a next step.

The review of the code should be done by Sergi. Let’s give him some time to do 
that.

—
Denis

> On Jan 31, 2017, at 5:13 AM, Alexey Kuznetsov  wrote:
> 
> Any feedback, please?
> 
> On Mon, Jan 30, 2017 at 9:38 PM, Alexey Kuznetsov 
> wrote:
> 
>> Hi All!
>> 
>> I prepared first version https://issues.apache.org/jira/browse/IGNITE-4436
>> (SQL: create API to collect list of running SQL queries and cancel them).
>> 
>> Please review it and give a feedback: http://reviews.ignit
>> e.apache.org/ignite/review/IGNT-CR-84
>> 
>> But also I have following questions:
>> 1) Current implementation will collect info and cancel only "TwoStep"
>> queries.
>>Should we also support LOCAL queries?
>> 
>> 2) Should we also support SqlQuery / ScanQuery ?
>> 
>> 
> 
> 
> -- 
> Alexey Kuznetsov

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

[Denis Magda]

Oleg,

Thanks for the clarification.

My opinion is that we should leave ‘modules/yardstick/README.txt’ and 
‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those 
instructions there that explain how to build and run benchmarks from 
‘modules/yardstick’. This existing files can refer to the sources and compiled 
benchmarks that are in Ignite binary releases but this should be a couple of 
statements, no more.

As for the instructions related to the sources and binaries added to Ignite 
binaries, preferably they need to be added to auto-generated README.txt. 
*Anton*, is it feasible to do?
 
Finally, when apply the reviews notes please build and share Ignite binaries 
with your patch. Want to see the new project structure and final version of the 
instructions. Presently I can’t merge your changes due to some conflicts.

*Anton*, please review modifications in the build procedures.

—
Denis

> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin  wrote:
> 
> Hi Denis,
> 
> Yes, we have included Ignite Yardstick source files with its pom.xml in
> Ignite binary release. "Building from standalone sources" is the building
> instruction for these source files. "Building from Ignite Sources" is the
> instruction for building Ignite Yardstick from `modules/yardstick` in
> Ignite source files.
> 
> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda  wrote:
> 
>> Hi Oleg,
>> 
>> Great progress, thanks for keep driving this!
>> 
>> I’ve left some minor notes in GitHub’s pull-request. I have the following
>> questions aside:
>> 
>> - What is the difference between "Building from standalone sources" and
>> "Building from Ignite Sources"? In my understanding, a user downloads
>> Apache Ignite release that has all the sources locally.
>> 
>> - I do remember we planned to add the benchmarks sources in a form of a
>> ready to be used project with its own pom.xml (similar to examples). Did
>> you put this task off?
>> 
>> —
>> Denis
>> 
>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin  wrote:
>>> 
>>> Hi!
>>> 
>>> I've changed the README.txt and DEVNOTES.txt files. Also added a simple
>>> config file for quick and easy start. Please take a look at them and tell
>>> me what you think.
>>> 
>>> https://github.com/apache/ignite/pull/1471
>>> 
>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov 
>> wrote:
>>> 
 Denis,
 
 I think we can remove all configs except:
 
 benchmark-multicast.properties
 
 benchmark.properties
 
 ignite-base-config.xml
 
 ignite-localhost-config.xml
 
 ignite-multicast-config.xml
 
 2016-12-28 2:49 GMT+03:00 Denis Magda :
 
> I would have only those configs that are useful. Ilya Suntsov, basing
>> on
> your experience, please suggest which configs makes sense to include
>> into
> every Ignite release.
> 
> Oleg, also please note that community decided to include not only the
> benchmarking binaries but the sources as well into every Apache Ignite
> release. I’ve update the ticket before. Hope you followed the
>> discussion
 ;)
> https://issues.apache.org/jira/browse/IGNITE-4212?
> focusedCommentId=15765151=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
> 
> —
> Denis
> 
>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin 
 wrote:
>> 
>> I mean removing those configs from binary assembly, not from
 repository.
>> 
>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin 
> wrote:
>> 
>>> Hello Igniters.
>>> I think it would be better to remove some configuration files from
>>> benchmarks/config:
>>> 
>>> ignite-base-load-config.xml
>>> ignite-cache-load-config.xml
>>> ignite-failover-base-config.xml
>>> ignite-failover-localhost-config.xml
>>> benchmark-cache-load.properties
>>> benchmark-cache-load-win.properties
>>> benchmark-failover.properties
>>> 
>>> because those configs do not relate to any of performance tests.
>>> 
>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda 
> wrote:
>>> 
 Summarized the discussion updating the ticket
 https://issues.apache.org/jira/browse/IGNITE-4212# <
 https://issues.apache.org/jira/browse/IGNITE-4212#>
 
 —
 Denis
 
> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
 wrote:
> 
> Sergey,
> 
> I am not sure I like "extras". I am voting for "benchmarks" folder
> right
> under the root folder.
> 
> D.
> 
> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
 skoz...@gridgain.com
>> 
> wrote:
> 
>> Formatting has cut lines:
>> 
>> — 

[GitHub] ignite pull request #1484: IGNITE-4631 Added check to ensure QueryEntity 'ke...

[skalashnikov]

GitHub user skalashnikov opened a pull request:

https://github.com/apache/ignite/pull/1484

IGNITE-4631 Added check to ensure QueryEntity 'keyFields' is subset o…

…f 'fields'

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4631

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1484.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1484


commit 81fa92dc8846cf6f9a031bb5d8a7255b02ab266e
Author: skalashnikov 
Date:   2017-01-31T17:26:44Z

IGNITE-4631 Added check to ensure QueryEntity 'keyFields' is subset of 
'fields'




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re:

[Denis Magda]

Hi,

Yes, you can and need to check changes incorporated in a pull-request using 
TeamCity. Refer to this documentation section:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-1.CreateGitHubpull-request
 


P.S. 
You’re welcomed to send messages like this to the dev list directly. There is a 
plenty of folks which can assist you earlier.

—
Denis

> On Jan 31, 2017, at 3:42 AM, ALEKSEY KUZNETSOV  
> wrote:
> 
> Hi! Is it possible pull request (into forked remote branch) to be build in 
> temacity?
> -- 
> Best Regards,
> 
> Kuznetsov Aleksey
> 

[GitHub] ignite pull request #1483: IGNITE-4570 Parsing for CREATE INDEX and DROP IND...

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/1483

IGNITE-4570 Parsing for CREATE INDEX and DROP INDEX



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4570

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1483.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1483


commit 12a7c8955cee563d43738a301dd9f34f75c689ad
Author: Alexander Paschenko 
Date:   2017-01-31T16:26:39Z

IGNITE-4570 Parsing for CREATE INDEX and DROP INDEX




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1482: IGNITE-4630 return valid date for org.apache.igni...

[ezhuravl]

GitHub user ezhuravl opened a pull request:

https://github.com/apache/ignite/pull/1482

IGNITE-4630 return valid date for 
org.apache.ignite.lang.IgniteProductVersion.releaseDate



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ezhuravl/ignite ignite-4630

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1482.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1482


commit d3d771672b76f86234cc41ec178ec8616c6470b4
Author: Evgenii Zhuravlev 
Date:   2017-01-31T14:07:45Z

IGNITE-4630 return valid date for 
org.apache.ignite.lang.IgniteProductVersion.releaseDate




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: How to improve SQL testing

[Oleg Ostanin]

Hi,

If there are no objections, I'd like implement this feature.

Oleg

On Tue, Jan 31, 2017 at 12:07 AM, Denis Magda  wrote:

> Hi,
>
> That’s true that the utility/testing framework needs to be as flexible as
> possible. If the community can’t reuse an existing one let’s make it up.
>
> As for VoltDB SQL Coverage Suite, if it’s Python based then can we
> implement those Python APIs directly calling Ignite.C++?
>
> —
> Denis
>
> > On Jan 30, 2017, at 4:39 AM, Sergey Kozlov  wrote:
> >
> > Alexey, I suppose that it's a hard way to do that and at least see a few
> > disadvantages below:
> >
> > 1. The VoltDB has no multithreaded mode. It means huge execution time
> > consumption (say we talk about thousands statement multiplied on caches
> > count, it's the millions of queries).
> > 2. We should add support of Ignite.
> > 3. We should add support of H2 as base engine for us (MySQL/Postgress
> have
> > less priority)
> > 4. They run it under Jenkins so we need to integrate it with TC somehow.
> >
> > From my standpoint better to make the utility in Ignite than spending
> time
> > and efforts to adopt an existing tool. At least we have good (or even
> best)
> > Java experts/developers in community and the task doesn't look a complex
> > one.
> >
> >
> > On Mon, Jan 30, 2017 at 2:58 PM, Alexey Kuznetsov  >
> > wrote:
> >
> >> Sergey,
> >>
> >> Could we use http://www.jython.org/ to make it works as-is?
> >>
> >> On Mon, Jan 30, 2017 at 6:06 PM, Sergey Kozlov 
> >> wrote:
> >>
> >>> Hi
> >>>
> >>> Hi reviewed the links below (thanks for Vova Ozerov to pointing me) and
> >>> found that the provided solution is really excellent!
> >>> Unfortunately we can't use it as is (from my standpoint) due lacks of
> >>> Python support in Ignite but we can re-use that idea for Ignite and
> make
> >>> SQL combinations coverage significantly better.
> >>>
> >>> I filed the ticket IGNITE-4627 Tester for SQL functionality
> >>> 
> >>>
> >>> Please share your thoughts and ideas.
> >>>
> >>> [1] https://www.voltdb.com/blog/testing-voltdb-sqlcoverage
> >>> [2] https://github.com/VoltDB/voltdb/tree/master/tests/sqlcoverage
> >>>
> >>> --
> >>> Sergey Kozlov
> >>> www.gridgain.com
> >>>
> >>
> >>
> >>
> >> --
> >> Alexey Kuznetsov
> >>
> >
> >
> >
> > --
> > Sergey Kozlov
> > GridGain Systems
> > www.gridgain.com
>
>

Re: How to improve SQL testing

[Evgeniy Stanilovskiy]

Sergey, did you take a look into https://en.wikipedia.org/wiki/YCSB ?
https://github.com/joshwilliams/YCSB
there is simple jdbc connector looks like it would be ok for us ?


Hi,

That’s true that the utility/testing framework needs to be as flexible  
as possible. If the community can’t reuse an existing one let’s make it  
up.


As for VoltDB SQL Coverage Suite, if it’s Python based then can we  
implement those Python APIs directly calling Ignite.C++?


—
Denis


On Jan 30, 2017, at 4:39 AM, Sergey Kozlov  wrote:

Alexey, I suppose that it's a hard way to do that and at least see a few
disadvantages below:

1. The VoltDB has no multithreaded mode. It means huge execution time
consumption (say we talk about thousands statement multiplied on caches
count, it's the millions of queries).
2. We should add support of Ignite.
3. We should add support of H2 as base engine for us (MySQL/Postgress  
have

less priority)
4. They run it under Jenkins so we need to integrate it with TC somehow.

From my standpoint better to make the utility in Ignite than spending  
time
and efforts to adopt an existing tool. At least we have good (or even  
best)

Java experts/developers in community and the task doesn't look a complex
one.


On Mon, Jan 30, 2017 at 2:58 PM, Alexey Kuznetsov  


wrote:


Sergey,

Could we use http://www.jython.org/ to make it works as-is?

On Mon, Jan 30, 2017 at 6:06 PM, Sergey Kozlov 
wrote:


Hi

Hi reviewed the links below (thanks for Vova Ozerov to pointing me)  
and

found that the provided solution is really excellent!
Unfortunately we can't use it as is (from my standpoint) due lacks of
Python support in Ignite but we can re-use that idea for Ignite and  
make

SQL combinations coverage significantly better.

I filed the ticket IGNITE-4627 Tester for SQL functionality


Please share your thoughts and ideas.

[1] https://www.voltdb.com/blog/testing-voltdb-sqlcoverage
[2] https://github.com/VoltDB/voltdb/tree/master/tests/sqlcoverage

--
Sergey Kozlov
www.gridgain.com





--
Alexey Kuznetsov





--
Sergey Kozlov
GridGain Systems
www.gridgain.com