Re: Service grid redesign

2018-11-20 Thread Vyacheslav Daradur
First, we must agree on names.

I think the valuable word is 'deployment', though 'exchange' may be
used too because there is an exchange of deployments maps during the
deployment process.

To avoid intersections with PME I suggest using the following names:

ServicesFullMapMessage -> ServicesFullDeploymentsMessage
ServicesSingleMapMessage -> ServicesSingleDeploymentsMessage
ServicesExchangeActions -> ServicesDeploymentActions
ServicesDeploymentExchangeId -> ServicesDeploymentId
ServicesDeploymentExchangeTask -> ServicesDeploymentTask
ServicesDeploymentExchangeWorker -> ServicesDeploymentWorker
ServicesDeploymentExchangeManager -> ServicesDeploymentManager
On Wed, Nov 21, 2018 at 10:49 AM Nikolay Izhikov  wrote:
>
> Hello, Vladimir.
>
> What is correct term?
>
>
> ср, 21 нояб. 2018 г., 10:29 Vladimir Ozerov voze...@gridgain.com:
>
> > Agree. Service deployment has nothing to do with PME. We should not use the
> > same term for different things.
> >
> > вт, 20 нояб. 2018 г. в 17:19, Denis Mekhanikov :
> >
> > > Vyacheslav,
> > >
> > > I'm in process of reviewing your changes. Sorry for taking so long.
> > > I posted the first portion of review comments yesterday.
> > > I'd like to finish looking through the code. I'll post more comments
> > later.
> > >
> > > I see, that you called things analogously to partition map exchange.
> > > I realize, that there is an analogy in used procedures, but I don't
> > really
> > > like the idea to use the same names for everything.
> > > The partition map exchange is called this way because it involves an
> > actual
> > > exchange of information.
> > > All nodes need to tell each other, which partitions they have, and what
> > > their states are.
> > >
> > > There is no exchange in case of service deployment, so I would skip the
> > > "exchange" part.
> > > And *single message ->* *full message* look more like *request ->
> > response*
> > > in case of services.
> > >
> > > Suppose we abandon the PME procedure and move to something else.
> > > Then *ServiceDeploymentExchange* name won't make sense.
> > > And I don't want to be in a situation, when I say to my colleague a word
> > > "exchange",
> > > and get "which one?" in return.
> > > So, I'm for the meaningful names rather than analogous to something else.
> > >
> > > What do you think?
> > >
> > > Denis
> > >
> > > вт, 20 нояб. 2018 г. в 12:09, Vyacheslav Daradur :
> > >
> > > > Denis, Yakov have you had a chance to review the solution?
> > > >
> > > > Igniters, we need to define a list of reviewers, otherwise no end in
> > > sign.
> > > >
> > > > I'm ready to continue work on the Service Grid, including new features
> > > > like hot-redeployment and versioning, also, I have ideas about new
> > > > tools for monitoring and management which will be useful for our
> > > > end-users.
> > > >
> > > > But for continuing work we need to overcome this first phase.
> > > >
> > > > On Tue, Nov 13, 2018 at 1:09 PM Vyacheslav Daradur <
> > daradu...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Denis, Yakov, feel free to contact me directly in case of questions.
> > > > Thanks!
> > > > >
> > > > > On Sun, Nov 11, 2018 at 10:09 PM Denis Mekhanikov <
> > > dmekhani...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > Guys,
> > > > > >
> > > > > > I'd like to take a look at the changes before they are merged.
> > > > > > I'll do my best to finish the review before the end of the upcoming
> > > > week.
> > > > > >
> > > > > > Thanks!
> > > > > > Denis
> > > > > >
> > > > > > сб, 10 нояб. 2018 г. в 14:25, Nikolay Izhikov  > >:
> > > > > >
> > > > > > > Hello, Vladimir.
> > > > > > >
> > > > > > > I'm agree with you.
> > > > > > >
> > > > > > > Can we write the list of reviewers for this feature?
> > > > > > > Without a date or similar.
> > > > > > > Just a list of experts who should review this feature.
> > > > > > >
> > > > > > > В Сб, 10/11/2018 в 14:01 +0300, Vladimir Ozerov пишет:
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > This is very huge thing with complex algorithms behind. We
> > should
> > > > not
> > > > > > > merge
> > > > > > > > it to the product unless several additional thorough reviews
> > are
> > > > ready,
> > > > > > > > irrespectively of how long will it take. We are about quality,
> > > not
> > > > speed.
> > > > > > > >
> > > > > > > > сб, 10 нояб. 2018 г. в 1:30, Denis Magda :
> > > > > > > >
> > > > > > > > > Vyacheslav,
> > > > > > > > >
> > > > > > > > > What are the cases when the service can be redeployed?
> > > Affinity,
> > > > > > > failure,
> > > > > > > > > etc., right. It would be good to list all the cases on the
> > wiki
> > > > and
> > > > > > > then
> > > > > > > > > our tech writers will get everything documented.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > > On Thu, Nov 8, 2018 at 11:06 PM Vyacheslav Daradur <
> > > > > > > daradu...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Denis,
> > > > > > 

Re: Service grid redesign

2018-11-20 Thread Nikolay Izhikov
Hello, Vladimir.

What is correct term?


ср, 21 нояб. 2018 г., 10:29 Vladimir Ozerov voze...@gridgain.com:

> Agree. Service deployment has nothing to do with PME. We should not use the
> same term for different things.
>
> вт, 20 нояб. 2018 г. в 17:19, Denis Mekhanikov :
>
> > Vyacheslav,
> >
> > I'm in process of reviewing your changes. Sorry for taking so long.
> > I posted the first portion of review comments yesterday.
> > I'd like to finish looking through the code. I'll post more comments
> later.
> >
> > I see, that you called things analogously to partition map exchange.
> > I realize, that there is an analogy in used procedures, but I don't
> really
> > like the idea to use the same names for everything.
> > The partition map exchange is called this way because it involves an
> actual
> > exchange of information.
> > All nodes need to tell each other, which partitions they have, and what
> > their states are.
> >
> > There is no exchange in case of service deployment, so I would skip the
> > "exchange" part.
> > And *single message ->* *full message* look more like *request ->
> response*
> > in case of services.
> >
> > Suppose we abandon the PME procedure and move to something else.
> > Then *ServiceDeploymentExchange* name won't make sense.
> > And I don't want to be in a situation, when I say to my colleague a word
> > "exchange",
> > and get "which one?" in return.
> > So, I'm for the meaningful names rather than analogous to something else.
> >
> > What do you think?
> >
> > Denis
> >
> > вт, 20 нояб. 2018 г. в 12:09, Vyacheslav Daradur :
> >
> > > Denis, Yakov have you had a chance to review the solution?
> > >
> > > Igniters, we need to define a list of reviewers, otherwise no end in
> > sign.
> > >
> > > I'm ready to continue work on the Service Grid, including new features
> > > like hot-redeployment and versioning, also, I have ideas about new
> > > tools for monitoring and management which will be useful for our
> > > end-users.
> > >
> > > But for continuing work we need to overcome this first phase.
> > >
> > > On Tue, Nov 13, 2018 at 1:09 PM Vyacheslav Daradur <
> daradu...@gmail.com>
> > > wrote:
> > > >
> > > > Denis, Yakov, feel free to contact me directly in case of questions.
> > > Thanks!
> > > >
> > > > On Sun, Nov 11, 2018 at 10:09 PM Denis Mekhanikov <
> > dmekhani...@gmail.com>
> > > wrote:
> > > > >
> > > > > Guys,
> > > > >
> > > > > I'd like to take a look at the changes before they are merged.
> > > > > I'll do my best to finish the review before the end of the upcoming
> > > week.
> > > > >
> > > > > Thanks!
> > > > > Denis
> > > > >
> > > > > сб, 10 нояб. 2018 г. в 14:25, Nikolay Izhikov  >:
> > > > >
> > > > > > Hello, Vladimir.
> > > > > >
> > > > > > I'm agree with you.
> > > > > >
> > > > > > Can we write the list of reviewers for this feature?
> > > > > > Without a date or similar.
> > > > > > Just a list of experts who should review this feature.
> > > > > >
> > > > > > В Сб, 10/11/2018 в 14:01 +0300, Vladimir Ozerov пишет:
> > > > > > > Igniters,
> > > > > > >
> > > > > > > This is very huge thing with complex algorithms behind. We
> should
> > > not
> > > > > > merge
> > > > > > > it to the product unless several additional thorough reviews
> are
> > > ready,
> > > > > > > irrespectively of how long will it take. We are about quality,
> > not
> > > speed.
> > > > > > >
> > > > > > > сб, 10 нояб. 2018 г. в 1:30, Denis Magda :
> > > > > > >
> > > > > > > > Vyacheslav,
> > > > > > > >
> > > > > > > > What are the cases when the service can be redeployed?
> > Affinity,
> > > > > > failure,
> > > > > > > > etc., right. It would be good to list all the cases on the
> wiki
> > > and
> > > > > > then
> > > > > > > > our tech writers will get everything documented.
> > > > > > > >
> > > > > > > > --
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > On Thu, Nov 8, 2018 at 11:06 PM Vyacheslav Daradur <
> > > > > > daradu...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Denis,
> > > > > > > > >
> > > > > > > > > Services reassignment process takes into account previous
> > > assignments
> > > > > > > > > to avoid redundant redeployments.
> > > > > > > > > So, in the described case, ServiceA won't be moved from
> node1
> > > to
> > > > > > node2.
> > > > > > > > > On Fri, Nov 9, 2018 at 4:41 AM Denis Magda <
> > dma...@apache.org>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Vyacheslav,
> > > > > > > > > >
> > > > > > > > > > First of all, thanks for archiving this milestone and
> > > rolling out
> > > > > > these
> > > > > > > > >
> > > > > > > > > new
> > > > > > > > > > capabilities.
> > > > > > > > > >
> > > > > > > > > > Speaking of the topology change events [1], does the new
> > > > > > architecture
> > > > > > > > >
> > > > > > > > > avoid
> > > > > > > > > > a running service redeployment when a new node joins? For
> > > instance,
> > > > > > > >
> > > > > > > > let's
> > > > > > > > > > say I have ServiceA running 

[GitHub] ignite pull request #5454: IGNITE-10356 JDBC thin driver returns wrong data ...

2018-11-20 Thread ldzhjn
GitHub user ldzhjn opened a pull request:

https://github.com/apache/ignite/pull/5454

IGNITE-10356 JDBC thin driver returns wrong data type for Date and Decimal 
SQL type



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ldzhjn/ignite IGNITE-10356

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5454.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5454


commit 1927c1617d165a8647ff45b010f395765d2ac68a
Author: rayliu 
Date:   2018-11-21T07:29:05Z

Fix bug




---


Re: Service grid redesign

2018-11-20 Thread Vladimir Ozerov
Agree. Service deployment has nothing to do with PME. We should not use the
same term for different things.

вт, 20 нояб. 2018 г. в 17:19, Denis Mekhanikov :

> Vyacheslav,
>
> I'm in process of reviewing your changes. Sorry for taking so long.
> I posted the first portion of review comments yesterday.
> I'd like to finish looking through the code. I'll post more comments later.
>
> I see, that you called things analogously to partition map exchange.
> I realize, that there is an analogy in used procedures, but I don't really
> like the idea to use the same names for everything.
> The partition map exchange is called this way because it involves an actual
> exchange of information.
> All nodes need to tell each other, which partitions they have, and what
> their states are.
>
> There is no exchange in case of service deployment, so I would skip the
> "exchange" part.
> And *single message ->* *full message* look more like *request -> response*
> in case of services.
>
> Suppose we abandon the PME procedure and move to something else.
> Then *ServiceDeploymentExchange* name won't make sense.
> And I don't want to be in a situation, when I say to my colleague a word
> "exchange",
> and get "which one?" in return.
> So, I'm for the meaningful names rather than analogous to something else.
>
> What do you think?
>
> Denis
>
> вт, 20 нояб. 2018 г. в 12:09, Vyacheslav Daradur :
>
> > Denis, Yakov have you had a chance to review the solution?
> >
> > Igniters, we need to define a list of reviewers, otherwise no end in
> sign.
> >
> > I'm ready to continue work on the Service Grid, including new features
> > like hot-redeployment and versioning, also, I have ideas about new
> > tools for monitoring and management which will be useful for our
> > end-users.
> >
> > But for continuing work we need to overcome this first phase.
> >
> > On Tue, Nov 13, 2018 at 1:09 PM Vyacheslav Daradur 
> > wrote:
> > >
> > > Denis, Yakov, feel free to contact me directly in case of questions.
> > Thanks!
> > >
> > > On Sun, Nov 11, 2018 at 10:09 PM Denis Mekhanikov <
> dmekhani...@gmail.com>
> > wrote:
> > > >
> > > > Guys,
> > > >
> > > > I'd like to take a look at the changes before they are merged.
> > > > I'll do my best to finish the review before the end of the upcoming
> > week.
> > > >
> > > > Thanks!
> > > > Denis
> > > >
> > > > сб, 10 нояб. 2018 г. в 14:25, Nikolay Izhikov :
> > > >
> > > > > Hello, Vladimir.
> > > > >
> > > > > I'm agree with you.
> > > > >
> > > > > Can we write the list of reviewers for this feature?
> > > > > Without a date or similar.
> > > > > Just a list of experts who should review this feature.
> > > > >
> > > > > В Сб, 10/11/2018 в 14:01 +0300, Vladimir Ozerov пишет:
> > > > > > Igniters,
> > > > > >
> > > > > > This is very huge thing with complex algorithms behind. We should
> > not
> > > > > merge
> > > > > > it to the product unless several additional thorough reviews are
> > ready,
> > > > > > irrespectively of how long will it take. We are about quality,
> not
> > speed.
> > > > > >
> > > > > > сб, 10 нояб. 2018 г. в 1:30, Denis Magda :
> > > > > >
> > > > > > > Vyacheslav,
> > > > > > >
> > > > > > > What are the cases when the service can be redeployed?
> Affinity,
> > > > > failure,
> > > > > > > etc., right. It would be good to list all the cases on the wiki
> > and
> > > > > then
> > > > > > > our tech writers will get everything documented.
> > > > > > >
> > > > > > > --
> > > > > > > Denis
> > > > > > >
> > > > > > > On Thu, Nov 8, 2018 at 11:06 PM Vyacheslav Daradur <
> > > > > daradu...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Denis,
> > > > > > > >
> > > > > > > > Services reassignment process takes into account previous
> > assignments
> > > > > > > > to avoid redundant redeployments.
> > > > > > > > So, in the described case, ServiceA won't be moved from node1
> > to
> > > > > node2.
> > > > > > > > On Fri, Nov 9, 2018 at 4:41 AM Denis Magda <
> dma...@apache.org>
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > Vyacheslav,
> > > > > > > > >
> > > > > > > > > First of all, thanks for archiving this milestone and
> > rolling out
> > > > > these
> > > > > > > >
> > > > > > > > new
> > > > > > > > > capabilities.
> > > > > > > > >
> > > > > > > > > Speaking of the topology change events [1], does the new
> > > > > architecture
> > > > > > > >
> > > > > > > > avoid
> > > > > > > > > a running service redeployment when a new node joins? For
> > instance,
> > > > > > >
> > > > > > > let's
> > > > > > > > > say I have ServiceA running node1, then node2 joins and I
> > don't
> > > > > want
> > > > > > >
> > > > > > > the
> > > > > > > > > service to be redeployed to any other node.
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584#ServiceGridredesign.Phase1.Implementationdetails.-Topologychange
> > > > > > > > >
> > > > > > > > > --
> > 

[jira] [Created] (IGNITE-10356) JDBC thin driver returns wrong data type for Date and Decimal SQL type

2018-11-20 Thread Ray (JIRA)
Ray created IGNITE-10356:


 Summary: JDBC thin driver returns wrong data type for Date and 
Decimal SQL type
 Key: IGNITE-10356
 URL: https://issues.apache.org/jira/browse/IGNITE-10356
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.6, 2.7
Reporter: Ray
Assignee: Ray
 Fix For: 2.8


JDBC thin driver will return wrong metadata for column type when user creates a 
table with Date and Decimal type.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Ignite web console on kubernetes with ingress

2018-11-20 Thread vbm
Hi,

I was trying to use Ignite web console on Kubernetes using the docker image
at 

For external access, I am using ingress kubernetes resource. I am able to
access Web console UI using the default href path (/).

But with non default href path, the ignite web console page does not load.  
Is there any workaround for this ?
 
Below is the ingress file, I am using:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web-console-ingress
  namespace: default
  annotations:
ingress.kubernetes.io/rewrite-target: "/"
spec:
  rules:
   - http:
   paths:
- backend:
serviceName: ignite-web-console-service
servicePort: 8080
  path: /ignite-web-console-ui


Regards,
Vishwas



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/


[GitHub] ignite pull request #5453: IGNITE-10355 Tx rollback failure on put operation...

2018-11-20 Thread x-kreator
GitHub user x-kreator opened a pull request:

https://github.com/apache/ignite/pull/5453

IGNITE-10355 Tx rollback failure on put operations with caches whose …

…topology fails validation

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/x-kreator/ignite ignite-10355

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5453.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5453


commit 1cded62c5ff245512cd904889a4ab2e7d85eedd9
Author: Dmitriy Sorokin 
Date:   2018-11-20T15:45:12Z

IGNITE-10355 Tx rollback failure on put operations with caches whose 
topology fails validation




---


Re: [ANNOUNCE] Welcome Igor Seliverstov as a new committer

2018-11-20 Thread Alexey Zinoviev
Great news, Igor, hope to use this great feature in ML module!

вт, 20 нояб. 2018 г. в 11:08, Павлухин Иван :

> Igor my congratulations! Keep going!
>
> 2018-11-20 9:01 GMT+01:00, Vladimir Ozerov :
> > The Apache Ignite Project Management Committee (PMC) has invited Igor
> > Seliverstov to become a new committer and are happy to announce that he
> has
> > accepted.
> >
> > Igor contributed a lot to "Transactional SQL" feature.
> >
> > Please join me in welcoming Igor and congratulating him on his new role
> in
> > the Apache Ignite Community.
> >
> > Good luck, Igor!
> >
> > Vladimir.
> >
>
>
> --
> Best regards,
> Ivan Pavlukhin
>


[jira] [Created] (IGNITE-10355) Tx rollback failure on put operations with caches whose topology fails validation.

2018-11-20 Thread Dmitriy Sorokin (JIRA)
Dmitriy Sorokin created IGNITE-10355:


 Summary: Tx rollback failure on put operations with caches whose 
topology fails validation.
 Key: IGNITE-10355
 URL: https://issues.apache.org/jira/browse/IGNITE-10355
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.6, 2.7
Reporter: Dmitriy Sorokin
Assignee: Dmitriy Sorokin


The issue may occur as stacktrace below:

{noformat}
[2018-10-12 18:47:28,351][ERROR][test-runner-#1][GridDhtColocatedCache] 
 Failed to rollback transaction (cache may contain stale locks): 
GridNearTxLocal[xid=95ce5f86661--08fd-9fc9--0002, 
xidVersion=GridCacheVersion [topVer=150839241, order=1539359239257, 
nodeOrder=2], concurrency=OPTIMISTIC, isolation=READ_COMMITTED, 
state=ROLLED_BACK, invalidate=false, rollbackOnly=true, 
nodeId=108e38a0-64e2-4da6-ab06-6db7e6ae9001, timeout=0, duration=0, label=null]
class org.apache.ignite.IgniteCheckedException: Failed to rollback transaction: 
GridNearTxLocal[xid=95ce5f86661--08fd-9fc9--0002, 
xidVersion=GridCacheVersion [topVer=150839241, order=1539359239257, 
nodeOrder=2], concurrency=OPTIMISTIC, isolation=READ_COMMITTED, 
state=ROLLED_BACK, invalidate=false, rollbackOnly=true, 
nodeId=108e38a0-64e2-4da6-ab06-6db7e6ae9001, timeout=0, duration=0, label=null]
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.doFinish(GridNearTxFinishFuture.java:514)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:425)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$27.apply(GridNearTxLocal.java:3962)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$27.apply(GridNearTxLocal.java:3950)
 at 
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
 at 
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:355)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.chainFinishFuture(GridNearTxLocal.java:3950)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.rollbackNearTxLocalAsync(GridNearTxLocal.java:3864)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.rollbackNearTxLocalAsync(GridNearTxLocal.java:3833)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$18.applyx(GridNearTxLocal.java:2966)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$18.applyx(GridNearTxLocal.java:2948)
 at 
org.apache.ignite.internal.util.lang.IgniteClosureX.apply(IgniteClosureX.java:38)
 at 
org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:78)
 at 
org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:70)
 at 
org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:30)
 at 
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
 at 
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:355)
 at 
org.apache.ignite.internal.util.future.GridFutureAdapter$ChainFuture.(GridFutureAdapter.java:574)
 at 
org.apache.ignite.internal.util.future.GridFutureAdapter.chain(GridFutureAdapter.java:360)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.optimisticPutFuture(GridNearTxLocal.java:2947)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAsync0(GridNearTxLocal.java:693)
 at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAsync(GridNearTxLocal.java:447)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2470)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2468)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4233)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2468)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2449)
 at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2426)
 at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1089)
 at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:820)
...
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [IMPORTANT] Future of Binary Objects

2018-11-20 Thread Alexey Zinoviev
I'd like @Vyacheslav Daradur approach.

Maybe somebody could have a look at UnsafeRow in Spark
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
UnsafeRow is a concrete InternalRow that represents a mutable internal
raw-memory (and hence unsafe) binary row format.

P.S. If somebody is interested in this apporach, I could share more
information

вт, 20 нояб. 2018 г. в 11:33, Sergi Vladykin :

> I really like Protobuf format. It is probably not what we need for O(1)
> fields access,
> but for compact data representation we can derive lots from there.
>
> Also IMO, restricting field type change is absolutely sane idea.
> The correct way to evolve schema in common case is to add new fields and
> gradually
> deprecate the old ones, if you can skip default/null fields in binary
> format this approach
> will not introduce any noticeable performance/size overhead.
>
> Sergi
>
> вт, 20 нояб. 2018 г. в 11:12, Vyacheslav Daradur :
>
> > I think, one of a possible way to reduce overhead and TCO - SQL Scheme
> > approach.
> >
> > That assumes that metadata will be stored separately from serialized
> > data to reduce size.
> > In this case, the most advantages of Binary Objects like access in
> > O(1) and access without deserialization may be achieved.
> > On Tue, Nov 20, 2018 at 10:56 AM Vladimir Ozerov 
> > wrote:
> > >
> > > Hi Alexey,
> > >
> > > Binary Objects only.
> > >
> > > On Tue, Nov 20, 2018 at 10:50 AM Alexey Zinoviev <
> zaleslaw@gmail.com
> > >
> > > wrote:
> > >
> > > > Do we discuss here Core features only or the roadmap for all
> > components?
> > > >
> > > > вт, 20 нояб. 2018 г. в 10:05, Vladimir Ozerov  >:
> > > >
> > > > > Igniters,
> > > > >
> > > > > It is very likely that Apache Ignite 3.0 will be released next
> year.
> > So
> > > > we
> > > > > need to start thinking about major product improvements. I'd like
> to
> > > > start
> > > > > with binary objects.
> > > > >
> > > > > Currently they are one of the main limiting factors for the
> product.
> > They
> > > > > are fat - 30+ bytes overhead on average, high TCO of Apache Ignite
> > > > > comparing to other vendors. They are slow - not suitable for SQL at
> > all.
> > > > >
> > > > > I would like to ask all of you who worked with binary objects to
> > share
> > > > your
> > > > > feedback and ideas, so that we understand how they should look like
> > in AI
> > > > > 3.0. This is a brain storm - let's accumulate ideas first and
> > minimize
> > > > > critics. Then we will work on ideas in separate topics.
> > > > >
> > > > > 1) Historical background
> > > > >
> > > > > BO were implemented around 2014 (Apache Ignite 1.5) when we started
> > > > working
> > > > > on .NET and CPP clients. During design we had several ideas in
> mind:
> > > > > - ability to read object fields in O(1) without deserialization
> > > > > - interoperabillty between Java, .NET and CPP.
> > > > >
> > > > > Since then a number of other concepts were mixed to the cocktail:
> > > > > - Affinity key fields
> > > > > - Strict typing for existing fields (aka metadata)
> > > > > - Binary Object as storage format
> > > > >
> > > > > 2) My proposals
> > > > >
> > > > > 2.1) Introduce "Data Row Format" interface
> > > > > Binary Objects are terrible candidates for storage. Too fat, too
> > slow.
> > > > > Efficient storage typically has <10 bytes overhead per row (no
> > metadata,
> > > > no
> > > > > length, no hash code, etc), allow supper-fast field access, support
> > > > > different string formats (ASCII, UTF-8, etc), support different
> > temporal
> > > > > types (date, time, timestamp, timestamp with timezone, etc), and
> > store
> > > > > these types as efficiently as possible.
> > > > >
> > > > > What we need is to introduce an interface which will convert a pair
> > of
> > > > > key-value objects into a row. This row will be used to store data
> > and to
> > > > > get fields from it. Care about memory consumption, need SQL and
> > strict
> > > > > schema - use one format. Need flexibility and prefer key-value
> > access -
> > > > use
> > > > > another format which will store binary objects unchanged (current
> > > > > behavior).
> > > > >
> > > > > interface DataRowFormat {
> > > > > DataRow create(Object key, Object value); // primitives or
> binary
> > > > > objects
> > > > > DataRowMetadata metadata();
> > > > > }
> > > > >
> > > > > 2.2) Remove affinity field from metadata
> > > > > Affinity rules are governed by cache, not type. We should remove
> > > > > "affintiyFieldName" from metadata.
> > > > >
> > > > > 2.3) Remove restrictions on changing field type
> > > > > I do not know why we did that in the first place. This restriction
> > > > prevents
> > > > > type evolution and confuses users.
> > > > >
> > > > > 2.4) Use bitmaps for "null" and default values and for fixed-length
> > > > fields,
> > > > > put fixed-length fields before variable-length.
> > > > > Motivation: to 

Re: New PMC Member: Nikolay Izhikov

2018-11-20 Thread Alexey Zinoviev
Happy to hear! Good luck with the closest inegration with Spark!

ср, 21 нояб. 2018 г. в 08:49, Vyacheslav Daradur :

> Great news! Congrats Nikolay!
>
> вт, 20 нояб. 2018 г. в 23:22, Dmitriy Pavlov :
>
> > Hi Nikolay,
> >
> > please accept my congratulations!
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 20 нояб. 2018 г. в 22:54, Denis Magda :
> >
> > > The Project Management Committee (PMC) for Apache Ignite
> > > has invited Nikolay Izhikov to become a PMC member and we are pleased
> > > to announce that he has accepted.
> > >
> > > Being a PMC member enables assistance with the management
> > > and to guide the direction of the project.
> > >
> > > Please welcome Nikolay! Great progress so far and we believe that more
> to
> > > come.
> > >
> > > Denis,
> > > On behalf of Ignite PMC
> > >
> >
> --
> Best Regards, Vyacheslav D.
>


[MTCGA]: new failures in builds [2364043] needs to be handled

2018-11-20 Thread dpavlov . tasks
Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *New stable failure of a flaky test in master 
IgniteWalRecoveryWithCompactionTest.testBinaryRecoverBeforePMEWhenMiddleCheckpoint
 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-2339377680760862233=%3Cdefault%3E=testDetails

 *New stable failure of a flaky test in master 
IgniteWalRecoveryTest.testBinaryRecoverBeforePMEWhenMiddleCheckpoint 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=250837897303695090=%3Cdefault%3E=testDetails
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 09:38:00 21-11-2018 


Re: New PMC Member: Nikolay Izhikov

2018-11-20 Thread Vyacheslav Daradur
Great news! Congrats Nikolay!

вт, 20 нояб. 2018 г. в 23:22, Dmitriy Pavlov :

> Hi Nikolay,
>
> please accept my congratulations!
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 20 нояб. 2018 г. в 22:54, Denis Magda :
>
> > The Project Management Committee (PMC) for Apache Ignite
> > has invited Nikolay Izhikov to become a PMC member and we are pleased
> > to announce that he has accepted.
> >
> > Being a PMC member enables assistance with the management
> > and to guide the direction of the project.
> >
> > Please welcome Nikolay! Great progress so far and we believe that more to
> > come.
> >
> > Denis,
> > On behalf of Ignite PMC
> >
>
-- 
Best Regards, Vyacheslav D.


[jira] [Created] (IGNITE-10354) Failing client node due to not receiving metrics updates

2018-11-20 Thread Roman Guseinov (JIRA)
Roman Guseinov created IGNITE-10354:
---

 Summary: Failing client node due to not receiving metrics updates
 Key: IGNITE-10354
 URL: https://issues.apache.org/jira/browse/IGNITE-10354
 Project: Ignite
  Issue Type: Bug
  Components: clients
Affects Versions: 2.6
Reporter: Roman Guseinov
 Attachments: ClientDisconnectedTest.java

In some cases after the coordinator change, the client node can be failed 
before it can establish a connection to another server from the cluster.

{code:java}
[2018-11-21 12:21:45,769][WARN 
][tcp-disco-msg-worker-#15%server-b%][TestTcpDiscoverySpi] Failing client node 
due to not receiving metrics updates from client node within 
'IgniteConfiguration.clientFailureDetectionTimeout' (consider increasing 
configuration property) [timeout=1, node=TcpDiscoveryNode 
[id=dc739711-f685-45e8-9017-1f91b1d86c8c, addrs=[0:0:0:0:0:0:0:1, 10.0.75.1, 
127.0.0.1, 192.168.1.51, 192.168.192.1], sockAddrs=[/0:0:0:0:0:0:0:1:0, 
LAPTOP-6FN8RAOS/10.0.75.1:0, /127.0.0.1:0, /192.168.192.1:0, /192.168.1.51:0], 
discPort=0, order=2, intOrder=2, lastExchangeTime=1542774105666, loc=false, 
ver=2.4.0#20180830-sha1:345c0a7c, isClient=true]]
[2018-11-21 12:21:45,791][INFO 
][tcp-client-disco-msg-worker-#10%client%][TestTcpDiscoverySpi] Client node 
disconnected from cluster, will try to reconnect with new id 
[newId=46812956-2fc4-4b74-9909-d523a547ba0e, 
prevId=dc739711-f685-45e8-9017-1f91b1d86c8c, locNode=TcpDiscoveryNode 
[id=dc739711-f685-45e8-9017-1f91b1d86c8c, addrs=[0:0:0:0:0:0:0:1, 10.0.75.1, 
127.0.0.1, 192.168.1.51, 192.168.192.1], sockAddrs=[/0:0:0:0:0:0:0:1:0, 
LAPTOP-6FN8RAOS/10.0.75.1:0, /127.0.0.1:0, /192.168.192.1:0, /192.168.1.51:0], 
discPort=0, order=2, intOrder=0, lastExchangeTime=1542774104031, loc=true, 
ver=2.4.0#20180830-sha1:345c0a7c, isClient=true]]
{code}

It looks like a race condition.

Steps to reproduce:

1. Start server A.
2. Start client.
3. Start server B.
4. Stop server A.

If add Thread.sleep(1) between (3) and (4) then the client node won't be 
disconnected from the cluster.

Reproducer is attached [^ClientDisconnectedTest.java].




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[MTCGA]: new failures in builds [2364039] needs to be handled

2018-11-20 Thread dpavlov . tasks
Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *New stable failure of a flaky test in master 
PdsWithTtlCompatibilityTest.testNodeStartByOldVersionPersistenceData_2_1 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-8078436609482188163=%3Cdefault%3E=testDetails
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 08:23:11 21-11-2018 


[MTCGA]: new failures in builds [2364187] needs to be handled

2018-11-20 Thread dpavlov . tasks
Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *New Critical Failure in master Compute (Affinity Run) 
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_ComputeAffinityRun=%3Cdefault%3E=buildTypeStatusDiv
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 07:08:11 21-11-2018 


Re: proposed realization KILL QUERY command

2018-11-20 Thread Denis Magda
Folks,

The decimal syntax is really odd - KILL QUERY '[node_order].[query_counter]'

Confusing, let's use a space to separate parameters.

Also, what if I want to halt a specific query with certain ID? Don't know
the node number, just know that the query is distributed and runs across
several machines. Sounds like the syntax still should consider
[node_order/id] as an optional parameter.

Probably, if you explain to me how an end user will use this command from
the very beginning (how do I look for a query id and node id, etc) then the
things get clearer.

--
Denis

On Tue, Nov 20, 2018 at 1:03 AM Юрий  wrote:

> Hi Vladimir,
>
> Thanks for your suggestion to use MANAGEMENT_POOL for processing
> cancellation requests.
>
> About your questions.
> 1) I'm going to implements SQL view to provide list of running queries. The
> SQL VIEW has been a little bit discussed earlier. Proposed name is
> *running_queries* with following columns: query_id, node_id, sql,
> schema_name, connection_id, duration. Currently most of the information can
> be  retrieved through cache API, however it doesn't matter, any case we
> need to expose SQL VIEW. Seem's you are right - the part should be
> implemented firstly.
> 2) Fully agree that we need to support all kind of SQL queries
> (SLECT/DML/DDL, transactional, non transnational, local, distributed). I
> definitely sure that it will possible for all of above, however I'm not
> sure about DDL - need to investigate it deeper. Also need to understand
> that canceled DML operation can lead to partially updated data for non
> transational caches.
>
>
>
> пн, 19 нояб. 2018 г. в 19:17, Vladimir Ozerov :
>
> > Hi Yuriy,
> >
> > I think we can use MANAGEMENT_POOL for this. It is already used for some
> > internal Ignite tasks, and it appears to be a good candidate to process
> > cancel requests.
> >
> > But there are several things which are not clear enough for me at the
> > moment:
> > 1) How user is going to get the list of running queries in the first
> place?
> > Do we already have any SQL commands/views to get this information?
> > 2) We need to ensure that KILL command will be processed properly by all
> > kinds of SQL queries - SELECT/DML/DDL, non-transactional or
> transactional,
> > local queries and distributed queries. Will we be able to support all
> these
> > modes?
> >
> > Vladimir.
> >
> > On Mon, Nov 19, 2018 at 6:37 PM Юрий 
> wrote:
> >
> > > Hi Igniters,
> > >
> > > Earlier we agreed about syntax KILL QUERY
> '[node_order].[query_counter]',
> > > e.g. KILL QUERY '25.123' for single query  or KILL QUERY '25.*' for all
> > > queries on the node. Which is part of IEP-29
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-29%3A+SQL+management+and+monitoring
> > > >
> > > .
> > >
> > > Now I want to discuss internal realization of KILL query feature.
> > >
> > > My current vision is following:
> > > After parsing, Ignite create KILL query command with two parameters:
> > > nodeOrderId, nodeQryId. To determine that need to kill all queries on a
> > > node we can use negative value of query id, due to qry id always have
> > > positive values.
> > > The command process at IgniteH2Indexing as native command.
> > > By nodeOrderId we find node which initial for the query and send to the
> > > node new GridQueryKillRequest with nodeQryId to TOPIC_QUERY with not
> > QUERY
> > > POOL executor.
> > > At GridReduceQueryExecutor we add support of processing new
> > > GridQueryKillRequest
> > > which just run already exists cancelQueries method with given qryId or
> > with
> > > all qryIds which currently running at the node in case at initial KILL
> > > QUERY parameters used star symbol.
> > >
> > > I have a doubt which of thread pool we should use to process
> > > GridQueryKillRequest.
> > > My opinion it shouldn't be QUERY pool, due to the pool can be fully
> used
> > by
> > > executing queries, it such case we can't cancel query immediately. May
> we
> > > use one of already existed pool or create new one? Or may be I'm
> mistaken
> > > and it should use QUERY pool.
> > >
> > > What do you think about proposed plan of implementation?
> > >
> > > And please give comments about which of thread pool will be better to
> use
> > > for kill query requests. It's small, but really important part of the
> > > realization.
> > >
> > >
> > > Thanks.
> > >
> > >
> > > --
> > > Живи с улыбкой! :D
> > >
> >
>
>
> --
> Живи с улыбкой! :D
>


Re: Request for Contributor Permissions

2018-11-20 Thread Dmitriy Pavlov
Hi Jonathan,

I've added your account to the list of contributors, so you can now assign
an issue to yourself.

Welcome to the Apache Ignite Community!

Sincerely,
Dmitriy Pavlov

вт, 20 нояб. 2018 г. в 23:53, Jonathan Camargo :

> Hi,
>
> I would like to participate and to take the Jira IGNITE-10353
> 
>
> Thanks
>
> --
> Jonathan Giovanny Camargo Sanabria
>


Re: [jira] [Created] (IGNITE-10353) Spring Add Update/Delete support for Spring Data

2018-11-20 Thread Jonathan Camargo
As a proposed design it will continue with the current way the lib is
working, which is for the methods with the information about the query
(i.e. findByIdOrderById() and in this case deleteByLastname()) it will
create the query working over the PartTree, which is provided by the Spring
framework, it also requires to validate when a @Query("") is being send,
and to be able to validate if that query is valid and therefore use the
SqlQueryFields from the Ignite core, or rather work with the method name,
if non is valid, then throw an IgniteException.

El mar., 20 nov. 2018 a las 15:53, Jonathan Camargo (JIRA) ()
escribió:

> Jonathan Camargo created IGNITE-10353:
> -
>
>  Summary: Spring Add Update/Delete support for Spring Data
>  Key: IGNITE-10353
>  URL: https://issues.apache.org/jira/browse/IGNITE-10353
>  Project: Ignite
>   Issue Type: Improvement
>   Components: spring
> Reporter: Jonathan Camargo
>
>
> In Spring Data 2.0 module, add the update/delete functionalities to work
> with the cache, this should be possible even using the method name in the
> IgniteRepository interface or by a @Query notation writing the SQL query
> following the ignite docs for it.
>
> The specification for the remaining methods for the Repository interface
> are mentioned on the Spring documentation.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>


-- 
Jonathan Giovanny Camargo Sanabria


Request for Contributor Permissions

2018-11-20 Thread Jonathan Camargo
Hi,

I would like to participate and to take the Jira IGNITE-10353


Thanks

-- 
Jonathan Giovanny Camargo Sanabria


[jira] [Created] (IGNITE-10353) Spring Add Update/Delete support for Spring Data

2018-11-20 Thread Jonathan Camargo (JIRA)
Jonathan Camargo created IGNITE-10353:
-

 Summary: Spring Add Update/Delete support for Spring Data
 Key: IGNITE-10353
 URL: https://issues.apache.org/jira/browse/IGNITE-10353
 Project: Ignite
  Issue Type: Improvement
  Components: spring
Reporter: Jonathan Camargo


In Spring Data 2.0 module, add the update/delete functionalities to work with 
the cache, this should be possible even using the method name in the 
IgniteRepository interface or by a @Query notation writing the SQL query 
following the ignite docs for it.

The specification for the remaining methods for the Repository interface are 
mentioned on the Spring documentation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: New PMC Member: Nikolay Izhikov

2018-11-20 Thread Dmitriy Pavlov
Hi Nikolay,

please accept my congratulations!

Sincerely,
Dmitriy Pavlov

вт, 20 нояб. 2018 г. в 22:54, Denis Magda :

> The Project Management Committee (PMC) for Apache Ignite
> has invited Nikolay Izhikov to become a PMC member and we are pleased
> to announce that he has accepted.
>
> Being a PMC member enables assistance with the management
> and to guide the direction of the project.
>
> Please welcome Nikolay! Great progress so far and we believe that more to
> come.
>
> Denis,
> On behalf of Ignite PMC
>


Re: Time to remove automated messages from the devlist?

2018-11-20 Thread Dmitriy Pavlov
One more thing I want to emphasize here. We can't just remove messages, it
_must_ be sent to some list, which is why we need some additional list,
e.g. notifications@ for this.

So only one option to proceed here is to run a formal vote on list creation
and redirection of github/gitbox messages to a new list.

пн, 19 нояб. 2018 г. в 18:23, Dmitriy Pavlov :

> Denis, we need because contributors do not announce their
> intent/designs/etc manually. It is the best way ever? No, of course.
>
> We have consensus on PR removal, so let's do it and see results.
>
> пн, 19 нояб. 2018 г. в 18:11, Denis Mekhanikov :
>
>> Dmitriy,
>>
>> If a person wants to track all new tickets, then he may go to JIRA, create
>> a filter for Ignite tickets
>> and subscribe to it. JIRA has a pretty flexible configuration of filters
>> and subscriptions, so you can
>> specify exactly what issues you are interested in, and how often you want
>> to receive these emails.
>> This is much more convenient and more flexible than filtering emails from
>> a
>> bot.
>>
>> So, most people ignore JIRA messages, and the ones who want to track new
>> tickets,
>> may go to JIRA and configure their own filters. I don't see, why we need
>> to
>> keep the forwarding to dev list.
>>
>> Denis
>>
>> пт, 16 нояб. 2018 г. в 22:30, Павлухин Иван :
>>
>> > Hi,
>> >
>> > Can we collect opinions about keeping messages of mentioned types on
>> > dev list? From my side (+ means keeping on dev list):
>> > TC bot +
>> > Jira -
>> > GitHub -
>> > пт, 16 нояб. 2018 г. в 22:25, Dmitriy Pavlov :
>> > >
>> > > Importance is hardly definable and it is not possible that importance
>> is
>> > > equal for everyone. You can say about other human emails it is not
>> > > important if some product area is not interesting for you. So I can
>> only
>> > > understand the terms: email needs action/does not need action.
>> > >
>> > > If some contributor never reacted to JIRA notification he or she may
>> > think
>> > > it is not important. But even we have a majority of contributors who
>> > > ignores JIRA, it does not mean it is a right decision to switch it
>> off.
>> > We
>> > > don't play in a democracy, hopefully.
>> > >
>> > > My suggestion now: keep showing an excellent example of human-human
>> > > interaction, announces, etc from all Ignite veterans (especially,
>> PMCs),
>> > so
>> > > newcomers can use the same approach.
>> > >
>> > > If PRs removal to notifications@ will show a positive tendency in
>> > > human-human interaction, I can easily agree with the second step. Only
>> > > practice is truth criteria.
>> > >
>> > > пт, 16 нояб. 2018 г. в 22:08, Vladimir Ozerov :
>> > >
>> > > > We want important emails to be easily observable. This is the only
>> > goal.
>> > > >
>> > > > пт, 16 нояб. 2018 г. в 21:51, Dmitriy Pavlov :
>> > > >
>> > > > > I suggest to think in another paradigm, let's not classify emails
>> to
>> > be
>> > > > > automatically issued or not, lets separate emails to other
>> classes: a
>> > > > > needed action from humans or not needed.
>> > > > >
>> > > > > If you don't have any interest in a change announced by JIRA issue
>> > > > created
>> > > > > email, you can just skip. If you can help with comments, review,
>> > etc, you
>> > > > > can become watcher or comment ticket, you can also point to
>> > duplicate.
>> > > > >
>> > > > > In that paradigm,
>> > > > > A) PR is perfectly ok to be redirected to notifications@ .- PR
>> > creation
>> > > > > does not require any action from anyone.
>> > > > > B) JIRA - I'm not sure (maybe as a second step, if we will see
>> > > > contributors
>> > > > > will write about important tickets). And instead we can discuss
>> Open
>> > ->
>> > > > > Patch available transition, as a reviewer needed.
>> > > > > C) TC Bot - I'm sure - should never be redirected. Hopefully, it
>> > will not
>> > > > > generate any alerts.
>> > > > >
>> > > > > I hardly understand goal: is our target metric - message count to
>> be
>> > as
>> > > > > less as possible? (extreme - 0 emails, let's not write here at
>> all,
>> > we
>> > > > can
>> > > > > get 0). Who can explain what do we want from redirection?
>> > > > >
>> > > > >
>> > > > > пт, 16 нояб. 2018 г. в 16:28, Sergi Vladykin <
>> > sergi.vlady...@gmail.com>:
>> > > > >
>> > > > > > I also would like to separate all the automated stuff.
>> > > > > >
>> > > > > > Sergi
>> > > > > >
>> > > > > > пт, 16 нояб. 2018 г. в 13:58, Павлухин Иван <
>> vololo...@gmail.com>:
>> > > > > >
>> > > > > > > Oleg,
>> > > > > > >
>> > > > > > > I join to Dmitriy. I found your summary quite interesting.
>> > > > > > > пт, 16 нояб. 2018 г. в 13:12, Dmitriy Pavlov <
>> dpav...@apache.org
>> > >:
>> > > > > > > >
>> > > > > > > > Oleg,
>> > > > > > > >
>> > > > > > > > excellent research! It allows me to avoid bothering
>> community
>> > > > > > developers
>> > > > > > > > once again.
>> > > > > > > >
>> > > > > > > > Thank you for your efforts and for contributing to this
>> > discussion.
>> > > > > 

[GitHub] ignite pull request #5452: Ignite 2.4.12 p2

2018-11-20 Thread ezhuravl
GitHub user ezhuravl opened a pull request:

https://github.com/apache/ignite/pull/5452

Ignite 2.4.12 p2



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4.12-p2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5452.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5452


commit cb89494be26be1b0e397e06f7eaeb6419aec1828
Author: Alexey Kuznetsov 
Date:   2018-03-02T09:35:26Z

IGNITE-7803 REST: Implemented possibility to get values from cache inserted 
via API or SQL.

(cherry picked from commit a84018b)

commit 149802a3be82ba1f88770dbff75d74b892bb0a94
Author: Alexey Kuznetsov 
Date:   2018-03-07T17:17:20Z

IGNITE-7803 Fixed regression in tests.

(cherry picked from commit 90c0af8)

commit 5b5571adc7282a29cdecd8ab2e945a446ea709fc
Author: Alexey Kuznetsov 
Date:   2018-03-23T02:55:13Z

IGNITE-8002 REST: Added support of new authentication via new API.

(cherry picked from commit 921f0cf)

commit 128fdfcd874286f4a358546c0f43b6ef082a064b
Author: devozerov 
Date:   2018-04-02T10:11:16Z

IGNITE-7884: Sample data set for JDBC. This closes #3702.

commit 247a9ef5a68de198106a866b277f1bee596c3633
Author: devozerov 
Date:   2018-04-02T10:16:00Z

Merge branch 'ignite-2.4.4' into ignite-2.4-master

commit 3949df02128e3003c409e0169ad0c80948ba134c
Author: Aleksey Plekhanov 
Date:   2018-02-13T13:43:49Z

IGNITE-5265 Eviction Rate memory metric to be implemented

Signed-off-by: Anton Vinogradov 

commit 7bc6c1ad3bacaf5e354d2710931eb398c346c06c
Author: Tim Onyschak 
Date:   2018-03-31T12:58:25Z

IGNITE-7090 Semaphore Stuck when no acquirers to assign permit - Fixes 
#3443.

Signed-off-by: dspavlov 

(cherry picked from commit 3fc5d57)

commit c854ad86f4c64260171085c8c7cce97ba1ad2530
Author: Pavel Kovalenko 
Date:   2018-02-22T09:02:31Z

IGNITE-7749 Fixed testDiscoCacheReuseOnNodeJoin test. - Fixes #3540.

Signed-off-by: Alexey Goncharuk 

commit b2e94b36bddebbf7b3ff40631e099939d16926d2
Author: Alexey Goncharuk 
Date:   2018-02-13T15:53:23Z

IGNITE-7692 Corrected test to not fail when SQL is executed on backup

commit 852425d4170ed1871c79f5ac26bfb3d03cc36a6f
Author: Алексей Стельмак 
Date:   2018-04-06T15:28:22Z

IGNITE-8049 Limit the number of operation cycles in B+Tree - Fixes #3769.

Signed-off-by: dpavlov 
(cherry picked from commit e491f10)

commit 42f529f0a04ce22786bb4a23032a64f93e214233
Author: Alexey Kuznetsov 
Date:   2018-04-09T02:25:50Z

IGNITE-8159 control.sh: Fixed NPE on adding nodes on empty baseline and not 
active cluster.

(cherry picked from commit 834869c)

commit b5f180838246f895d36846ea707790c1ff7fe70a
Author: Stanislav Lukyanov 
Date:   2018-04-09T11:33:13Z

IGNITE-7904: Changed IgniteUtils::cast not to trim exception chains. This 
closes #3683.

(cherry picked from commit 3a4f23b)

commit 1ce8e1a8fd48469073592e2fb77e2881a168a219
Author: Roman Guseinov 
Date:   2018-04-09T11:45:44Z

IGNITE-7944: Disconnected client node tries to send JOB_CANCEL message. 
Applied fix:
- Skip sending message if client disconnected;
- Throw IgniteCheckedException if a client node is disconnected and 
communication client is null.
This closes #3737.

(cherry picked from commit d70477b)

commit 840e193b3e604e8b0d90be5fac16cf11d8c832e6
Author: Ilya Lantukh 
Date:   2018-04-06T10:49:10Z

ignite-8087 : Backported ignite-8018.

(cherry picked from commit 175edf3)

commit 94e857c94ce5e7998fc39e38deee69f389ddbc5b
Author: Alexey Goncharuk 
Date:   2018-04-10T17:33:47Z

IGNITE-6430 Complete failing test early

commit 770f7469d4b86ce9af61e9e43c7262d86ee05b54
Author: Eduard Shangareev 
Date:   2018-04-06T16:22:07Z

IGNITE-8114 Add fail recovery mechanism to tracking pages

commit 549d6e3de86a22697577f77a2ebab3d0dca7338d
Author: Alexey Kukushkin 
Date:   2018-04-11T13:29:07Z

IGNITE-8221: Security for thin clients.

commit 94c0f56ef78e6c61f0e753c8aa0185c611630d45
Author: devozerov 
Date:   2018-04-11T13:44:33Z

IGNITE-8148: JDBC thin: semicolon as delimiter for properties. This closes 
#3794.

commit d99a50ccd6923aa48da78955207fe747b287ea81
Author: mcherkasov 
Date:   2018-04-11T00:23:29Z

IGNITE-8153 Nodes fail to connect each other when SSL is enabled - Fixes 
#3773.

Signed-off-by: Valentin Kulichenko 

commit 03285e953d005a18fb88a35d21701100f58f7a29
Author: devozerov 
Date:   2018-04-12T07:37:36Z

IGNITE-8042: .NET thin client: authentication support. This closes #3790.

commit 8e740164e8a0102a94f408d72b41f73df675cfb1
Author: Ilya Kasnacheev 
Date:   2018-04-05T09:55:36Z

IGNITE-7712: SQL: system property to force lazy query execution.


New PMC Member: Nikolay Izhikov

2018-11-20 Thread Denis Magda
The Project Management Committee (PMC) for Apache Ignite
has invited Nikolay Izhikov to become a PMC member and we are pleased
to announce that he has accepted.

Being a PMC member enables assistance with the management
and to guide the direction of the project.

Please welcome Nikolay! Great progress so far and we believe that more to
come.

Denis,
On behalf of Ignite PMC


Re: TcpCommunicationSpi extension to ignore docker bridge network

2018-11-20 Thread David Harvey
What we prototyped was configuring via spring the list of IPs to ignore,
because a given installation seemed to have a constant address for the
bridge network, and this approach was reliable, once you know the bridge
IPs.   It is also a more general solution.

When the container starts, you get a list of IP addresses from the kernel.
 At that point it is impossible to know from inside the container which of
those addresses can be used by other ignite nodes, at least without
external information.   For exampe, ifI have ignite running on an AWS
instance that has an internal and external address, it is impossible to
know which address will be able to reach the other nodes, unless you are
told.   So perhaps we should have used a list of ranges rather than a list
in our prototype.

For the docker sub-case where all the nodes seem to get the same useless
address, I would think we can ignore IP address/port pairs that are the
current node is also advertising.That does not generalize to other
cases were the kernel provides unusable addresses.I didn't quite
understand why if we try to connect to port we are advertising, this would
need to timeout, rather than getting immediately rejected, unless Ignite
has explicit code to do detected and ignore a self message.   But if there
is a IP:port pair that the current node is claiming as an endpoint, it
should not try to use that IP:port to connect to other nodes.

On Tue, Nov 20, 2018 at 2:27 PM David Harvey  wrote:

> What we prototyped was configuring via spring the list of IPs to ignore,
> because a given installation seemed to have a constant address for the
> bridge network, and this approach was reliable, once you know the bridge
> IPs.
>
> When the container starts, you get a list of IP addresses from the
> kernel.   At that point it is impossible to know from inside the container
> which of those addresses can be used by other ignite nodes, at least
> without external information.   Similarly, if I have an AWS instance
>
> I am wondering
>
>
>
> On Tue, Nov 20, 2018 at 2:08 PM Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
>> Hi David,
>>
>> This is something we have also encountered recently and I was wondering
>> how
>> this can be mitigated in a general case. Do you know if an application can
>> detect that it is being run in a docker container and add the
>> corresponding
>> list of bridge IPs automatically on start? If so, I this we can add this
>> to
>> the Ignite so that it works out of the box.
>>
>> --AG
>>
>>
>> вт, 20 нояб. 2018 г. в 19:58, David Harvey :
>>
>> > We see some annoying behavior with S3 discovery because Ignite will
>> push to
>> > the discovery S3 bucket the IP address of the local docker bridge
>> network
>> > (172.17.0.1) in our case.   Basically, each node when coming online
>> tries
>> > that address first, and has to go through a network timeout to recover.
>> >
>> > To address this, have prototyped a simple extension to
>> TcpCommunicationSpi
>> > to allow configuration of a list of IP addresses that should be
>> completely
>> > ignored, and will create a ticket and generate a pull request for it.
>> >
>> > If there is a better approach, please let us know.
>> >
>> > Thanks
>> > Dave Harvey
>> >
>>
>


Re: TcpCommunicationSpi extension to ignore docker bridge network

2018-11-20 Thread David Harvey
What we prototyped was configuring via spring the list of IPs to ignore,
because a given installation seemed to have a constant address for the
bridge network, and this approach was reliable, once you know the bridge
IPs.

When the container starts, you get a list of IP addresses from the kernel.
 At that point it is impossible to know from inside the container which of
those addresses can be used by other ignite nodes, at least without
external information.   Similarly, if I have an AWS instance

I am wondering



On Tue, Nov 20, 2018 at 2:08 PM Alexey Goncharuk 
wrote:

> Hi David,
>
> This is something we have also encountered recently and I was wondering how
> this can be mitigated in a general case. Do you know if an application can
> detect that it is being run in a docker container and add the corresponding
> list of bridge IPs automatically on start? If so, I this we can add this to
> the Ignite so that it works out of the box.
>
> --AG
>
>
> вт, 20 нояб. 2018 г. в 19:58, David Harvey :
>
> > We see some annoying behavior with S3 discovery because Ignite will push
> to
> > the discovery S3 bucket the IP address of the local docker bridge network
> > (172.17.0.1) in our case.   Basically, each node when coming online tries
> > that address first, and has to go through a network timeout to recover.
> >
> > To address this, have prototyped a simple extension to
> TcpCommunicationSpi
> > to allow configuration of a list of IP addresses that should be
> completely
> > ignored, and will create a ticket and generate a pull request for it.
> >
> > If there is a better approach, please let us know.
> >
> > Thanks
> > Dave Harvey
> >
>


Re: Spring Update and delete queries

2018-11-20 Thread Denis Magda
Excellent,

Please get to know basics about Ignite's contribution process, create a
ticket in JIRA and propose a design here.

--
Denis



On Tue, Nov 20, 2018 at 11:02 AM Jonathan Camargo 
wrote:

> Hi,
>
> Sure, I'm ready to start working on the DELETE/UPDATE.
>
> El mar., 20 nov. 2018 a las 13:10, Denis Magda ()
> escribió:
>
> > Hello Jonathan,
> >
> > Thanks for the feedback. Are you ready to work on missing DELETE/UPDATE
> > capabilities? Our community experts can guide you and do reviews.
> >
> > --
> > Denis
> >
> > On Tue, Nov 20, 2018 at 7:34 AM Jonathan Camargo 
> > wrote:
> >
> > > Hi community,
> > >
> > > First let me introduce myself, My name is Jonathan Camargo, I'm a Java
> > > developer at a software outsourcing company, I'm currently working for
> a
> > > legacy project.
> > >
> > > I'm currently learning new technologies and concepts such as IMDG, I
> have
> > > been reading concepts and code about that mainly because I have find
> this
> > > concept fascinating.
> > >
> > > On the test project I'm working on I am using Spring Boot 2, Ignite
> > Spring
> > > Data 2.0 and a H2 DB (It could be any other but seems easier this way)
> > and
> > > aside from the already documented issues I have find that the
> > > IgniteRepository does not support the Update and Delete queries, on the
> > > code even is throwing a controlled exception:
> > >
> > > if (parts.isDelete())
> > > throw new UnsupportedOperationException("DELETE clause is
> not
> > > supported now.");
> > > else {
> > > sql.append("SELECT ");
> > >
> > > if (parts.isDistinct())
> > > throw new UnsupportedOperationException("DISTINCT
> clause
> > in
> > > not supported.");
> > >
> > > And I would like to point that the lib is validating if if is going to
> > > generate the query or if it is going to use the @Query SQL that is
> being
> > > sent, and that SQL is executed through the SqlFieldsQuery of the core
> > lib.
> > >
> > > So my suggestion would be to add the Update and Delete query to the
> > > validation so those functions will run over the core.
> > >
> > > I would not like to Inject the Ignite object, get or create the cache
> and
> > > run those statements over there since that way the Ignite Repository
> > would
> > > not be needed and I would prefer just to remove the Ignite Spring lib
> and
> > > work directly with the client and the server on other applications.
> > >
> > > If is there any issue I could help with I would be there.
> > >
> > > And I would like to know if is there any filter on Jira over the issues
> > > that I could look.
> > >
> > > Thanks.
> > >
> > > --
> > > Jonathan Giovanny Camargo Sanabria
> > >
> >
>
>
> --
> Jonathan Giovanny Camargo Sanabria
>


Re: TcpCommunicationSpi extension to ignore docker bridge network

2018-11-20 Thread Alexey Goncharuk
Hi David,

This is something we have also encountered recently and I was wondering how
this can be mitigated in a general case. Do you know if an application can
detect that it is being run in a docker container and add the corresponding
list of bridge IPs automatically on start? If so, I this we can add this to
the Ignite so that it works out of the box.

--AG


вт, 20 нояб. 2018 г. в 19:58, David Harvey :

> We see some annoying behavior with S3 discovery because Ignite will push to
> the discovery S3 bucket the IP address of the local docker bridge network
> (172.17.0.1) in our case.   Basically, each node when coming online tries
> that address first, and has to go through a network timeout to recover.
>
> To address this, have prototyped a simple extension to TcpCommunicationSpi
> to allow configuration of a list of IP addresses that should be completely
> ignored, and will create a ticket and generate a pull request for it.
>
> If there is a better approach, please let us know.
>
> Thanks
> Dave Harvey
>


Re: Spring Update and delete queries

2018-11-20 Thread Jonathan Camargo
Hi,

Sure, I'm ready to start working on the DELETE/UPDATE.

El mar., 20 nov. 2018 a las 13:10, Denis Magda ()
escribió:

> Hello Jonathan,
>
> Thanks for the feedback. Are you ready to work on missing DELETE/UPDATE
> capabilities? Our community experts can guide you and do reviews.
>
> --
> Denis
>
> On Tue, Nov 20, 2018 at 7:34 AM Jonathan Camargo 
> wrote:
>
> > Hi community,
> >
> > First let me introduce myself, My name is Jonathan Camargo, I'm a Java
> > developer at a software outsourcing company, I'm currently working for a
> > legacy project.
> >
> > I'm currently learning new technologies and concepts such as IMDG, I have
> > been reading concepts and code about that mainly because I have find this
> > concept fascinating.
> >
> > On the test project I'm working on I am using Spring Boot 2, Ignite
> Spring
> > Data 2.0 and a H2 DB (It could be any other but seems easier this way)
> and
> > aside from the already documented issues I have find that the
> > IgniteRepository does not support the Update and Delete queries, on the
> > code even is throwing a controlled exception:
> >
> > if (parts.isDelete())
> > throw new UnsupportedOperationException("DELETE clause is not
> > supported now.");
> > else {
> > sql.append("SELECT ");
> >
> > if (parts.isDistinct())
> > throw new UnsupportedOperationException("DISTINCT clause
> in
> > not supported.");
> >
> > And I would like to point that the lib is validating if if is going to
> > generate the query or if it is going to use the @Query SQL that is being
> > sent, and that SQL is executed through the SqlFieldsQuery of the core
> lib.
> >
> > So my suggestion would be to add the Update and Delete query to the
> > validation so those functions will run over the core.
> >
> > I would not like to Inject the Ignite object, get or create the cache and
> > run those statements over there since that way the Ignite Repository
> would
> > not be needed and I would prefer just to remove the Ignite Spring lib and
> > work directly with the client and the server on other applications.
> >
> > If is there any issue I could help with I would be there.
> >
> > And I would like to know if is there any filter on Jira over the issues
> > that I could look.
> >
> > Thanks.
> >
> > --
> > Jonathan Giovanny Camargo Sanabria
> >
>


-- 
Jonathan Giovanny Camargo Sanabria


Re: Spring Update and delete queries

2018-11-20 Thread Denis Magda
Hello Jonathan,

Thanks for the feedback. Are you ready to work on missing DELETE/UPDATE
capabilities? Our community experts can guide you and do reviews.

--
Denis

On Tue, Nov 20, 2018 at 7:34 AM Jonathan Camargo 
wrote:

> Hi community,
>
> First let me introduce myself, My name is Jonathan Camargo, I'm a Java
> developer at a software outsourcing company, I'm currently working for a
> legacy project.
>
> I'm currently learning new technologies and concepts such as IMDG, I have
> been reading concepts and code about that mainly because I have find this
> concept fascinating.
>
> On the test project I'm working on I am using Spring Boot 2, Ignite Spring
> Data 2.0 and a H2 DB (It could be any other but seems easier this way) and
> aside from the already documented issues I have find that the
> IgniteRepository does not support the Update and Delete queries, on the
> code even is throwing a controlled exception:
>
> if (parts.isDelete())
> throw new UnsupportedOperationException("DELETE clause is not
> supported now.");
> else {
> sql.append("SELECT ");
>
> if (parts.isDistinct())
> throw new UnsupportedOperationException("DISTINCT clause in
> not supported.");
>
> And I would like to point that the lib is validating if if is going to
> generate the query or if it is going to use the @Query SQL that is being
> sent, and that SQL is executed through the SqlFieldsQuery of the core lib.
>
> So my suggestion would be to add the Update and Delete query to the
> validation so those functions will run over the core.
>
> I would not like to Inject the Ignite object, get or create the cache and
> run those statements over there since that way the Ignite Repository would
> not be needed and I would prefer just to remove the Ignite Spring lib and
> work directly with the client and the server on other applications.
>
> If is there any issue I could help with I would be there.
>
> And I would like to know if is there any filter on Jira over the issues
> that I could look.
>
> Thanks.
>
> --
> Jonathan Giovanny Camargo Sanabria
>


[GitHub] ignite pull request #5451: IGNITE-10352 Fixed mapping of cache get request t...

2018-11-20 Thread agura
GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/5451

IGNITE-10352 Fixed mapping of cache get request to node while partition is 
in MOVING state

…on is in MOVING state

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agura/incubator-ignite ignite-10352

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5451.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5451


commit 9572eab23e1106aaf504df2b41b5ed471b54b4e8
Author: Andrey Gura 
Date:   2018-11-20T17:49:34Z

IGNITE-10352 Fixed mapping of cache get request to node while partition is 
in MOVING state




---


[GitHub] ignite pull request #5046: IGNITE-9959: Set consistent id in tests

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5046


---


[jira] [Created] (IGNITE-10352) Cache get request can be mapped to node while partition in MOVING state

2018-11-20 Thread Andrey Gura (JIRA)
Andrey Gura created IGNITE-10352:


 Summary: Cache get request can be mapped to node while partition 
in MOVING state
 Key: IGNITE-10352
 URL: https://issues.apache.org/jira/browse/IGNITE-10352
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
 Fix For: 2.8


Cache get request can be mapped to node while partition in MOVING state.

It leads to assertions like quoted below and eventually to failure handler 
action.

{noformat}
java.lang.AssertionError: Wrong ready topology version for invalid partitions 
response [topVer=AffinityTopologyVersion [topVer=19, minorTopVer=0], 
req=GridNearSingleGetRequest [futId=1542371256329, key=KeyCacheObjectImpl 
[part=4595, val=com.sbt.cdm.api.model.published.ae.PublishedOperationCondition, 
hasValBytes=true], flags=1, topVer=AffinityTopologyVersion [topVer=19, 
minorTopVer=0], subjId=ea523c3a-4d22-4a3c-b1b0-3137da762021, taskNameHash=0, 
createTtl=-1, accessTtl=-1, mvccSnapshot=null]]
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$6.apply(GridDhtCacheAdapter.java:975)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$6.apply(GridDhtCacheAdapter.java:938)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:355)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processNearSingleGetRequest(GridDhtCacheAdapter.java:938)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$2.apply(GridDhtTransactionalCacheAdapter.java:152)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$2.apply(GridDhtTransactionalCacheAdapter.java:150)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$ArrayListener.onMessage(GridIoManager.java:2349)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:745)
{noformat}

and

{noformat}
java.lang.AssertionError: Got invalid partitions for local node but topology 
version did not change [topVer=AffinityTopologyVersion [topVer=18, 
minorTopVer=0], updTopVer=AffinityTopologyVersion [topVer=18, minorTopVer=0], 
invalidParts=[3032]]
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:254)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.access$600(GridPartitionedSingleGetFuture.java:69)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture$4.run(GridPartitionedSingleGetFuture.java:774)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6887)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #5450: IGNITE-10298 Cover possible deadlock in case of c...

2018-11-20 Thread Jokser
GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/5450

IGNITE-10298 Cover possible deadlock in case of caches start and frequent 
checkpoints



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10298

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5450.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5450


commit efe84b1356bbb89ca059691fbbb112716a6aa85c
Author: Pavel Kovalenko 
Date:   2018-11-20T17:26:52Z

IGNITE-10298 Cover possible deadlock in case of caches start and frequent 
checkpoints.




---


[jira] [Created] (IGNITE-10351) Web Console: add new fields of IgniteConfiguration (sysWorkerBlockedTimeout and checkpointReadLockTimeout) to UI configurator

2018-11-20 Thread Alexey Kuznetsov (JIRA)
Alexey Kuznetsov created IGNITE-10351:
-

 Summary: Web Console: add new fields of IgniteConfiguration 
(sysWorkerBlockedTimeout and checkpointReadLockTimeout) to UI configurator
 Key: IGNITE-10351
 URL: https://issues.apache.org/jira/browse/IGNITE-10351
 Project: Ignite
  Issue Type: Task
  Components: wizards
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 2.8


We need to add new options to UI configurator.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


JdbcColumnMeta

2018-11-20 Thread Peter Borissow
Dear Devs,    I would like to help expand the jdbc resultset metadata available 
when querying a table. Consider the following example:


CREATE TABLE ARTICLE (
    ID BIGINT,
    BODY text,
    LAST_MODIFIED TIMESTAMP with time zone,
    CONSTRAINT PK_ ARTICLE PRIMARY KEY (ID)
 );
After creating the table, I can query the column metadata using the 
java.sql.ResultSetMetaData. Example:

 java.sql.ResultSet rs = stmt.executeQuery("select * from article);
java.sql.ResultSetMetaData rsmd = rs.getMetaData();

However, the resultset /column metadata will return the following types:

- id: long- body: object- last_modified: object
Note that the body and last_modified fields are identified as generic objects 
instead of their formal java types. This is not consistent with other jdbc 
drivers (e.g. postgresql, h2, etc). 

 After a little digging, I found the JdbcColumnMeta(GridQueryFieldMetadata 
info) constructor which calls the JdbcThinUtils class. However, at this point 
the GridQueryFieldMetadata is already mapped to a generic "object" so it is 
impossible to return the correct type from the JdbcThinUtils class.

Next, I decided to see how tables are created in the first place. Specifically, 
the GridSqlQueryParser.parseCreateTable() method. In there, I can see the 
correct column metadata. For example, if I add the following code in the 
parseCreateTable method...
for (Column col : data.columns){
    System.out.println(col.getName() + " (" + col.getType() + ")");
} 

...it will print the following:
ID (5)
BODY (16)
LAST_MODIFIED (24)

I would like to preserve this metadata so that I can see it when using the jdbc 
resultset metadata object. Any suggestions?

Note that the ID column type (5) is preserved. I suspect the other types 
identified in the JdbcThinUtils class are preserved as well.


Thanks in advance,Peter









[jira] [Created] (IGNITE-10350) Document that now it's allowed to stop caches created via SQL/DDL

2018-11-20 Thread Eduard Shangareev (JIRA)
Eduard Shangareev created IGNITE-10350:
--

 Summary: Document that now it's allowed to stop caches created via 
SQL/DDL
 Key: IGNITE-10350
 URL: https://issues.apache.org/jira/browse/IGNITE-10350
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Reporter: Eduard Shangareev


We need to update our documentation that behavior has changed.

https://issues.apache.org/jira/browse/IGNITE-10328



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


TcpCommunicationSpi extension to ignore docker bridge network

2018-11-20 Thread David Harvey
We see some annoying behavior with S3 discovery because Ignite will push to
the discovery S3 bucket the IP address of the local docker bridge network
(172.17.0.1) in our case.   Basically, each node when coming online tries
that address first, and has to go through a network timeout to recover.

To address this, have prototyped a simple extension to TcpCommunicationSpi
to allow configuration of a list of IP addresses that should be completely
ignored, and will create a ticket and generate a pull request for it.

If there is a better approach, please let us know.

Thanks
Dave Harvey


[jira] [Created] (IGNITE-10349) Web Console: Add check for supported MongoDb version before WC start

2018-11-20 Thread Alexey Kuznetsov (JIRA)
Alexey Kuznetsov created IGNITE-10349:
-

 Summary: Web Console: Add check for supported MongoDb version 
before WC start
 Key: IGNITE-10349
 URL: https://issues.apache.org/jira/browse/IGNITE-10349
 Project: Ignite
  Issue Type: Task
  Components: wizards
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 2.8


Web console can work only with Mongo >= 3.2.x and <= 3.4.x

Lets stop backend if it was started with other version of Mongo.
And print a message about wrong Mongo version.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10348) Safely recreate metastore to mitigate IGNITE-8735

2018-11-20 Thread Alexey Goncharuk (JIRA)
Alexey Goncharuk created IGNITE-10348:
-

 Summary: Safely recreate metastore to mitigate IGNITE-8735
 Key: IGNITE-10348
 URL: https://issues.apache.org/jira/browse/IGNITE-10348
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.4
Reporter: Alexey Goncharuk
 Fix For: 2.8


We've fixed the issue IGNITE-8735, so new Ignite deployments are not affected 
by this issue, but old deployments still may fail if a wrong page was already 
put to the free list. We need to find out a way to repair this situation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #5435: temporary PR to verify renamed tests on Teamcity ...

2018-11-20 Thread oignatenko
Github user oignatenko closed the pull request at:

https://github.com/apache/ignite/pull/5435


---


Spring Update and delete queries

2018-11-20 Thread Jonathan Camargo
Hi community,

First let me introduce myself, My name is Jonathan Camargo, I'm a Java
developer at a software outsourcing company, I'm currently working for a
legacy project.

I'm currently learning new technologies and concepts such as IMDG, I have
been reading concepts and code about that mainly because I have find this
concept fascinating.

On the test project I'm working on I am using Spring Boot 2, Ignite Spring
Data 2.0 and a H2 DB (It could be any other but seems easier this way) and
aside from the already documented issues I have find that the
IgniteRepository does not support the Update and Delete queries, on the
code even is throwing a controlled exception:

if (parts.isDelete())
throw new UnsupportedOperationException("DELETE clause is not
supported now.");
else {
sql.append("SELECT ");

if (parts.isDistinct())
throw new UnsupportedOperationException("DISTINCT clause in
not supported.");

And I would like to point that the lib is validating if if is going to
generate the query or if it is going to use the @Query SQL that is being
sent, and that SQL is executed through the SqlFieldsQuery of the core lib.

So my suggestion would be to add the Update and Delete query to the
validation so those functions will run over the core.

I would not like to Inject the Ignite object, get or create the cache and
run those statements over there since that way the Ignite Repository would
not be needed and I would prefer just to remove the Ignite Spring lib and
work directly with the client and the server on other applications.

If is there any issue I could help with I would be there.

And I would like to know if is there any filter on Jira over the issues
that I could look.

Thanks.

-- 
Jonathan Giovanny Camargo Sanabria


[jira] [Created] (IGNITE-10347) Expose system SQL view for running queries

2018-11-20 Thread Yury Gerzhedovich (JIRA)
Yury Gerzhedovich created IGNITE-10347:
--

 Summary: Expose system SQL view for running queries
 Key: IGNITE-10347
 URL: https://issues.apache.org/jira/browse/IGNITE-10347
 Project: Ignite
  Issue Type: Task
Reporter: Yury Gerzhedovich
Assignee: Yury Gerzhedovich


Need to  expose system SQL view to provide list of running queries. Proposed 
name is *running_queries* with following columns: query_id, node_id, sql, 
schema_name, connection_id, duration.

Where,

query_id - cluster unique id of query.

node_id - id of node where query was started

sql - sql command

schema_name - SQL schema name

connection_id - unique connection id

duration - time in ms from start of execution of query

 

The view should contains all running queries on grid.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Service grid redesign

2018-11-20 Thread Vyacheslav Daradur
Denis, thanks for your participation!

>> There is no exchange in case of service deployment
There is some kind of exchange of services map which describes mapping
services instances to nodes in the cluster.

I'm a bit confused because of your notes about naming, the main goal
was to do the code to be transparent for Ignites experts and to not
confuse them.

Also, the messages names and structure has been presented and
discussed with community [1] during a design overview.

[1] 
http://apache-ignite-developers.2346864.n4.nabble.com/Service-Grid-new-design-overview-td34201.html
On Tue, Nov 20, 2018 at 5:19 PM Denis Mekhanikov  wrote:
>
> Vyacheslav,
>
> I'm in process of reviewing your changes. Sorry for taking so long.
> I posted the first portion of review comments yesterday.
> I'd like to finish looking through the code. I'll post more comments later.
>
> I see, that you called things analogously to partition map exchange.
> I realize, that there is an analogy in used procedures, but I don't really
> like the idea to use the same names for everything.
> The partition map exchange is called this way because it involves an actual
> exchange of information.
> All nodes need to tell each other, which partitions they have, and what
> their states are.
>
> There is no exchange in case of service deployment, so I would skip the
> "exchange" part.
> And *single message ->* *full message* look more like *request -> response*
> in case of services.
>
> Suppose we abandon the PME procedure and move to something else.
> Then *ServiceDeploymentExchange* name won't make sense.
> And I don't want to be in a situation, when I say to my colleague a word
> "exchange",
> and get "which one?" in return.
> So, I'm for the meaningful names rather than analogous to something else.
>
> What do you think?
>
> Denis
>
> вт, 20 нояб. 2018 г. в 12:09, Vyacheslav Daradur :
>
> > Denis, Yakov have you had a chance to review the solution?
> >
> > Igniters, we need to define a list of reviewers, otherwise no end in sign.
> >
> > I'm ready to continue work on the Service Grid, including new features
> > like hot-redeployment and versioning, also, I have ideas about new
> > tools for monitoring and management which will be useful for our
> > end-users.
> >
> > But for continuing work we need to overcome this first phase.
> >
> > On Tue, Nov 13, 2018 at 1:09 PM Vyacheslav Daradur 
> > wrote:
> > >
> > > Denis, Yakov, feel free to contact me directly in case of questions.
> > Thanks!
> > >
> > > On Sun, Nov 11, 2018 at 10:09 PM Denis Mekhanikov 
> > wrote:
> > > >
> > > > Guys,
> > > >
> > > > I'd like to take a look at the changes before they are merged.
> > > > I'll do my best to finish the review before the end of the upcoming
> > week.
> > > >
> > > > Thanks!
> > > > Denis
> > > >
> > > > сб, 10 нояб. 2018 г. в 14:25, Nikolay Izhikov :
> > > >
> > > > > Hello, Vladimir.
> > > > >
> > > > > I'm agree with you.
> > > > >
> > > > > Can we write the list of reviewers for this feature?
> > > > > Without a date or similar.
> > > > > Just a list of experts who should review this feature.
> > > > >
> > > > > В Сб, 10/11/2018 в 14:01 +0300, Vladimir Ozerov пишет:
> > > > > > Igniters,
> > > > > >
> > > > > > This is very huge thing with complex algorithms behind. We should
> > not
> > > > > merge
> > > > > > it to the product unless several additional thorough reviews are
> > ready,
> > > > > > irrespectively of how long will it take. We are about quality, not
> > speed.
> > > > > >
> > > > > > сб, 10 нояб. 2018 г. в 1:30, Denis Magda :
> > > > > >
> > > > > > > Vyacheslav,
> > > > > > >
> > > > > > > What are the cases when the service can be redeployed? Affinity,
> > > > > failure,
> > > > > > > etc., right. It would be good to list all the cases on the wiki
> > and
> > > > > then
> > > > > > > our tech writers will get everything documented.
> > > > > > >
> > > > > > > --
> > > > > > > Denis
> > > > > > >
> > > > > > > On Thu, Nov 8, 2018 at 11:06 PM Vyacheslav Daradur <
> > > > > daradu...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Denis,
> > > > > > > >
> > > > > > > > Services reassignment process takes into account previous
> > assignments
> > > > > > > > to avoid redundant redeployments.
> > > > > > > > So, in the described case, ServiceA won't be moved from node1
> > to
> > > > > node2.
> > > > > > > > On Fri, Nov 9, 2018 at 4:41 AM Denis Magda 
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > Vyacheslav,
> > > > > > > > >
> > > > > > > > > First of all, thanks for archiving this milestone and
> > rolling out
> > > > > these
> > > > > > > >
> > > > > > > > new
> > > > > > > > > capabilities.
> > > > > > > > >
> > > > > > > > > Speaking of the topology change events [1], does the new
> > > > > architecture
> > > > > > > >
> > > > > > > > avoid
> > > > > > > > > a running service redeployment when a new node joins? For
> > instance,
> > > > > > >
> > > > > > > let's
> > > > > > > > > say I have ServiceA running 

[GitHub] ignite pull request #5449: Tests for ignite 10343

2018-11-20 Thread 1vanan
GitHub user 1vanan opened a pull request:

https://github.com/apache/ignite/pull/5449

Tests for ignite 10343



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/1vanan/ignite tests-IGNITE-10343

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5449.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5449


commit e15efe2e6ea7d4476cad271a0c767236528b30cd
Author: Fedotov 
Date:   2018-11-20T13:53:52Z

add try/finally block

commit e4fc5c8dacd459f959945469df365cc110510d4c
Author: Fedotov 
Date:   2018-11-20T14:57:23Z

run 100 tests




---


[jira] [Created] (IGNITE-10346) Improve TcpDiscoveryVmIpFinder usability in Yardstick

2018-11-20 Thread Oleg Ostanin (JIRA)
Oleg Ostanin created IGNITE-10346:
-

 Summary: Improve TcpDiscoveryVmIpFinder usability in Yardstick
 Key: IGNITE-10346
 URL: https://issues.apache.org/jira/browse/IGNITE-10346
 Project: Ignite
  Issue Type: Improvement
Reporter: Oleg Ostanin
Assignee: Oleg Ostanin


 

Currently we have to set IP address list in both property file and Ignite XML 
config. However using 
 cfg.customProperties().get("SERVER_HOSTS")
 before starting the node we can get that list, set it to 
TcpDiscoveryVmIpFinder and start node. I think it will improve Yardstick 
usability a little bit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] asfgit closed pull request #75: IGNITE-10319 Suite compilation error failure handling added

2018-11-20 Thread GitBox
asfgit closed pull request #75: IGNITE-10319 Suite compilation error failure 
handling added
URL: https://github.com/apache/ignite-teamcity-bot/pull/75
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
index c7f0e11a..3d392371 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
@@ -578,7 +578,8 @@ private Build realLoadBuild(String href1) {
 
 private void registerCriticalBuildProblemInStat(BuildRef build, 
ProblemOccurrences problems) {
 boolean criticalFail = 
problems.getProblemsNonNull().stream().anyMatch(occurrence ->
-occurrence.isExecutionTimeout() || occurrence.isJvmCrash() || 
occurrence.isFailureOnMetric());
+occurrence.isExecutionTimeout() || occurrence.isJvmCrash() || 
occurrence.isFailureOnMetric()
+|| occurrence.isCompilationError());
 
 String suiteId = build.suiteId();
 Integer buildId = build.getId();
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/ISuiteResults.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/ISuiteResults.java
index 0d1b1755..ad2df5cf 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/ISuiteResults.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/ISuiteResults.java
@@ -21,6 +21,9 @@
  * Results from one or several builds for specific build type.
  */
 public interface ISuiteResults {
+/** */
+boolean hasCompilationProblem();
+
 boolean hasTimeoutProblem();
 
 boolean hasJvmCrashProblem();
@@ -30,14 +33,15 @@
 boolean hasExitCodeProblem();
 
 default boolean hasCriticalProblem() {
-return hasJvmCrashProblem() || hasTimeoutProblem();
+return hasJvmCrashProblem() || hasTimeoutProblem() || 
hasCompilationProblem();
 }
 
 default boolean hasSuiteIncompleteFailure() {
 return hasJvmCrashProblem()
 || hasTimeoutProblem()
 || hasOomeProblem()
-|| hasExitCodeProblem();
+|| hasExitCodeProblem()
+|| hasCompilationProblem();
 }
 
 String suiteId();
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/MultBuildRunCtx.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/MultBuildRunCtx.java
index 10b20fce..c6c203a7 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/MultBuildRunCtx.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/MultBuildRunCtx.java
@@ -111,6 +111,16 @@ public boolean onlyCancelledBuilds() {
 return builds;
 }
 
+/** {@inheritDoc} */
+@Override public boolean hasCompilationProblem() {
+return getCompilationProblemCount() > 0;
+}
+
+/** */
+public long getCompilationProblemCount() {
+return 
buildsStream().filter(ISuiteResults::hasCompilationProblem).count();
+}
+
 public boolean hasTimeoutProblem() {
 return getExecutionTimeoutCount() > 0;
 }
@@ -172,11 +182,13 @@ public String getResult() {
 addKnownProblemCnt(res, "JVM CRASH", getJvmCrashProblemCount());
 addKnownProblemCnt(res, "Out Of Memory Error", getOomeProblemCount());
 addKnownProblemCnt(res, "Exit Code", getExitCodeProblemsCount());
+addKnownProblemCnt(res, "Compilation Error", 
getCompilationProblemCount());
 
 {
 Stream stream =
 allProblemsInAllBuilds().filter(p ->
 !p.isFailedTests(compactor)
+&& !p.isCompilationError(compactor)
 && !p.isSnapshotDepProblem(compactor)
 && !p.isExecutionTimeout(compactor)
 && !p.isJvmCrash(compactor)
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/SingleBuildRunCtx.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/SingleBuildRunCtx.java
index 942d8903..8cdb7b16 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/SingleBuildRunCtx.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/analysis/SingleBuildRunCtx.java
@@ -68,6 +68,11 @@ public Integer buildId() {
 return buildCompacted.id() < 0 ? null : buildCompacted.id();
 }
 
+/** {@inheritDoc} */
+@Override public boolean hasCompilationProblem() {
+return getProblemsStream().anyMatch(p -> 

Re: Service grid redesign

2018-11-20 Thread Denis Mekhanikov
Vyacheslav,

I'm in process of reviewing your changes. Sorry for taking so long.
I posted the first portion of review comments yesterday.
I'd like to finish looking through the code. I'll post more comments later.

I see, that you called things analogously to partition map exchange.
I realize, that there is an analogy in used procedures, but I don't really
like the idea to use the same names for everything.
The partition map exchange is called this way because it involves an actual
exchange of information.
All nodes need to tell each other, which partitions they have, and what
their states are.

There is no exchange in case of service deployment, so I would skip the
"exchange" part.
And *single message ->* *full message* look more like *request -> response*
in case of services.

Suppose we abandon the PME procedure and move to something else.
Then *ServiceDeploymentExchange* name won't make sense.
And I don't want to be in a situation, when I say to my colleague a word
"exchange",
and get "which one?" in return.
So, I'm for the meaningful names rather than analogous to something else.

What do you think?

Denis

вт, 20 нояб. 2018 г. в 12:09, Vyacheslav Daradur :

> Denis, Yakov have you had a chance to review the solution?
>
> Igniters, we need to define a list of reviewers, otherwise no end in sign.
>
> I'm ready to continue work on the Service Grid, including new features
> like hot-redeployment and versioning, also, I have ideas about new
> tools for monitoring and management which will be useful for our
> end-users.
>
> But for continuing work we need to overcome this first phase.
>
> On Tue, Nov 13, 2018 at 1:09 PM Vyacheslav Daradur 
> wrote:
> >
> > Denis, Yakov, feel free to contact me directly in case of questions.
> Thanks!
> >
> > On Sun, Nov 11, 2018 at 10:09 PM Denis Mekhanikov 
> wrote:
> > >
> > > Guys,
> > >
> > > I'd like to take a look at the changes before they are merged.
> > > I'll do my best to finish the review before the end of the upcoming
> week.
> > >
> > > Thanks!
> > > Denis
> > >
> > > сб, 10 нояб. 2018 г. в 14:25, Nikolay Izhikov :
> > >
> > > > Hello, Vladimir.
> > > >
> > > > I'm agree with you.
> > > >
> > > > Can we write the list of reviewers for this feature?
> > > > Without a date or similar.
> > > > Just a list of experts who should review this feature.
> > > >
> > > > В Сб, 10/11/2018 в 14:01 +0300, Vladimir Ozerov пишет:
> > > > > Igniters,
> > > > >
> > > > > This is very huge thing with complex algorithms behind. We should
> not
> > > > merge
> > > > > it to the product unless several additional thorough reviews are
> ready,
> > > > > irrespectively of how long will it take. We are about quality, not
> speed.
> > > > >
> > > > > сб, 10 нояб. 2018 г. в 1:30, Denis Magda :
> > > > >
> > > > > > Vyacheslav,
> > > > > >
> > > > > > What are the cases when the service can be redeployed? Affinity,
> > > > failure,
> > > > > > etc., right. It would be good to list all the cases on the wiki
> and
> > > > then
> > > > > > our tech writers will get everything documented.
> > > > > >
> > > > > > --
> > > > > > Denis
> > > > > >
> > > > > > On Thu, Nov 8, 2018 at 11:06 PM Vyacheslav Daradur <
> > > > daradu...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Denis,
> > > > > > >
> > > > > > > Services reassignment process takes into account previous
> assignments
> > > > > > > to avoid redundant redeployments.
> > > > > > > So, in the described case, ServiceA won't be moved from node1
> to
> > > > node2.
> > > > > > > On Fri, Nov 9, 2018 at 4:41 AM Denis Magda 
> > > > wrote:
> > > > > > > >
> > > > > > > > Vyacheslav,
> > > > > > > >
> > > > > > > > First of all, thanks for archiving this milestone and
> rolling out
> > > > these
> > > > > > >
> > > > > > > new
> > > > > > > > capabilities.
> > > > > > > >
> > > > > > > > Speaking of the topology change events [1], does the new
> > > > architecture
> > > > > > >
> > > > > > > avoid
> > > > > > > > a running service redeployment when a new node joins? For
> instance,
> > > > > >
> > > > > > let's
> > > > > > > > say I have ServiceA running node1, then node2 joins and I
> don't
> > > > want
> > > > > >
> > > > > > the
> > > > > > > > service to be redeployed to any other node.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > >
> > > > > >
> > > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584#ServiceGridredesign.Phase1.Implementationdetails.-Topologychange
> > > > > > > >
> > > > > > > > --
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > On Wed, Nov 7, 2018 at 7:04 AM Vyacheslav Daradur <
> > > > daradu...@gmail.com
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Dmitriy, I published documentation in wiki:
> > > > > > > > >
> > > > > >
> > > > > >
> > > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584
> > > > > > > > >
> > > > > > > > > Thank you!
> > > > > > > > > On Wed, Nov 7, 2018 at 5:10 PM Dmitriy Pavlov <
> > > > 

[GitHub] ignite pull request #5436: IGNITE-10043 Do not reset LOST partitions when on...

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5436


---


[GitHub] ignite pull request #5448: IGNITE-10277 set prepared state before send prepa...

2018-11-20 Thread akalash
GitHub user akalash opened a pull request:

https://github.com/apache/ignite/pull/5448

IGNITE-10277 set prepared state before send prepare message



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10277

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5448.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5448


commit 29d79dc940a4457f7d730b9c53b2a7a9d0e9f37d
Author: Anton Kalashnikov 
Date:   2018-11-20T14:16:34Z

IGNITE-10277 set prepared state before send prepare message




---


[GitHub] ignite pull request #5447: IGNITE-10343

2018-11-20 Thread 1vanan
GitHub user 1vanan opened a pull request:

https://github.com/apache/ignite/pull/5447

IGNITE-10343



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/1vanan/ignite IGNITE-10343

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5447.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5447


commit e15efe2e6ea7d4476cad271a0c767236528b30cd
Author: Fedotov 
Date:   2018-11-20T13:53:52Z

add try/finally block




---


[jira] [Created] (IGNITE-10345) Discontinue forceServerMode support for client nodes.

2018-11-20 Thread Andrew Mashenkov (JIRA)
Andrew Mashenkov created IGNITE-10345:
-

 Summary: Discontinue forceServerMode support for client nodes.
 Key: IGNITE-10345
 URL: https://issues.apache.org/jira/browse/IGNITE-10345
 Project: Ignite
  Issue Type: Task
  Components: general
Reporter: Andrew Mashenkov
 Fix For: 3.0


1. Drop forceServerMode flag from TcpDiscoverySpi. It is already marked as 
deprecated.
2. remove all related tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #5446: IGNITE-10193: IgniteBaselineAffinityTopologyActiv...

2018-11-20 Thread avplatonov
GitHub user avplatonov opened a pull request:

https://github.com/apache/ignite/pull/5446

IGNITE-10193: 
IgniteBaselineAffinityTopologyActivationTest#testBaselineTopologyHistoryIsDeletedOnBaselineDelete
 fails



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10193

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5446.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5446


commit e05fae1aab267a72ef8d3351f54dff80b4304acb
Author: Alexey Platonov 
Date:   2018-11-20T13:41:50Z

remove old test

commit c8967246484415a806107850ad821a877618db48
Author: Alexey Platonov 
Date:   2018-11-20T13:42:09Z

Merge branch 'master' of https://github.com/apache/ignite into ignite-10193




---


[jira] [Created] (IGNITE-10344) Speed up cleanupRestoredCaches

2018-11-20 Thread Pavel Voronkin (JIRA)
Pavel Voronkin created IGNITE-10344:
---

 Summary: Speed up cleanupRestoredCaches
 Key: IGNITE-10344
 URL: https://issues.apache.org/jira/browse/IGNITE-10344
 Project: Ignite
  Issue Type: Improvement
Reporter: Pavel Voronkin


if (!cctx.kernalContext().clientNode() && !isLocalNodeInBaseline()) {
 // Stop all recovered caches and groups.
 cctx.cache().onKernalStopCaches(true);

 cctx.cache().stopCaches(true);

 cctx.database().cleanupRestoredCaches();

 // Set initial node started marker.
 cctx.database().nodeStart(null);
}

If we have many cache groups we spent a lot of time about 30sec to 
cleanupRestoredCaches().

We need to speed up this phase and add metrics on this.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #5371: IGNITE-9999 Logical recovery cleanup and refactor...

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5371


---


[GitHub] ignite pull request #5189: IGNITE-9828: Continuous query failover.

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5189


---


[GitHub] ignite pull request #5444: IGNITE-10341 Added lost policy tests with persist...

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5444


---


[GitHub] SomeFire commented on a change in pull request #76: IGNITE-10275 Refactor of visa caching.

2018-11-20 Thread GitBox
SomeFire commented on a change in pull request #76: IGNITE-10275 Refactor of 
visa caching.
URL: https://github.com/apache/ignite-teamcity-bot/pull/76#discussion_r234972281
 
 

 ##
 File path: 
ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/ObserverTask.java
 ##
 @@ -57,88 +48,69 @@
 /** Logger. */
 private static final Logger logger = 
LoggerFactory.getLogger(ObserverTask.class);
 
-/** */
-public static final String BUILDS_CACHE_NAME = "compactBuildsInfosCache";
-
 /** Helper. */
 @Inject private ITcHelper tcHelper;
 
 /** Helper. */
 @Inject private IJiraIntegration jiraIntegration;
 
-/** Ignite. */
-@Inject private Ignite ignite;
-
 /** */
 @Inject private VisasHistoryStorage visasHistStorage;
 
 /** */
-@Inject private IStringCompactor strCompactor;
+private ReentrantLock observationLock = new ReentrantLock();
 
 /** */
-private ReentrantLock observationLock = new ReentrantLock();
+private Map infos = new HashMap<>();
 
 /**
  */
 ObserverTask() {
 }
 
 /** */
-private IgniteCache 
compactInfos() {
-return 
ignite.getOrCreateCache(TcHelperDb.getCacheV2TxConfig(BUILDS_CACHE_NAME));
+public void init() {
+visasHistStorage.getLastVisas().stream()
+.filter(req -> req.isObserving())
+.forEach(req -> infos.put(req.getInfo().getContributionKey(), 
req.getInfo()));
 }
 
 /** */
 @Nullable public BuildsInfo getInfo(ContributionKey key) {
-CompactBuildsInfo compactBuildsInfo = compactInfos().get(new 
CompactContributionKey(key, strCompactor));
-
-return Objects.isNull(compactBuildsInfo) ? null : 
compactBuildsInfo.toBuildInfo(strCompactor);
+return infos.get(key);
 }
 
 
 /** */
 public Collection getInfos() {
-List buildsInfos = new ArrayList<>();
-
-compactInfos().forEach(entry -> 
buildsInfos.add(entry.getValue().toBuildInfo(strCompactor)));
-
-return buildsInfos;
+return infos.values();
 }
 
 /** */
 public void addInfo(BuildsInfo info) {
-visasHistStorage.put(new VisaRequest(info));
+visasHistStorage.updateLastVisaRequest(info.getContributionKey(), req 
-> req.setObservingStatus(false));
 
 Review comment:
   Add observation lock here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] ignite pull request #5407: Ignite-10217

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5407


---


[jira] [Created] (IGNITE-10342) TC: Create "Run :: MVCC" suite.

2018-11-20 Thread Andrew Mashenkov (JIRA)
Andrew Mashenkov created IGNITE-10342:
-

 Summary: TC: Create "Run :: MVCC" suite.
 Key: IGNITE-10342
 URL: https://issues.apache.org/jira/browse/IGNITE-10342
 Project: Ignite
  Issue Type: Sub-task
  Components: mvcc
Reporter: Andrew Mashenkov
Assignee: Andrew Mashenkov
 Fix For: 2.8


Create MVCC version of IgniteCacheTestSuite3 and add it to TC.

All non-relevant tests should be marked as ignored.
 Failed tests should be muted and tickets should be created for unknown failure 
reasons.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #5431: IGNITE-10328

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5431


---


[GitHub] ignite pull request #5445: IGNITE-10236: MVCC: Create "Cache 3" test suite f...

2018-11-20 Thread AMashenkov
GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/5445

IGNITE-10236: MVCC: Create "Cache 3" test suite for MVCC mode.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10236

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5445.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5445


commit 50b9e9dcf5875727b42abcf707fdb50f03f898ce
Author: Andrey V. Mashenkov 
Date:   2018-10-25T14:55:01Z

IGNITE-10002: Add MvccCacheTestSuite2. Add flag to force mvcc mode.

commit b307bfb1a73955ff416d8c5b9dd74c4e2990f91d
Author: Andrey V. Mashenkov 
Date:   2018-10-25T16:21:24Z

IGNITE-10002: Ignore AtomicCache tests.
Mute NearCache tests.

commit 3bb55091b240fce4888fc1e1c44b9132532dba1a
Author: Andrey V. Mashenkov 
Date:   2018-10-26T18:02:55Z

IGNITE-10004: Mute NearCache and CacheStore tests.

commit 3c0071d2a6494dae2881a3c19849e26390bfd5d9
Author: Andrey V. Mashenkov 
Date:   2018-10-26T18:17:39Z

IGNITE-10004: WIP.

commit 915fc162783dfa05f14ebd44918eec9452cf5ae9
Author: Andrey V. Mashenkov 
Date:   2018-10-26T18:32:33Z

IGNITE-10004: Mute unsupported cases.

commit 6257df961693b15433568d1223976151ac37458e
Author: Andrey V. Mashenkov 
Date:   2018-10-29T12:26:39Z

IGNITE-10004: WIP.

commit 3732a8b049200022e0af85e2441e6c1921461b4d
Author: Andrey V. Mashenkov 
Date:   2018-10-29T13:40:37Z

IGNITE-10004: WIP.

commit 185baf7ce1ccc414067640fdf2f13b00e052ed23
Author: Andrey V. Mashenkov 
Date:   2018-10-29T13:41:27Z

IGNITE-10004: WIP.

commit 4c65f29b7b11556a0d9c38f107f1e3f5eb0cebe9
Author: Andrey V. Mashenkov 
Date:   2018-10-29T13:53:40Z

IGNITE-10004: WIP.

commit 3d48430a957459cd0819e1b1f676b251b7b86438
Author: Andrey V. Mashenkov 
Date:   2018-10-29T14:20:37Z

IGNITE-10004: WIP.

commit a44f9be8bac14c31ebd1b84351b5a659156b4466
Author: Andrey V. Mashenkov 
Date:   2018-10-29T16:13:22Z

IGNITE-10004: WIP.

commit 7d242f4b6b6b13a2876fc8068e2d8f37aa9b2a57
Author: Andrey V. Mashenkov 
Date:   2018-10-29T16:15:26Z

IGNITE-10004: WIP.

commit 0ac066f7a10938d2309b56fefb8735fdc1675100
Author: Andrey V. Mashenkov 
Date:   2018-10-30T08:54:58Z

IGNITE-10002: WIP.

commit 28f105989b8270e96c5bf59f24639a92c74aca97
Author: Andrey V. Mashenkov 
Date:   2018-10-30T09:04:44Z

IGNITE-10002: Minor.

commit 2a61ec244c9a02e200beb211a3c7b0c73be1ec2a
Author: Andrey V. Mashenkov 
Date:   2018-10-30T10:49:35Z

IGNITE-10002: Disable non-supported Tx modes.

commit 160b39753c41de3d0e3041b07db893fe327c891b
Author: Andrey V. Mashenkov 
Date:   2018-10-30T12:19:42Z

IGNITE-10002: Disable non-supported Tx modes.

commit 58f00edd49e2a45c650a9c0089ffc9b13217ca74
Author: Andrey V. Mashenkov 
Date:   2018-10-30T12:49:42Z

IGNITE-10002: Minor.

commit e5f5838fd1e1655283ba5c56bc8a6cc4ae69412c
Author: Andrey V. Mashenkov 
Date:   2018-11-13T08:15:53Z

IGNITE-10002: Fix test.

commit 982b4574d201c055ce52005e61308536cb532547
Author: Andrey V. Mashenkov 
Date:   2018-11-14T09:34:10Z

IGNITE-10002: Fix FORCE_MVCC flag.

commit 4c3ccee618180a7282bedc49752c9c75f0711e68
Author: Andrey V. Mashenkov 
Date:   2018-11-14T09:34:20Z

IGNITE-10002: Fix tests.

commit 9d65e2614c6182c4fcaf6fe74bf299c73af2bc07
Author: Andrey V. Mashenkov 
Date:   2018-11-14T11:03:26Z

IGNITE-10002: Fix tx tests.

commit ddb59fcea609a62aa77532db5bcde83c96e0f382
Author: Andrey V. Mashenkov 
Date:   2018-11-14T12:10:07Z

IGNITE-10002: Fix hanged test.

commit 712e6d8ddcdaa5aecd7762d25cb062f94ef06ce5
Author: Andrey V. Mashenkov 
Date:   2018-11-14T13:34:41Z

IGNITE-10002: Fix node stop.

commit db9aca4792753e6b0ca2eddb8c8795b41498e682
Author: Andrey V. Mashenkov 
Date:   2018-11-14T15:08:51Z

IGNITE-10002: Mute hanged test.

commit 30d24b1c0a961d3cb9e64231867cc87982f84332
Author: user 
Date:   2018-11-14T19:38:52Z

IGNITE-10052: Fix and mute mvcc tests.

commit 191500222e87256f3417019bd2df3dc886918806
Author: user 
Date:   2018-11-14T19:48:10Z

IGNITE-10002: Mute mvcc tests.

commit 0293f36c7621843a8cf84d848df37f6996a21d71
Author: user 
Date:   2018-11-14T20:10:59Z

IGNITE-10002: Mute mvcc tests.

commit 760972570935ad59a118267ce3589ccad77f7ad0
Author: Andrey V. Mashenkov 
Date:   2018-11-15T09:05:46Z

IGNITE-10002: Fix FORCE_MVCC flag.

commit 219c402f6c95ca12015bf8ba314b9c20cc1b6683
Author: Andrey V. Mashenkov 
Date:   2018-11-15T09:07:42Z

IGNITE-10002: Unmute mvcc localPeek tests.

commit c0fc6c9c113c23be058d56b63fbe336ffd165491
Author: Andrey V. Mashenkov 
Date:   2018-11-15T09:35:00Z

IGNITE-10002: Mute tests.




---


Re: Code inspection

2018-11-20 Thread Nikolay Izhikov
Hello, Vyacheslav.

Yes, we have.

Maxim Muzafarov, can you fix it, please?

вт, 20 нояб. 2018 г., 13:10 Vyacheslav Daradur daradu...@gmail.com:

> Guys, why we have 2 different inspection files in the repo?
> idea\ignite_inspections.xml
> idea\ignite_inspections_teamcity.xml
>
> AFAIK TeamCity is able to use the same inspection file with IDE.
>
> I've imported 'idea\ignite_inspections.xml' in the IDE, but now see
> inspection warnings for my PR on TC because of different rules.
>
>
> On Sun, Nov 11, 2018 at 6:06 PM Maxim Muzafarov 
> wrote:
> >
> > Yakov, Dmitry,
> >
> > Which example of unsuccessful suite execution do we need?
> > Does the current fail [1] in the master branch enough to configure
> > notifications by TC.Bot?
> >
> > > Please consider adding more checks
> > > - line endings. I think we should only have \n
> > > - ensure blank line at the end of file
> >
> > It seems to me that `line endings` is easy to add, but for the `blank
> > line at the end` we need as special regexp. Can we focus on built-in
> > IntelliJ inspections at first and fix others special further?
> >
> > [1]
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
> > On Sun, 11 Nov 2018 at 17:55, Maxim Muzafarov 
> wrote:
> > >
> > > Igniters,
> > >
> > > Since the inspection rules are included in RunAll a few members of the
> > > community mentioned a wide distributed execution time on TC agents:
> > >  - 1h:27m:38s publicagent17_9094
> > >  - 38m:04s publicagent17_9094
> > >  - 33m:29s publicagent17_9094
> > >  - 17m:13s publicagent17_9094
> > > It seems that we should configure the resources distribution across TC
> > > containers. Can anyone take a look at it?
> > >
> > >
> > > I've also prepared the short list of rules to work on:
> > > + Inconsistent line separators (6 matches)
> > > + Problematic whitespace (4 matches)
> > > + expression.equals("literal")' rather than
> > > '"literal".equals(expression) (53 matches)
> > > + Unnecessary 'null' check before 'instanceof' expression or call (42
> matches)
> > > + Redundant 'if' statement (69 matches)
> > > + Redundant interface declaration (28 matches)
> > > + Double negation (0 matches)
> > > + Unnecessary code block (472 matches)
> > > + Line is longer than allowed by code style (2614 matches) (Is it
> > > possible to implement?)
> > >
> > > WDYT?
> > >
> > > On Fri, 26 Oct 2018 at 23:43, Dmitriy Pavlov 
> wrote:
> > > >
> > > > Hi Maxim,
> > > >
> > > >  thank you for your efforts to make this happen. Keep the pace!
> > > >
> > > > Could you please provide an example of how Inspections can fail, so
> I or
> > > > another contributor could implement support of these failures
> validation in
> > > > the Tc Bot.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > пт, 26 окт. 2018 г. в 18:27, Yakov Zhdanov :
> > > >
> > > > > Maxim,
> > > > >
> > > > > Thanks for response, let's do it the way you suggested.
> > > > >
> > > > > Please consider adding more checks
> > > > > - line endings. I think we should only have \n
> > > > > - ensure blank line in the end of file
> > > > >
> > > > > All these are code reviews issues I pointed out many times when
> reviewing
> > > > > conributions. It would be cool if we have TC build failing if
> there is any.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > --Yakov
> > > > >
>
>
>
> --
> Best Regards, Vyacheslav D.
>


[GitHub] ignite pull request #5444: IGNITE-10341 Added lost policy tests with persist...

2018-11-20 Thread dgovorukhin
GitHub user dgovorukhin opened a pull request:

https://github.com/apache/ignite/pull/5444

IGNITE-10341 Added lost policy tests with persistence enable



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10341

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5444.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5444


commit 38f940ef1499e57beb52ad7791deedea02b113f1
Author: Dmitriy Govorukhin 
Date:   2018-11-20T10:48:45Z

IGNITE-10341 Added lost policy tests with persistence enable

Signed-off-by: Dmitriy Govorukhin 




---


[GitHub] ignite pull request #5443: IGNITE-10318 Web console: use newer panels in que...

2018-11-20 Thread Klaster1
GitHub user Klaster1 opened a pull request:

https://github.com/apache/ignite/pull/5443

IGNITE-10318 Web console: use newer panels in query notebook screen

[The issue](https://issues.apache.org/jira/browse/IGNITE-10318).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10318

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5443.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5443


commit 76abafd5e3675c291279f029ff129ddb421c6e55
Author: Ilya Borisov 
Date:   2018-11-20T03:50:05Z

IGNITE-10318 Use new panels, move query actions to context menu.

commit 135ed23f7ce761da79c7bbdb089562ddf5dc4ae8
Author: Ilya Borisov 
Date:   2018-11-20T10:40:23Z

IGNITE-10318 Remove icon from input dialog template.

commit 181367357e77d97db3bd71cdca5e76c14b60cb92
Author: Ilya Borisov 
Date:   2018-11-20T10:40:40Z

IGNITE-10318 Change wording in query rename prompt.




---


[jira] [Created] (IGNITE-10341) Missed loss policy tests with persistence

2018-11-20 Thread Dmitriy Govorukhin (JIRA)
Dmitriy Govorukhin created IGNITE-10341:
---

 Summary: Missed loss policy tests with persistence
 Key: IGNITE-10341
 URL: https://issues.apache.org/jira/browse/IGNITE-10341
 Project: Ignite
  Issue Type: Bug
Reporter: Dmitriy Govorukhin


After IGNITE-10207 was implemented, the test was removed (check policy if 
persistence enables), it is a mistake, need to revert this changes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #5442: IGNITE-8913

2018-11-20 Thread devozerov
GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/5442

IGNITE-8913



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8913-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5442.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5442


commit 26543e10f8143dbc2d313b870081d633baf4cd05
Author: SGrimstad 
Date:   2018-08-14T11:21:13Z

IGNITE-9141  Implemented

commit ee631080782fe17a007e10117a96fc1a72990854
Author: devozerov 
Date:   2018-08-14T12:54:32Z

Merge branch 'master' into ignite-9141

commit 4587bfc24f25045fd7fc2197076797cc6ca54e32
Author: devozerov 
Date:   2018-08-14T13:23:19Z

Review.

commit 1511eb31b644113091be156d872f5adb2daecb84
Author: SGrimstad 
Date:   2018-08-14T11:21:13Z

IGNITE-9141  Implemented

commit 0425a32a3d490165b29bcf9236a49a48f7761666
Author: devozerov 
Date:   2018-08-14T13:23:19Z

Review.

commit a7209f870148f984cdbf3e6437337c7b665defb5
Author: SGrimstad 
Date:   2018-08-20T11:56:11Z

IGNITE-9141 Modified according to review comments. Integration tests added

commit 08e196ad4503d4487fda0da2b61406ef3bd84cbf
Author: SGrimstad 
Date:   2018-08-20T12:28:46Z

IGNITE-9141 javadoc added

commit b006a8dba4ae4319aa78a0d3c6fd6eef47bb1da8
Author: SGrimstad 
Date:   2018-08-20T12:29:46Z

IGNITE-9141 javadoc added

commit 0ed04a7ca655866685841b2ec9c2ed64797bf233
Author: devozerov 
Date:   2018-08-28T08:05:04Z

Merge branch 'master' into ignite-9141

commit 02db267355a8405c64c45d51c81e342df664d612
Author: devozerov 
Date:   2018-08-28T08:45:55Z

Review comments.

commit 0c4301cdc0d6108ed5b51173144e19d3ad450e63
Author: SGrimstad 
Date:   2018-08-28T11:08:47Z

IGNITE-9141 Fixes according to review

commit dd35f024d77d12badf711bce3644450008e38921
Author: SGrimstad 
Date:   2018-08-29T12:15:43Z

IGNITE-8913 Query cancelled messages are enriched with details, tests 
updated

commit ce15a79ded414f3b41bbe8ca3521f5ff4aaa3174
Author: zaleslaw 
Date:   2018-08-28T11:03:37Z

IGNITE-9393:[ML] KMeans fails on complex data in cache

this closes #4628

commit 435184a203598740ace2f0b53cf136dc3443e168
Author: Roman Guseinov 
Date:   2018-08-28T11:41:09Z

IGNITE-9367: Fixed crash in ODBC on executing query with closed connection

This closes #4621

commit 82e55e927260327996495360b5ae7019a1c958a6
Author: Denis Mekhanikov 
Date:   2018-08-28T14:46:29Z

IGNITE-9389 Avoid deadlock on cache#close(). - Fixes #4626.

Signed-off-by: Alexey Goncharuk 

commit a4dedb746a5997d8f318b7ee059825e7b7101687
Author: Aleksei Scherbakov 
Date:   2018-08-28T15:18:26Z

IGNITE-9401 Fixed race in tx rollback test - Fixes #4633.

Signed-off-by: Alexey Goncharuk 

commit b8b2e7098a675a4f37264579be1dfd2c89f3316c
Author: Ivan Rakov 
Date:   2018-08-29T12:32:59Z

IGNITE-6879 Support Spring Data 2.0 - Added package description to 
parent/pom.xml

commit c1f5a85c61ad6086b8b180b3169fed30f1155841
Author: Alexey Goncharuk 
Date:   2018-08-30T08:35:53Z

IGNITE-9326 Fixed deferred serialization error handling for EntryProcessor 
result - Fixes #4588.

Signed-off-by: Alexey Goncharuk 

commit ee2c5a7b7c7f0a0684fdaa063c759057c4bfa6ba
Author: zaleslaw 
Date:   2018-08-30T08:56:39Z

IGNITE-9421: ML Examples: LogisticRegressionSGDTrainerExample
example result not correct

this closes #4646

commit f858474292805cf93a30c01676f519790402df1c
Author: Alexey Goncharuk 
Date:   2018-08-30T09:39:22Z

IGNITE-9429 Fixed flaky GridCacheReplicatedDataStructuresFailoverSelfTest

commit 88d16494516ab66af3849aae8e3aa6faf7f32efb
Author: Vasiliy Sisko 
Date:   2018-08-30T11:11:21Z

IGNITE-9370 Fixed execution of REST commands in demo mode.

commit 23b33b617356704370f6b15b05b41f3edc5b2bd9
Author: Igor Seliverstov 
Date:   2018-08-30T11:52:49Z

IGNITE-4191: MVCC and transactional SQL support. Joint multi-man-years 
efforts of Semen Boikov, Igor Seliverstov, Alexander Paschenko, Igor Sapego, 
Sergey Kalashnikov, Roman Kondakov, Pavel Kuznetsov, Ivan Pavlukhin, Andrey 
Mashenkov, and many others. Special thanks for design ideas and review to 
Alexey Goncharuk and Sergi Vladykin. This closes #3220.

commit eec22c55249669091edf404f126181a350197ada
Author: Anton Kalashnikov 
Date:   2018-08-30T12:02:50Z

IGNITE-6552 Added ability to set WAL history size in bytes - Fixes #3559.

commit 8af84fe0f62880937b7350efa9a9646086a98ce3
Author: EdShangGG 
Date:   2018-08-30T13:47:23Z

IGNITE-9302 Added timeouts for Java Thin Clients tests - Fixes #4562.

Signed-off-by: Alexey Goncharuk 

commit 1d4ccec375a67fa0abd3b6beaa5d80ded8ed8424
Author: Alexey Platonov 
Date:   2018-08-30T16:28:34Z

IGNITE-9237: [ML] Random forest 

[GitHub] ignite pull request #5441: IGNITE-5439 Jdbc thin cancel implemented.

2018-11-20 Thread sanpwc
GitHub user sanpwc opened a pull request:

https://github.com/apache/ignite/pull/5441

IGNITE-5439 Jdbc thin cancel implemented.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5439

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5441.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5441


commit 25c6a1d742ec7ed8bd87e828479069be45ec1bf7
Author: sanpwc 
Date:   2018-11-12T08:31:10Z

IGNITE-5439: Initial implementation.

commit 359d2c9b60730b4a9c6cc9e053d022e1f36548d9
Author: alapin 
Date:   2018-11-12T09:59:01Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-5439

commit 423a1b2d2143fe82dd6a0c07b8541bc2a42a1c4b
Author: sanpwc 
Date:   2018-11-13T11:07:13Z

IGNITE-5439: Work in progress;

commit 99dc0798d1b8a55b402e9dcd248c35320cfa3ee6
Author: sanpwc 
Date:   2018-11-15T15:12:06Z

IGNITE-5439: Request Id addedboth to JdbcRequest and JdbcResponse, 
cancellation thread-safety added, minor changes;

commit a2e3a010df6fad1ce2902a4c641911d44ffe2b32
Author: alapin 
Date:   2018-11-15T15:13:45Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-5439

commit 6d12f3648aa3a7254ec42be40e9a965cc9ed9bf4
Author: sanpwc 
Date:   2018-11-16T12:03:12Z

IGNITE-5439: Support for canceling file uploads added, few more tests added;

commit 67f5f91b1f5e650943edf6c0e1e2efe7db020305
Author: alapin 
Date:   2018-11-16T12:04:01Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-5439

commit fb7f3c7db2a04b9c89f212b3ae9e8d865404fd91
Author: sanpwc 
Date:   2018-11-16T12:52:22Z

IGNITE-5439: bug in tests fixed;

commit cfb918d672824cced7a852cb6e164e2a17ce8860
Author: sanpwc 
Date:   2018-11-16T12:53:15Z

IGNITE-5439: warning suppresed in tests;

commit 440795fa04143e35c7b09c04e2fa2fcbc9406969
Author: sanpwc 
Date:   2018-11-16T14:25:40Z

IGNITE-5439: more tests added: multistatement and batch canceling;

commit 95966c89f25d790d3b4241efce4972a630cd18fd
Author: sanpwc 
Date:   2018-11-19T10:59:21Z

IGNITE-5439: Issues fixed after review;

commit 4c1ecfd0261f0a779e8aade6dd8ea5433ff570d1
Author: tledkov 
Date:   2018-11-19T11:15:58Z

IGNITE-5439: minors

commit b592040147d803d8b7acdd6534e104efc7cfe3e4
Author: tledkov 
Date:   2018-11-19T12:20:28Z

IGNITE-5439: minors

commit c7cf72c5540493f0510fa6f06ddfa6efe056a95f
Author: alapin 
Date:   2018-11-19T13:37:06Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-5439

commit ec7f979034f22498849d4f94608f3fb84686a0cd
Author: alapin 
Date:   2018-11-19T13:42:25Z

Merge branch 'ignite-5439' of https://github.com/gridgain/apache-ignite 
into ignite-5439

commit 33d396007c350057c6c8c9f0cb53167ffdfb6cb7
Author: sanpwc 
Date:   2018-11-19T13:45:20Z

IGNITE-5439: Error handling, logging and security handling added within 
GridNioPriorityQueryFilter;

commit 3ae5b07f161401607a7c71f9da02394c2e85cb30
Author: sanpwc 
Date:   2018-11-19T14:12:13Z

IGNITE-5439: Error handling within JdbcRequestHandler#cancelQuery added;

commit cdb7ab9ccc65fb2929cba1fc49de086852410d53
Author: sanpwc 
Date:   2018-11-19T15:00:54Z

IGNITE-5439: More methods of JdbcThinStatement now throws 
SQLException(Query was cancelled).

commit ac7635d25421bf0d7bb0d6cf85f6c1342a0ad6b6
Author: sanpwc 
Date:   2018-11-19T15:50:15Z

IGNITE-5439: RequestToCursorsMapping cleanup implemented.

commit 7615488361c13db65675ea7e06abb759d4ec46ad
Author: sanpwc 
Date:   2018-11-19T16:07:56Z

IGNITE-5439: ensureAlive() restored in several cases;

commit 6254a5f91569066f902dc20e9d0c86c60709998a
Author: sanpwc 
Date:   2018-11-20T08:55:33Z

IGNITE-5439: synchronizedToCAS optimizations;

commit c3e4115a4b9f47759440640820f63ba09f61a31d
Author: alapin 
Date:   2018-11-20T08:56:13Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-5439

commit 91df0d6505b59220e7293bc95b3b5e30575be040
Author: sanpwc 
Date:   2018-11-20T10:06:32Z

IGNITE-5439: failing 
testExpectSQLExceptionAndAFAPControlRetrievalAfterCancelingLongRunningFileUpload
 because of IGNITE-10340s;




---


Re: Code inspection

2018-11-20 Thread Vyacheslav Daradur
Guys, why we have 2 different inspection files in the repo?
idea\ignite_inspections.xml
idea\ignite_inspections_teamcity.xml

AFAIK TeamCity is able to use the same inspection file with IDE.

I've imported 'idea\ignite_inspections.xml' in the IDE, but now see
inspection warnings for my PR on TC because of different rules.


On Sun, Nov 11, 2018 at 6:06 PM Maxim Muzafarov  wrote:
>
> Yakov, Dmitry,
>
> Which example of unsuccessful suite execution do we need?
> Does the current fail [1] in the master branch enough to configure
> notifications by TC.Bot?
>
> > Please consider adding more checks
> > - line endings. I think we should only have \n
> > - ensure blank line at the end of file
>
> It seems to me that `line endings` is easy to add, but for the `blank
> line at the end` we need as special regexp. Can we focus on built-in
> IntelliJ inspections at first and fix others special further?
>
> [1] 
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
> On Sun, 11 Nov 2018 at 17:55, Maxim Muzafarov  wrote:
> >
> > Igniters,
> >
> > Since the inspection rules are included in RunAll a few members of the
> > community mentioned a wide distributed execution time on TC agents:
> >  - 1h:27m:38s publicagent17_9094
> >  - 38m:04s publicagent17_9094
> >  - 33m:29s publicagent17_9094
> >  - 17m:13s publicagent17_9094
> > It seems that we should configure the resources distribution across TC
> > containers. Can anyone take a look at it?
> >
> >
> > I've also prepared the short list of rules to work on:
> > + Inconsistent line separators (6 matches)
> > + Problematic whitespace (4 matches)
> > + expression.equals("literal")' rather than
> > '"literal".equals(expression) (53 matches)
> > + Unnecessary 'null' check before 'instanceof' expression or call (42 
> > matches)
> > + Redundant 'if' statement (69 matches)
> > + Redundant interface declaration (28 matches)
> > + Double negation (0 matches)
> > + Unnecessary code block (472 matches)
> > + Line is longer than allowed by code style (2614 matches) (Is it
> > possible to implement?)
> >
> > WDYT?
> >
> > On Fri, 26 Oct 2018 at 23:43, Dmitriy Pavlov  wrote:
> > >
> > > Hi Maxim,
> > >
> > >  thank you for your efforts to make this happen. Keep the pace!
> > >
> > > Could you please provide an example of how Inspections can fail, so I or
> > > another contributor could implement support of these failures validation 
> > > in
> > > the Tc Bot.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 26 окт. 2018 г. в 18:27, Yakov Zhdanov :
> > >
> > > > Maxim,
> > > >
> > > > Thanks for response, let's do it the way you suggested.
> > > >
> > > > Please consider adding more checks
> > > > - line endings. I think we should only have \n
> > > > - ensure blank line in the end of file
> > > >
> > > > All these are code reviews issues I pointed out many times when 
> > > > reviewing
> > > > conributions. It would be cool if we have TC build failing if there is 
> > > > any.
> > > >
> > > > Thanks!
> > > >
> > > > --Yakov
> > > >



-- 
Best Regards, Vyacheslav D.


[jira] [Created] (IGNITE-10340) JDBC thin: in some cases it's not possible to convert IllegalStateException to SQLException with appropriate message.

2018-11-20 Thread Alexander Lapin (JIRA)
Alexander Lapin created IGNITE-10340:


 Summary: JDBC thin: in some cases it's not possible to convert 
IllegalStateException to SQLException with appropriate message.
 Key: IGNITE-10340
 URL: https://issues.apache.org/jira/browse/IGNITE-10340
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.6
Reporter: Alexander Lapin


In case of using already closed JdbcBulkLoadProcessor following exception might 
be thrown:
{code}
[12:04:15] (err) Error processing file batchjava.lang.IllegalStateException: 
Data streamer has been closed.
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.closedException(DataStreamerImpl.java:1081)
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.lock(DataStreamerImpl.java:443)
...
{code}

However, cause cancellationReason is null it's not possible to detect whether 
query was cancelled or not:
{code:title=DataStreamerImpl.java|borderStyle=solid}
private void closedException() {
throw new IllegalStateException("Data streamer has been closed.", 
cancellationReason);
}
{code}

Reproducer:
{code}
org.apache.ignite.jdbc.thin.JdbcThinStatementCancelSelfTest
#testExpectSQLExceptionAndAFAPControlRetrievalAfterCancelingLongRunningFileUpload
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Apache Ignite 2.7. Last Mile

2018-11-20 Thread Dmitrii Ryabov
Yes, revert both.

вт, 20 нояб. 2018 г., 11:52 Vladimir Ozerov voze...@gridgain.com:

> +1 for reverting both.
>
> On Tue, Nov 20, 2018 at 9:43 AM Nikolay Izhikov 
> wrote:
>
> > Hello, Dmitrii.
> >
> > I see 2 tickets for this improvement:
> >
> > IGNITE-602 - [Test] GridToStringBuilder is vulnerable for
> > StackOverflowError caused by infinite recursion [1]
> > IGNITE-9209 - GridDistributedTxMapping.toString() returns broken string
> [2]
> >
> > Should we revert both commits?
> >
> > [1] https://github.com/apache/ignite/commit/d67c5bf
> > [2] https://github.com/apache/ignite/commit/9bb9c04
> >
> >
> > В Пн, 19/11/2018 в 13:36 +0300, Dmitrii Ryabov пишет:
> > > I agree to revert and make fix for 2.8. So, we will have more time to
> > test
> > > it.
> > >
> > > пн, 19 нояб. 2018 г., 10:53 Vladimir Ozerov voze...@gridgain.com:
> > >
> > > > +1 for revert.
> > > >
> > > > On Sun, Nov 18, 2018 at 11:31 PM Dmitriy Pavlov 
> > > > wrote:
> > > >
> > > > > I personally don't mind.
> > > > >
> > > > > But I would like Dmitry Ryabov and Alexey Goncharuck share their
> > > >
> > > > opinions.
> > > > >
> > > > > вс, 18 нояб. 2018 г., 20:43 Nikolay Izhikov :
> > > > >
> > > > > > Yes, I think so.
> > > > > >
> > > > > > вс, 18 нояб. 2018 г., 20:34 Denis Magda dma...@apache.org:
> > > > > >
> > > > > > > Sounds good to me. Are we starting the vote then?
> > > > > > >
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Nov 18, 2018 at 8:25 AM Nikolay Izhikov <
> > nizhi...@apache.org
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello, Igniters.
> > > > > > > >
> > > > > > > > This issue is the only ticket that blocks 2.7 release.
> > > > > > > >
> > > > > > > > I looked at IGNITE-602 PR and GridToStringBuilder.
> > > > > > > > The code looks complicated for me.
> > > > > > > > And it's not obvious for me how to fix this issue in a short
> > period
> > > > >
> > > > > of
> > > > > > > > time.
> > > > > > > > Especially, code deals with recursion and other things that
> can
> > > >
> > > > lead
> > > > > to
> > > > > > > > very dangerous errors.
> > > > > > > >
> > > > > > > > Let's revert this patch and fix it in calmly.
> > > > > > > > Also, we need additional tests for it.
> > > > > > > >
> > > > > > > > В Пт, 16/11/2018 в 17:57 +0300, Dmitrii Ryabov пишет:
> > > > > > > > > Ok, I'll check the issue.
> > > > > > > > > пт, 16 нояб. 2018 г. в 17:52, Alexey Goncharuk <
> > > > > > > >
> > > > > > > > alexey.goncha...@gmail.com>:
> > > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > > > > > > > > I've just found that S.toString() implementation is
> broken
> > in
> > > > > > > >
> > > > > > > > ignite-2.7 and master [1]. It leads to a message
> > > > > > > > > > Wrapper [p=Parent [a=0]Child [b=0, super=]]
> > > > > > > > > > being formed instead of
> > > > > > > > > > Wrapper [p=Child [b=0, super=Parent [a=0]]]
> > > > > > > > > > for classes with inheritance that use
> > > >
> > > > S.toString(SomeClass.class,
> > > > > > > > this, super.toString()) embedded to some other object.
> > > > > > > > > >
> > > > > > > > > > Dmitrii Ryabov, I've reverted two commits related to
> > IGNITE-602
> > > > >
> > > > > and
> > > > > > > > IGNITE-9209 tickets locally and it fixes the issue. Can you
> > take a
> > > > >
> > > > > look
> > > > > > > at
> > > > > > > > the issue?
> > > > > > > > > >
> > > > > > > > > > I think this regression essentially makes our logs
> > unreadable
> > > >
> > > > in
> > > > > > some
> > > > > > > > cases and I would like to get it fixed in ignite-2.7 or
> revert
> > both
> > > > > > >
> > > > > > > commits
> > > > > > > > from the release.
> > > > > > > > > >
> > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-10301
> > > > > > > > > >
> > > > > > > > > > пт, 9 нояб. 2018 г. в 09:22, Nikolay Izhikov <
> > > > >
> > > > > nizhi...@apache.org
> > > > > > > :
> > > > > > > > > > >
> > > > > > > > > > > Hello, Igniters.
> > > > > > > > > > >
> > > > > > > > > > > We still have 5 tickets for 2.7:
> > > > > > > > > > >
> > > > > > > > > > > IGNITE-10052Andrew Mashenkov Restart node during TX
> > > >
> > > > causes
> > > > > > > > vacuum error.
> > > > > > > > > > > IGNITE-10170Unassigned   .NET:
> > > > >
> > > > > Services.ServicesTestAsync
> > > > > > > > fails
> > > > > > > > > > > IGNITE-10196Maxim Pudov  Remove
> > kafka-clients-*-test
> > > > > > > >
> > > > > > > > dependency
> > > > > > > > > > > IGNITE-10154Andrey Gura  Critical worker
> liveness
> > > >
> > > > check
> > > > > > > > configuration is non-trivial and inconsistent
> > > > > > > > > > > IGNITE-9996 Nikolay Izhikov  Investigate possible
> > > > >
> > > > > performance
> > > > > > > > drop in FSYNC mode for ignite-2.7 compared to ignite-2.6
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > В Чт, 08/11/2018 в 14:25 +0300, Nikolay Izhikov пишет:
> > > > > > > > > > > > I'm OK with this.
> > > > > > > > > > > >
> > 

[GitHub] ignite pull request #5421: IGNITE-8765: Fixed event's query type

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5421


---


[GitHub] ignite pull request #5336: IGNITE-10159: IgniteCacheAbstractQuerySelfTest.te...

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5336


---


[GitHub] ignite pull request #5362: IGNITE-9996: comment out pageSize for encmgr

2018-11-20 Thread nizhikov
Github user nizhikov closed the pull request at:

https://github.com/apache/ignite/pull/5362


---


Re: Service grid redesign

2018-11-20 Thread Vyacheslav Daradur
Denis, Yakov have you had a chance to review the solution?

Igniters, we need to define a list of reviewers, otherwise no end in sign.

I'm ready to continue work on the Service Grid, including new features
like hot-redeployment and versioning, also, I have ideas about new
tools for monitoring and management which will be useful for our
end-users.

But for continuing work we need to overcome this first phase.

On Tue, Nov 13, 2018 at 1:09 PM Vyacheslav Daradur  wrote:
>
> Denis, Yakov, feel free to contact me directly in case of questions. Thanks!
>
> On Sun, Nov 11, 2018 at 10:09 PM Denis Mekhanikov  
> wrote:
> >
> > Guys,
> >
> > I'd like to take a look at the changes before they are merged.
> > I'll do my best to finish the review before the end of the upcoming week.
> >
> > Thanks!
> > Denis
> >
> > сб, 10 нояб. 2018 г. в 14:25, Nikolay Izhikov :
> >
> > > Hello, Vladimir.
> > >
> > > I'm agree with you.
> > >
> > > Can we write the list of reviewers for this feature?
> > > Without a date or similar.
> > > Just a list of experts who should review this feature.
> > >
> > > В Сб, 10/11/2018 в 14:01 +0300, Vladimir Ozerov пишет:
> > > > Igniters,
> > > >
> > > > This is very huge thing with complex algorithms behind. We should not
> > > merge
> > > > it to the product unless several additional thorough reviews are ready,
> > > > irrespectively of how long will it take. We are about quality, not 
> > > > speed.
> > > >
> > > > сб, 10 нояб. 2018 г. в 1:30, Denis Magda :
> > > >
> > > > > Vyacheslav,
> > > > >
> > > > > What are the cases when the service can be redeployed? Affinity,
> > > failure,
> > > > > etc., right. It would be good to list all the cases on the wiki and
> > > then
> > > > > our tech writers will get everything documented.
> > > > >
> > > > > --
> > > > > Denis
> > > > >
> > > > > On Thu, Nov 8, 2018 at 11:06 PM Vyacheslav Daradur <
> > > daradu...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Denis,
> > > > > >
> > > > > > Services reassignment process takes into account previous 
> > > > > > assignments
> > > > > > to avoid redundant redeployments.
> > > > > > So, in the described case, ServiceA won't be moved from node1 to
> > > node2.
> > > > > > On Fri, Nov 9, 2018 at 4:41 AM Denis Magda 
> > > wrote:
> > > > > > >
> > > > > > > Vyacheslav,
> > > > > > >
> > > > > > > First of all, thanks for archiving this milestone and rolling out
> > > these
> > > > > >
> > > > > > new
> > > > > > > capabilities.
> > > > > > >
> > > > > > > Speaking of the topology change events [1], does the new
> > > architecture
> > > > > >
> > > > > > avoid
> > > > > > > a running service redeployment when a new node joins? For 
> > > > > > > instance,
> > > > >
> > > > > let's
> > > > > > > say I have ServiceA running node1, then node2 joins and I don't
> > > want
> > > > >
> > > > > the
> > > > > > > service to be redeployed to any other node.
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > >
> > > > >
> > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584#ServiceGridredesign.Phase1.Implementationdetails.-Topologychange
> > > > > > >
> > > > > > > --
> > > > > > > Denis
> > > > > > >
> > > > > > > On Wed, Nov 7, 2018 at 7:04 AM Vyacheslav Daradur <
> > > daradu...@gmail.com
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dmitriy, I published documentation in wiki:
> > > > > > > >
> > > > >
> > > > >
> > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584
> > > > > > > >
> > > > > > > > Thank you!
> > > > > > > > On Wed, Nov 7, 2018 at 5:10 PM Dmitriy Pavlov <
> > > dpavlov@gmail.com
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Hi I think wiki is better than any attached docs. Could you
> > > please
> > > > > > > >
> > > > > > > > create a
> > > > > > > > > page?
> > > > > > > > >
> > > > > > > > > ср, 7 нояб. 2018 г., 14:39 Vyacheslav Daradur <
> > > daradu...@gmail.com
> > > > > >
> > > > > > :
> > > > > > > > >
> > > > > > > > > > I prepared a description of the implemented solution and
> > > attached
> > > > > >
> > > > > > it
> > > > > > > > > > to the issue [1].
> > > > > > > > > >
> > > > > > > > > > This should help during a review. Should I post the document
> > > into
> > > > > >
> > > > > > wiki
> > > > > > > > or
> > > > > > > > > > IEP?
> > > > > > > > > >
> > > > > > > > > > I'd like to ask Ignite's experts review the solution [1] 
> > > > > > > > > > [2],
> > > > > >
> > > > > > please?
> > > > > > > > > >
> > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-9607
> > > > > > > > > > [2] https://github.com/apache/ignite/pull/4434
> > > > > > > > > > On Wed, Oct 31, 2018 at 5:04 PM Vyacheslav Daradur <
> > > > > > > >
> > > > > > > > daradu...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi, Igniters! Good news!
> > > > > > > > > > >
> > > > > > > > > > > Service Grid Redesign Phase 1 - is in Patch Available now.
> > > > > > > > > > >
> > > > > > > > > > > 

Re: proposed realization KILL QUERY command

2018-11-20 Thread Юрий
Hi Vladimir,

Thanks for your suggestion to use MANAGEMENT_POOL for processing
cancellation requests.

About your questions.
1) I'm going to implements SQL view to provide list of running queries. The
SQL VIEW has been a little bit discussed earlier. Proposed name is
*running_queries* with following columns: query_id, node_id, sql,
schema_name, connection_id, duration. Currently most of the information can
be  retrieved through cache API, however it doesn't matter, any case we
need to expose SQL VIEW. Seem's you are right - the part should be
implemented firstly.
2) Fully agree that we need to support all kind of SQL queries
(SLECT/DML/DDL, transactional, non transnational, local, distributed). I
definitely sure that it will possible for all of above, however I'm not
sure about DDL - need to investigate it deeper. Also need to understand
that canceled DML operation can lead to partially updated data for non
transational caches.



пн, 19 нояб. 2018 г. в 19:17, Vladimir Ozerov :

> Hi Yuriy,
>
> I think we can use MANAGEMENT_POOL for this. It is already used for some
> internal Ignite tasks, and it appears to be a good candidate to process
> cancel requests.
>
> But there are several things which are not clear enough for me at the
> moment:
> 1) How user is going to get the list of running queries in the first place?
> Do we already have any SQL commands/views to get this information?
> 2) We need to ensure that KILL command will be processed properly by all
> kinds of SQL queries - SELECT/DML/DDL, non-transactional or transactional,
> local queries and distributed queries. Will we be able to support all these
> modes?
>
> Vladimir.
>
> On Mon, Nov 19, 2018 at 6:37 PM Юрий  wrote:
>
> > Hi Igniters,
> >
> > Earlier we agreed about syntax KILL QUERY '[node_order].[query_counter]',
> > e.g. KILL QUERY '25.123' for single query  or KILL QUERY '25.*' for all
> > queries on the node. Which is part of IEP-29
> > <
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-29%3A+SQL+management+and+monitoring
> > >
> > .
> >
> > Now I want to discuss internal realization of KILL query feature.
> >
> > My current vision is following:
> > After parsing, Ignite create KILL query command with two parameters:
> > nodeOrderId, nodeQryId. To determine that need to kill all queries on a
> > node we can use negative value of query id, due to qry id always have
> > positive values.
> > The command process at IgniteH2Indexing as native command.
> > By nodeOrderId we find node which initial for the query and send to the
> > node new GridQueryKillRequest with nodeQryId to TOPIC_QUERY with not
> QUERY
> > POOL executor.
> > At GridReduceQueryExecutor we add support of processing new
> > GridQueryKillRequest
> > which just run already exists cancelQueries method with given qryId or
> with
> > all qryIds which currently running at the node in case at initial KILL
> > QUERY parameters used star symbol.
> >
> > I have a doubt which of thread pool we should use to process
> > GridQueryKillRequest.
> > My opinion it shouldn't be QUERY pool, due to the pool can be fully used
> by
> > executing queries, it such case we can't cancel query immediately. May we
> > use one of already existed pool or create new one? Or may be I'm mistaken
> > and it should use QUERY pool.
> >
> > What do you think about proposed plan of implementation?
> >
> > And please give comments about which of thread pool will be better to use
> > for kill query requests. It's small, but really important part of the
> > realization.
> >
> >
> > Thanks.
> >
> >
> > --
> > Живи с улыбкой! :D
> >
>


-- 
Живи с улыбкой! :D


[jira] [Created] (IGNITE-10339) Connection to cluster failed in control.sh while using --cache

2018-11-20 Thread Ivan Daschinskiy (JIRA)
Ivan Daschinskiy created IGNITE-10339:
-

 Summary: Connection to cluster failed in control.sh while using 
--cache
 Key: IGNITE-10339
 URL: https://issues.apache.org/jira/browse/IGNITE-10339
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.6
Reporter: Ivan Daschinskiy
Assignee: Ivan Daschinskiy
 Fix For: 2.8






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Apache Ignite 2.7. Last Mile

2018-11-20 Thread Vladimir Ozerov
+1 for reverting both.

On Tue, Nov 20, 2018 at 9:43 AM Nikolay Izhikov  wrote:

> Hello, Dmitrii.
>
> I see 2 tickets for this improvement:
>
> IGNITE-602 - [Test] GridToStringBuilder is vulnerable for
> StackOverflowError caused by infinite recursion [1]
> IGNITE-9209 - GridDistributedTxMapping.toString() returns broken string [2]
>
> Should we revert both commits?
>
> [1] https://github.com/apache/ignite/commit/d67c5bf
> [2] https://github.com/apache/ignite/commit/9bb9c04
>
>
> В Пн, 19/11/2018 в 13:36 +0300, Dmitrii Ryabov пишет:
> > I agree to revert and make fix for 2.8. So, we will have more time to
> test
> > it.
> >
> > пн, 19 нояб. 2018 г., 10:53 Vladimir Ozerov voze...@gridgain.com:
> >
> > > +1 for revert.
> > >
> > > On Sun, Nov 18, 2018 at 11:31 PM Dmitriy Pavlov 
> > > wrote:
> > >
> > > > I personally don't mind.
> > > >
> > > > But I would like Dmitry Ryabov and Alexey Goncharuck share their
> > >
> > > opinions.
> > > >
> > > > вс, 18 нояб. 2018 г., 20:43 Nikolay Izhikov :
> > > >
> > > > > Yes, I think so.
> > > > >
> > > > > вс, 18 нояб. 2018 г., 20:34 Denis Magda dma...@apache.org:
> > > > >
> > > > > > Sounds good to me. Are we starting the vote then?
> > > > > >
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Sun, Nov 18, 2018 at 8:25 AM Nikolay Izhikov <
> nizhi...@apache.org
> > > > > > wrote:
> > > > > >
> > > > > > > Hello, Igniters.
> > > > > > >
> > > > > > > This issue is the only ticket that blocks 2.7 release.
> > > > > > >
> > > > > > > I looked at IGNITE-602 PR and GridToStringBuilder.
> > > > > > > The code looks complicated for me.
> > > > > > > And it's not obvious for me how to fix this issue in a short
> period
> > > >
> > > > of
> > > > > > > time.
> > > > > > > Especially, code deals with recursion and other things that can
> > >
> > > lead
> > > > to
> > > > > > > very dangerous errors.
> > > > > > >
> > > > > > > Let's revert this patch and fix it in calmly.
> > > > > > > Also, we need additional tests for it.
> > > > > > >
> > > > > > > В Пт, 16/11/2018 в 17:57 +0300, Dmitrii Ryabov пишет:
> > > > > > > > Ok, I'll check the issue.
> > > > > > > > пт, 16 нояб. 2018 г. в 17:52, Alexey Goncharuk <
> > > > > > >
> > > > > > > alexey.goncha...@gmail.com>:
> > > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > I've just found that S.toString() implementation is broken
> in
> > > > > > >
> > > > > > > ignite-2.7 and master [1]. It leads to a message
> > > > > > > > > Wrapper [p=Parent [a=0]Child [b=0, super=]]
> > > > > > > > > being formed instead of
> > > > > > > > > Wrapper [p=Child [b=0, super=Parent [a=0]]]
> > > > > > > > > for classes with inheritance that use
> > >
> > > S.toString(SomeClass.class,
> > > > > > > this, super.toString()) embedded to some other object.
> > > > > > > > >
> > > > > > > > > Dmitrii Ryabov, I've reverted two commits related to
> IGNITE-602
> > > >
> > > > and
> > > > > > > IGNITE-9209 tickets locally and it fixes the issue. Can you
> take a
> > > >
> > > > look
> > > > > > at
> > > > > > > the issue?
> > > > > > > > >
> > > > > > > > > I think this regression essentially makes our logs
> unreadable
> > >
> > > in
> > > > > some
> > > > > > > cases and I would like to get it fixed in ignite-2.7 or revert
> both
> > > > > >
> > > > > > commits
> > > > > > > from the release.
> > > > > > > > >
> > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-10301
> > > > > > > > >
> > > > > > > > > пт, 9 нояб. 2018 г. в 09:22, Nikolay Izhikov <
> > > >
> > > > nizhi...@apache.org
> > > > > > :
> > > > > > > > > >
> > > > > > > > > > Hello, Igniters.
> > > > > > > > > >
> > > > > > > > > > We still have 5 tickets for 2.7:
> > > > > > > > > >
> > > > > > > > > > IGNITE-10052Andrew Mashenkov Restart node during TX
> > >
> > > causes
> > > > > > > vacuum error.
> > > > > > > > > > IGNITE-10170Unassigned   .NET:
> > > >
> > > > Services.ServicesTestAsync
> > > > > > > fails
> > > > > > > > > > IGNITE-10196Maxim Pudov  Remove
> kafka-clients-*-test
> > > > > > >
> > > > > > > dependency
> > > > > > > > > > IGNITE-10154Andrey Gura  Critical worker liveness
> > >
> > > check
> > > > > > > configuration is non-trivial and inconsistent
> > > > > > > > > > IGNITE-9996 Nikolay Izhikov  Investigate possible
> > > >
> > > > performance
> > > > > > > drop in FSYNC mode for ignite-2.7 compared to ignite-2.6
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > В Чт, 08/11/2018 в 14:25 +0300, Nikolay Izhikov пишет:
> > > > > > > > > > > I'm OK with this.
> > > > > > > > > > >
> > > > > > > > > > > чт, 8 нояб. 2018 г., 13:44 Andrey Gura
> ag...@apache.org:
> > > > > > > > > > > > Long, long way to release :)
> > > > > > > > > > > >
> > > > > > > > > > > > Guys, we have a breaking change in Ignite 2.7 so we
> must
> > > >
> > > > add
> > > > > > > > > > > > IGNITE-10154 [1] fix to the release.
> > > > > > > > > > > >
> > > > > > > > > > > > [1]
> 

[GitHub] ololo3000 opened a new pull request #76: IGNITE-10275 Refactor of visa caching,

2018-11-20 Thread GitBox
ololo3000 opened a new pull request #76: IGNITE-10275 Refactor of visa caching,
URL: https://github.com/apache/ignite-teamcity-bot/pull/76
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


Re: [IMPORTANT] Future of Binary Objects

2018-11-20 Thread Sergi Vladykin
I really like Protobuf format. It is probably not what we need for O(1)
fields access,
but for compact data representation we can derive lots from there.

Also IMO, restricting field type change is absolutely sane idea.
The correct way to evolve schema in common case is to add new fields and
gradually
deprecate the old ones, if you can skip default/null fields in binary
format this approach
will not introduce any noticeable performance/size overhead.

Sergi

вт, 20 нояб. 2018 г. в 11:12, Vyacheslav Daradur :

> I think, one of a possible way to reduce overhead and TCO - SQL Scheme
> approach.
>
> That assumes that metadata will be stored separately from serialized
> data to reduce size.
> In this case, the most advantages of Binary Objects like access in
> O(1) and access without deserialization may be achieved.
> On Tue, Nov 20, 2018 at 10:56 AM Vladimir Ozerov 
> wrote:
> >
> > Hi Alexey,
> >
> > Binary Objects only.
> >
> > On Tue, Nov 20, 2018 at 10:50 AM Alexey Zinoviev  >
> > wrote:
> >
> > > Do we discuss here Core features only or the roadmap for all
> components?
> > >
> > > вт, 20 нояб. 2018 г. в 10:05, Vladimir Ozerov :
> > >
> > > > Igniters,
> > > >
> > > > It is very likely that Apache Ignite 3.0 will be released next year.
> So
> > > we
> > > > need to start thinking about major product improvements. I'd like to
> > > start
> > > > with binary objects.
> > > >
> > > > Currently they are one of the main limiting factors for the product.
> They
> > > > are fat - 30+ bytes overhead on average, high TCO of Apache Ignite
> > > > comparing to other vendors. They are slow - not suitable for SQL at
> all.
> > > >
> > > > I would like to ask all of you who worked with binary objects to
> share
> > > your
> > > > feedback and ideas, so that we understand how they should look like
> in AI
> > > > 3.0. This is a brain storm - let's accumulate ideas first and
> minimize
> > > > critics. Then we will work on ideas in separate topics.
> > > >
> > > > 1) Historical background
> > > >
> > > > BO were implemented around 2014 (Apache Ignite 1.5) when we started
> > > working
> > > > on .NET and CPP clients. During design we had several ideas in mind:
> > > > - ability to read object fields in O(1) without deserialization
> > > > - interoperabillty between Java, .NET and CPP.
> > > >
> > > > Since then a number of other concepts were mixed to the cocktail:
> > > > - Affinity key fields
> > > > - Strict typing for existing fields (aka metadata)
> > > > - Binary Object as storage format
> > > >
> > > > 2) My proposals
> > > >
> > > > 2.1) Introduce "Data Row Format" interface
> > > > Binary Objects are terrible candidates for storage. Too fat, too
> slow.
> > > > Efficient storage typically has <10 bytes overhead per row (no
> metadata,
> > > no
> > > > length, no hash code, etc), allow supper-fast field access, support
> > > > different string formats (ASCII, UTF-8, etc), support different
> temporal
> > > > types (date, time, timestamp, timestamp with timezone, etc), and
> store
> > > > these types as efficiently as possible.
> > > >
> > > > What we need is to introduce an interface which will convert a pair
> of
> > > > key-value objects into a row. This row will be used to store data
> and to
> > > > get fields from it. Care about memory consumption, need SQL and
> strict
> > > > schema - use one format. Need flexibility and prefer key-value
> access -
> > > use
> > > > another format which will store binary objects unchanged (current
> > > > behavior).
> > > >
> > > > interface DataRowFormat {
> > > > DataRow create(Object key, Object value); // primitives or binary
> > > > objects
> > > > DataRowMetadata metadata();
> > > > }
> > > >
> > > > 2.2) Remove affinity field from metadata
> > > > Affinity rules are governed by cache, not type. We should remove
> > > > "affintiyFieldName" from metadata.
> > > >
> > > > 2.3) Remove restrictions on changing field type
> > > > I do not know why we did that in the first place. This restriction
> > > prevents
> > > > type evolution and confuses users.
> > > >
> > > > 2.4) Use bitmaps for "null" and default values and for fixed-length
> > > fields,
> > > > put fixed-length fields before variable-length.
> > > > Motivation: to save space.
> > > >
> > > > What else? Please share your ideas.
> > > >
> > > > Vladimir.
> > > >
> > >
>
>
>
> --
> Best Regards, Vyacheslav D.
>


[GitHub] ignite pull request #5347: IGNITE-10162: IgniteCacheAtomicQuerySelfTest#test...

2018-11-20 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5347


---


Re: [IMPORTANT] Future of Binary Objects

2018-11-20 Thread Vyacheslav Daradur
I think, one of a possible way to reduce overhead and TCO - SQL Scheme approach.

That assumes that metadata will be stored separately from serialized
data to reduce size.
In this case, the most advantages of Binary Objects like access in
O(1) and access without deserialization may be achieved.
On Tue, Nov 20, 2018 at 10:56 AM Vladimir Ozerov  wrote:
>
> Hi Alexey,
>
> Binary Objects only.
>
> On Tue, Nov 20, 2018 at 10:50 AM Alexey Zinoviev 
> wrote:
>
> > Do we discuss here Core features only or the roadmap for all components?
> >
> > вт, 20 нояб. 2018 г. в 10:05, Vladimir Ozerov :
> >
> > > Igniters,
> > >
> > > It is very likely that Apache Ignite 3.0 will be released next year. So
> > we
> > > need to start thinking about major product improvements. I'd like to
> > start
> > > with binary objects.
> > >
> > > Currently they are one of the main limiting factors for the product. They
> > > are fat - 30+ bytes overhead on average, high TCO of Apache Ignite
> > > comparing to other vendors. They are slow - not suitable for SQL at all.
> > >
> > > I would like to ask all of you who worked with binary objects to share
> > your
> > > feedback and ideas, so that we understand how they should look like in AI
> > > 3.0. This is a brain storm - let's accumulate ideas first and minimize
> > > critics. Then we will work on ideas in separate topics.
> > >
> > > 1) Historical background
> > >
> > > BO were implemented around 2014 (Apache Ignite 1.5) when we started
> > working
> > > on .NET and CPP clients. During design we had several ideas in mind:
> > > - ability to read object fields in O(1) without deserialization
> > > - interoperabillty between Java, .NET and CPP.
> > >
> > > Since then a number of other concepts were mixed to the cocktail:
> > > - Affinity key fields
> > > - Strict typing for existing fields (aka metadata)
> > > - Binary Object as storage format
> > >
> > > 2) My proposals
> > >
> > > 2.1) Introduce "Data Row Format" interface
> > > Binary Objects are terrible candidates for storage. Too fat, too slow.
> > > Efficient storage typically has <10 bytes overhead per row (no metadata,
> > no
> > > length, no hash code, etc), allow supper-fast field access, support
> > > different string formats (ASCII, UTF-8, etc), support different temporal
> > > types (date, time, timestamp, timestamp with timezone, etc), and store
> > > these types as efficiently as possible.
> > >
> > > What we need is to introduce an interface which will convert a pair of
> > > key-value objects into a row. This row will be used to store data and to
> > > get fields from it. Care about memory consumption, need SQL and strict
> > > schema - use one format. Need flexibility and prefer key-value access -
> > use
> > > another format which will store binary objects unchanged (current
> > > behavior).
> > >
> > > interface DataRowFormat {
> > > DataRow create(Object key, Object value); // primitives or binary
> > > objects
> > > DataRowMetadata metadata();
> > > }
> > >
> > > 2.2) Remove affinity field from metadata
> > > Affinity rules are governed by cache, not type. We should remove
> > > "affintiyFieldName" from metadata.
> > >
> > > 2.3) Remove restrictions on changing field type
> > > I do not know why we did that in the first place. This restriction
> > prevents
> > > type evolution and confuses users.
> > >
> > > 2.4) Use bitmaps for "null" and default values and for fixed-length
> > fields,
> > > put fixed-length fields before variable-length.
> > > Motivation: to save space.
> > >
> > > What else? Please share your ideas.
> > >
> > > Vladimir.
> > >
> >



-- 
Best Regards, Vyacheslav D.


Re: [ANNOUNCE] Welcome Igor Seliverstov as a new committer

2018-11-20 Thread Павлухин Иван
Igor my congratulations! Keep going!

2018-11-20 9:01 GMT+01:00, Vladimir Ozerov :
> The Apache Ignite Project Management Committee (PMC) has invited Igor
> Seliverstov to become a new committer and are happy to announce that he has
> accepted.
>
> Igor contributed a lot to "Transactional SQL" feature.
>
> Please join me in welcoming Igor and congratulating him on his new role in
> the Apache Ignite Community.
>
> Good luck, Igor!
>
> Vladimir.
>


-- 
Best regards,
Ivan Pavlukhin


[ANNOUNCE] Welcome Igor Seliverstov as a new committer

2018-11-20 Thread Vladimir Ozerov
The Apache Ignite Project Management Committee (PMC) has invited Igor
Seliverstov to become a new committer and are happy to announce that he has
accepted.

Igor contributed a lot to "Transactional SQL" feature.

Please join me in welcoming Igor and congratulating him on his new role in
the Apache Ignite Community.

Good luck, Igor!

Vladimir.