[GitHub] ignite pull request #3154: IGNITE-369 Runtime statistics enable

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3154


---

[GitHub] ignite pull request #3729: IGNITE-8069 IgniteOutOfMemoryException should be ...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3729


---

[GitHub] ignite pull request #3749: IGNITE-6892 OOM should be covered by failure hand...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3749


---

[GitHub] ignite pull request #3474: IGNITE-5265 Eviction Rate memory metric

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3474


---

[GitHub] ignite pull request #3369: IGNITE-6923 Cache metrics optimization

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3369


---

[GitHub] ignite pull request #3630: IGNITE-7831 Throw Exceptions instead of Assertion...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3630


---

[GitHub] ignite pull request #3195: JMX metrics implementation

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3195


---

[GitHub] ignite pull request #3349: IGNITE-7346: Enable Ignite cache events per cache

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3349


---

[GitHub] ignite pull request #3146: IGNITE-6867 JMX metrics for topology monitoring -...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3146


---

[GitHub] ignite pull request #3139: IGNITE-6871 Group name added to group metrics MXB...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3139


---

[GitHub] ignite pull request #3123: IGNITE-6868 nodeId type fixed, unused methods rem...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3123


---

[GitHub] ignite pull request #3051: IGNITE-6870 Added new JMX metric for cache topolo...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3051


---

[GitHub] ignite pull request #3099: IGNITE-6871 Implement new JMX metrics for partiti...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3099


---

[GitHub] ignite pull request #3034: IGNITE-6868 Added new JMX metrics for TcpCommunic...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3034


---

[GitHub] ignite pull request #3053: IGNITE-6869 Added new JMX metric: Total jobs exec...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3053


---

[GitHub] ignite pull request #3080: IGNITE-6985 Wrong value of RebalancingStartTime c...

[alex-plekhanov]

Github user alex-plekhanov closed the pull request at:

https://github.com/apache/ignite/pull/3080


---

[jira] [Created] (IGNITE-8287) Web console: place sign up inputs in two columns

[Ilya Borisov (JIRA)]

Ilya Borisov created IGNITE-8287:


 Summary: Web console: place sign up inputs in two columns
 Key: IGNITE-8287
 URL: https://issues.apache.org/jira/browse/IGNITE-8287
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov


The sign up page controls take a lot of vertical space, let's put some of these 
side by side, like on configuration and user profile pages.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3836: IGNITE-8201 Refactor REST API for authentication.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3836


---

[jira] [Created] (IGNITE-8286) ScanQuery ignore setLocal with non local partition

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-8286:


 Summary: ScanQuery ignore setLocal with non local partition
 Key: IGNITE-8286
 URL: https://issues.apache.org/jira/browse/IGNITE-8286
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Alexander Belyak


1) Create partitioned cache on 2+ nodes cluster
2) Select some partition N, local node should not be OWNER of partition N
3) execute: cache.query(new ScanQuery<>().setLocal(true).setPartition(N))
Expected result:
empty result (probaply with logging smth like "Trying to execute local query 
 with non local partition N") or even throw excedption
Actual result:
executing (with ScanQueryFallbackClosableIterator) query on remote node.
Problem is that we execute local query on remote node.
Same behaviour can be achieved if we get empty node list from 
GridCacheQueryAdapter.node() by any reasons, for example - if we run "local" 
query from non data node from given cache (see 
GridDiscoveryNamager.cacheAffinityNode(ClusterNode node, String cacheName) in 
GridcacheQueryAdapter.executeScanQuery()



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Service grid redesign

[Valentin Kulichenko]

I agree we shouldn't do anything synchronously within discovery threads. If
something goes wrong, we just need to properly notify the user, logging and
events seem to be right options to achieve that.

BTW, with this design I'm not sure init() method makes sense, probably we
should deprecate it.

-Val

On Mon, Apr 16, 2018 at 12:03 PM, Denis Magda  wrote:

> Denis,
>
> In general, service initialization shouldn't prevent a node from joining
> the cluster or slowing down that process. Thus, I would start the
> initialization routines only after the node is accepted by the cluster. If
> the initialization fails then we need to report a respective message to the
> logs and, probably, generate a system event the user can be subscribed to.
>
> Regardless, of the service initialization time, I think we still need to
> utilize discovery SPI to avoid problems discussed later.
>
> Val, others, what do you think about that?
>
>
> --
> Denis
>
> On Mon, Apr 16, 2018 at 10:29 AM, Denis Mekhanikov 
> wrote:
>
> > Basically, my question is: at which moment services should be deployed on
> > connecting nodes?
> > Should we reject a node from being included into a topology, if services,
> > that are assigned to it, fail to deploy?
> >
> > It would be good to be sure, that all assigned services are initialised
> and
> > working, when node start is finished.
> > Otherwise it's unclear, how to notify a user about failures in service
> > initialisation on new nodes.
> >
> > If we decide to provide such guarantee, then how are we going to do that?
> > Is procedure, that I described, viable?
> > It requires hacking through the discovery protocol, which is a thing,
> that
> > should be avoided.
> > So, maybe there is another way to achieve the same thing?
> >
> > Denis
> >
> > сб, 14 апр. 2018 г. в 1:48, Denis Magda :
> >
> > > It sounds like it's not a trivial thing to support the automatic
> services
> > > redeployment after a restart. Let's postpone it for now, guys
> > concentrating
> > > on more urgent things related to the services.
> > >
> > > Alex, Vladimir,
> > >
> > > Could you have a look at Denis question about the discovery-based
> > > deployment? Guess it's the only one thing that prevents us from the IEP
> > > finalization.
> > >
> > > --
> > > Denis
> > >
> > > On Fri, Apr 13, 2018 at 5:30 AM, Denis Mekhanikov <
> dmekhani...@gmail.com
> > >
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > Currently we don't save binary metadata to disk, when persistence is
> > > > disabled.
> > > > But we still persist marshaller mappings for some reason, and I
> > > personally
> > > > believe, that we shouldn't.
> > > >
> > > > But I agree, that we should separate data and service persistence
> > > > configuration.
> > > > Right now persistence of services is configured in a pretty
> non-obvious
> > > > manner.
> > > > It should be a clear way to tell Ignite, whether you want services to
> > be
> > > > persisted or not.
> > > >
> > > > I'm not sure, that we should make "statefullness" in general
> > > configurable.
> > > > Users don't care much, whether metadata is preserved on restarts, or
> > not.
> > > >
> > > > Denis
> > > >
> > > > пт, 13 апр. 2018 г. в 14:29, Vladimir Ozerov :
> > > >
> > > > > Alex,
> > > > >
> > > > > I would say that we've already had this behavior for years -
> > marshaller
> > > > > cache. I think it is time to agree that "in-memory" != stateless.
> > > Instead
> > > > > "in-memory" means "data is not stored on disk".
> > > > > May be we can have a flag which will wipe out all metadata on node
> > > > restart
> > > > > (e.g. it could make sense for embedded clients)?
> > > > >
> > > > > On Fri, Apr 13, 2018 at 12:48 PM, Alexey Goncharuk <
> > > > > alexey.goncha...@gmail.com> wrote:
> > > > >
> > > > > > Denis,
> > > > > >
> > > > > > This is a subtle question. It looks like we have now a number of
> > > > > use-cases
> > > > > > when persistent storage is required even for a pure in-memory
> mode.
> > > One
> > > > > of
> > > > > > the use-cases is thin client authentication, the other is service
> > > grid
> > > > > > configuration persistence.
> > > > > >
> > > > > > Generally, I would agree that this is an expected behavior.
> > However,
> > > > this
> > > > > > means that a user cannot simply start and stop nodes randomly
> > > anymore.
> > > > > > Ignite start will require some sort of installation or work
> folder
> > > > > > initialization (sort of initdb in postgres) which is ok for
> > > > > > persistence-enabled modes, but I am not sure if this is expected
> > for
> > > > > > in-memory. Of course, we can run this initialization
> automatically,
> > > but
> > > > > it
> > > > > > is not always a good idea.
> > > > > >
> > > > > > If we are ok to have this restrictions for in-memory mode, then
> > > service
> > > > > > persistence makes sense.
> > > > > >
> > > > > > --AG
> > > > > >
> > > > > > 

[GitHub] ignite pull request #3836: IGNITE-8201 Refactor REST API for authentication.

[nva]

GitHub user nva opened a pull request:

https://github.com/apache/ignite/pull/3836

IGNITE-8201 Refactor REST API for authentication.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/ignite ignite-8201

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3836.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3836


commit 5a1d8a059276df3ffb709c8b39781b91045ff6fa
Author: Alexey Kuznetsov 
Date:   2018-04-10T10:37:09Z

IGNITE-8201 WIP

commit a8089323273f0ac3e8d5eb9f7460686978414535
Author: Alexey Kuznetsov 
Date:   2018-04-10T13:43:12Z

IGNITE-8201 WIP.

commit df938d562e9fb59938c1628be2163a5c273d21d4
Author: Alexey Kuznetsov 
Date:   2018-04-11T07:00:06Z

Merge branches 'ignite-8201' and 'master' of 
https://git-wip-us.apache.org/repos/asf/ignite into ignite-8201

commit 86d0bcb7d134686fb521dd7da5dc0e583d7f917c
Author: Alexey Kuznetsov 
Date:   2018-04-11T08:19:58Z

IGNITE-8201 WIP.

commit fe21ed5710396c6b2cb7b9cbb1f8f115e697dc49
Author: Alexey Kuznetsov 
Date:   2018-04-11T10:39:20Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-8201

commit 3780ac0a09198e2918206d23ee515a2fe522352c
Author: Alexey Kuznetsov 
Date:   2018-04-11T12:28:11Z

IGNITE-8201 WIP.

commit 6c01882d7ef6a94bbb0b6f10d5290cb1a1e4a04f
Author: Alexey Kuznetsov 
Date:   2018-04-11T17:29:56Z

IGNITE-8201 Fixed tests.

commit 4ba2436f6d01007e939a6ca409ee3cb1aa85af50
Author: Alexey Kuznetsov 
Date:   2018-04-11T17:32:20Z

Merge branches 'ignite-8201' and 'master' of 
https://git-wip-us.apache.org/repos/asf/ignite into ignite-8201

commit 3316080a67edb27a7ee6c35dcd2579be8ecc8dcb
Author: Alexey Kuznetsov 
Date:   2018-04-12T03:32:31Z

IGNITE-8201 Fixed tests.

commit bc20f78270591eac5661db85018c07a83c5cb28b
Author: Alexey Kuznetsov 
Date:   2018-04-12T07:56:48Z

IGNITE-8201 Added check for token len.

commit f31452dc84e36ba3decda7de48cfc708bb000c02
Author: Alexey Kuznetsov 
Date:   2018-04-13T03:16:52Z

IGNITE-8201 REST: Fixed old API.




---

[jira] [Created] (IGNITE-8285) Web console: debug log not removed

[Ilya Borisov (JIRA)]

Ilya Borisov created IGNITE-8285:


 Summary: Web console: debug log not removed
 Key: IGNITE-8285
 URL: https://issues.apache.org/jira/browse/IGNITE-8285
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov


*How to reproduce:*
1. Open new cluster creation form.
2. Kill the backend.
3. Click "Save".

*What happens:*
"UNDOED {type: "APPLY_ACTIONS_UNDO", state: {…}}" is logged to browser console.

*What should happen:*
Nothing should get logged.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: affinityRun/Call in C++

[Lucas Beeler]

Hi Guys,

Having (very unfortunately) done a boatload of C++ programming while I was at 
university, there are some issues to consider here. One of the biggest is 
compiler name mangling 
.
 Now, I’m guessing you’ve heard about this, but it’s a lot nastier than people 
think it is. For example, you can’t link an object built with GCC on Linux to 
one built with LLVM on Linux (though this may be changing with the introduction 
of a “standard” Linux ABI and name mangling scheme, c.f. this Oracle TechNet 
article 
).
 Essentially, C++ object code built with different compilers—or even different 
versions of the same compiler—have totally different name mangling schemes and 
ABIs. So we’d need to pick one and stick with it. The Wikipedia article 
linked-to above explains this problem better than I can.

So we’d have to standardize on one compiler ABI, then build the native code 
portions of our C++-to-JVM bridge with that compiler, such that object code 
loaded from the client side during a compute invocation could be dynamically 
linked to our native code interface on the server side, since code built with 
different name mangling schemes and different ABIs can’t be linked together. 
The most obvious choice is the GCC/Linux ABI, since most of our users deploy on 
Linux and GCC comes as the standard compiler suite on most distros. What’s 
more, IIRC, it’s possible to force Visual Studio on Windows and LLVM on the Mac 
to output GCC Linux ABI-compliant code. This way, we’d give our users the 
capability to run unit tests in CppUnit, etc. on their development laptops and 
then have that same compiled object be able to work seamlessly when our server 
side code loads it and links it into the server C++ runtime.

Hope that all makes sense. I’m happy to contribute to this effort.

Take care,
Lucas

--
Lucas BEELER
Technical Consultant, Professional Services
GridGain Systems
www.gridgain.com

> On Apr 16, 2018, at 9:02 AM, Igor Sapego  wrote:
> 
> It depends on which functionality is needed, as there are
> several versions of those methods. Some would require
> implementing of Cluster API, which is not present in C++ also.
> 
> Best Regards,
> Igor
> 
> On Mon, Apr 16, 2018 at 6:41 PM, Denis Magda  wrote:
> 
>> Do we target it for 2.6 release? It shouldn't be time-consuming to support
>> it, right?
>> 
>> --
>> Denis
>> 
>> On Mon, Apr 16, 2018 at 2:48 AM, Igor Sapego  wrote:
>> 
>>> Guys,
>>> 
>>> I've created a ticket: [1]
>>> 
>>> [1] - https://issues.apache.org/jira/browse/IGNITE-8273
>>> 
>>> Best Regards,
>>> Igor
>>> 
>>> On Wed, Apr 11, 2018 at 2:44 AM, Valentin Kulichenko <
>>> valentin.kuliche...@gmail.com> wrote:
>>> 
 Is there a ticket? Let's create one if not.
 
 -Val
 
 On Tue, Apr 10, 2018 at 6:17 AM, Vladimir Ozerov >> 
 wrote:
 
> Val,
> 
> They are simply not implemented yet. I am not aware of concrete plans
>>> to
> support them.
> 
> On Mon, Apr 9, 2018 at 11:33 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> 
>> Guys,
>> 
>> Is there a way to run collocated compute job in C++? I can't find
>> affinityRun and affinityCall method in C++ compute API, am I
>> missing
>> something? If we really don't have them, is there any particular
>>> reason
> for
>> this and/or plans to add them?
>> 
>> -Val
>> 
> 
 
>>> 
>> 

Proposal

[MS Zeliha Omer Faruk]

Hello

Greeetings to you please did you get my previous email regarding my
investment proposal last week friday ?

MS.Zeliha ömer faruk
zeliha.omer.fa...@gmail.com

Re: Service grid redesign

[Denis Magda]

Denis,

In general, service initialization shouldn't prevent a node from joining
the cluster or slowing down that process. Thus, I would start the
initialization routines only after the node is accepted by the cluster. If
the initialization fails then we need to report a respective message to the
logs and, probably, generate a system event the user can be subscribed to.

Regardless, of the service initialization time, I think we still need to
utilize discovery SPI to avoid problems discussed later.

Val, others, what do you think about that?


--
Denis

On Mon, Apr 16, 2018 at 10:29 AM, Denis Mekhanikov 
wrote:

> Basically, my question is: at which moment services should be deployed on
> connecting nodes?
> Should we reject a node from being included into a topology, if services,
> that are assigned to it, fail to deploy?
>
> It would be good to be sure, that all assigned services are initialised and
> working, when node start is finished.
> Otherwise it's unclear, how to notify a user about failures in service
> initialisation on new nodes.
>
> If we decide to provide such guarantee, then how are we going to do that?
> Is procedure, that I described, viable?
> It requires hacking through the discovery protocol, which is a thing, that
> should be avoided.
> So, maybe there is another way to achieve the same thing?
>
> Denis
>
> сб, 14 апр. 2018 г. в 1:48, Denis Magda :
>
> > It sounds like it's not a trivial thing to support the automatic services
> > redeployment after a restart. Let's postpone it for now, guys
> concentrating
> > on more urgent things related to the services.
> >
> > Alex, Vladimir,
> >
> > Could you have a look at Denis question about the discovery-based
> > deployment? Guess it's the only one thing that prevents us from the IEP
> > finalization.
> >
> > --
> > Denis
> >
> > On Fri, Apr 13, 2018 at 5:30 AM, Denis Mekhanikov  >
> > wrote:
> >
> > > Vladimir,
> > >
> > > Currently we don't save binary metadata to disk, when persistence is
> > > disabled.
> > > But we still persist marshaller mappings for some reason, and I
> > personally
> > > believe, that we shouldn't.
> > >
> > > But I agree, that we should separate data and service persistence
> > > configuration.
> > > Right now persistence of services is configured in a pretty non-obvious
> > > manner.
> > > It should be a clear way to tell Ignite, whether you want services to
> be
> > > persisted or not.
> > >
> > > I'm not sure, that we should make "statefullness" in general
> > configurable.
> > > Users don't care much, whether metadata is preserved on restarts, or
> not.
> > >
> > > Denis
> > >
> > > пт, 13 апр. 2018 г. в 14:29, Vladimir Ozerov :
> > >
> > > > Alex,
> > > >
> > > > I would say that we've already had this behavior for years -
> marshaller
> > > > cache. I think it is time to agree that "in-memory" != stateless.
> > Instead
> > > > "in-memory" means "data is not stored on disk".
> > > > May be we can have a flag which will wipe out all metadata on node
> > > restart
> > > > (e.g. it could make sense for embedded clients)?
> > > >
> > > > On Fri, Apr 13, 2018 at 12:48 PM, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com> wrote:
> > > >
> > > > > Denis,
> > > > >
> > > > > This is a subtle question. It looks like we have now a number of
> > > > use-cases
> > > > > when persistent storage is required even for a pure in-memory mode.
> > One
> > > > of
> > > > > the use-cases is thin client authentication, the other is service
> > grid
> > > > > configuration persistence.
> > > > >
> > > > > Generally, I would agree that this is an expected behavior.
> However,
> > > this
> > > > > means that a user cannot simply start and stop nodes randomly
> > anymore.
> > > > > Ignite start will require some sort of installation or work folder
> > > > > initialization (sort of initdb in postgres) which is ok for
> > > > > persistence-enabled modes, but I am not sure if this is expected
> for
> > > > > in-memory. Of course, we can run this initialization automatically,
> > but
> > > > it
> > > > > is not always a good idea.
> > > > >
> > > > > If we are ok to have this restrictions for in-memory mode, then
> > service
> > > > > persistence makes sense.
> > > > >
> > > > > --AG
> > > > >
> > > > > 2018-04-11 22:36 GMT+03:00 Denis Magda :
> > > > >
> > > > >> Denis,
> > > > >>
> > > > >> I think that the service deployment state needs be persisted
> > > > cluster-wide.
> > > > >> I guess that our meta-store is capable of doing so. Alex G.,
> > Vladimir,
> > > > >> could you confirm?
> > > > >>
> > > > >> As for the split-brain scenarios, I would put them aside for now
> > > > because,
> > > > >> anyway, they have to be solved at lower levels (meta store,
> > discovery,
> > > > >> etc.).
> > > > >>
> > > > >> Also, I heard that presently we store a service configuration in
> the
> > > > >> system
> > > > >> cache that 

Re: [jira] [Created] (IGNITE-8279) Clients can't operate on services after deactivation

[Raymond Wilson]

Hi Denis,

Would this be better to target 2.5? It seems like a significant regression...

Thanks,
Raymond. 

Sent from my iPhone

> On 17/04/2018, at 1:46 AM, Denis Mekhanikov (JIRA)  wrote:
> 
> Denis Mekhanikov created IGNITE-8279:
> 
> 
> Summary: Clients can't operate on services after deactivation
> Key: IGNITE-8279
> URL: https://issues.apache.org/jira/browse/IGNITE-8279
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Denis Mekhanikov
>Assignee: Denis Mekhanikov
> Fix For: 2.6
> Attachments: ServiceDeploymentAfterDeactivationTest.java
> 
> When this cluster gets deactivated and activated back again, clients become 
> incapable of service deployment. Calls to {{IgniteService.deploy*}} methods 
> hang indefinitely, and no services are getting deployed to these clients.
> 
> After deactivation, {{ServiceEntriesListener}} stops being invoked in service 
> cache changes.
> 
> Find attached test, reproducing this problem.
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)

[GitHub] ignite pull request #3810: IGNITE-7972: Fixed NPE in TTL manager on unwindEv...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3810


---

[GitHub] ignite pull request #3813: IGNITE-8242: Remove method GAGridUtils.getGenesFo...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3813


---

Re: Service grid redesign

[Denis Mekhanikov]

Basically, my question is: at which moment services should be deployed on
connecting nodes?
Should we reject a node from being included into a topology, if services,
that are assigned to it, fail to deploy?

It would be good to be sure, that all assigned services are initialised and
working, when node start is finished.
Otherwise it's unclear, how to notify a user about failures in service
initialisation on new nodes.

If we decide to provide such guarantee, then how are we going to do that?
Is procedure, that I described, viable?
It requires hacking through the discovery protocol, which is a thing, that
should be avoided.
So, maybe there is another way to achieve the same thing?

Denis

сб, 14 апр. 2018 г. в 1:48, Denis Magda :

> It sounds like it's not a trivial thing to support the automatic services
> redeployment after a restart. Let's postpone it for now, guys concentrating
> on more urgent things related to the services.
>
> Alex, Vladimir,
>
> Could you have a look at Denis question about the discovery-based
> deployment? Guess it's the only one thing that prevents us from the IEP
> finalization.
>
> --
> Denis
>
> On Fri, Apr 13, 2018 at 5:30 AM, Denis Mekhanikov 
> wrote:
>
> > Vladimir,
> >
> > Currently we don't save binary metadata to disk, when persistence is
> > disabled.
> > But we still persist marshaller mappings for some reason, and I
> personally
> > believe, that we shouldn't.
> >
> > But I agree, that we should separate data and service persistence
> > configuration.
> > Right now persistence of services is configured in a pretty non-obvious
> > manner.
> > It should be a clear way to tell Ignite, whether you want services to be
> > persisted or not.
> >
> > I'm not sure, that we should make "statefullness" in general
> configurable.
> > Users don't care much, whether metadata is preserved on restarts, or not.
> >
> > Denis
> >
> > пт, 13 апр. 2018 г. в 14:29, Vladimir Ozerov :
> >
> > > Alex,
> > >
> > > I would say that we've already had this behavior for years - marshaller
> > > cache. I think it is time to agree that "in-memory" != stateless.
> Instead
> > > "in-memory" means "data is not stored on disk".
> > > May be we can have a flag which will wipe out all metadata on node
> > restart
> > > (e.g. it could make sense for embedded clients)?
> > >
> > > On Fri, Apr 13, 2018 at 12:48 PM, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com> wrote:
> > >
> > > > Denis,
> > > >
> > > > This is a subtle question. It looks like we have now a number of
> > > use-cases
> > > > when persistent storage is required even for a pure in-memory mode.
> One
> > > of
> > > > the use-cases is thin client authentication, the other is service
> grid
> > > > configuration persistence.
> > > >
> > > > Generally, I would agree that this is an expected behavior. However,
> > this
> > > > means that a user cannot simply start and stop nodes randomly
> anymore.
> > > > Ignite start will require some sort of installation or work folder
> > > > initialization (sort of initdb in postgres) which is ok for
> > > > persistence-enabled modes, but I am not sure if this is expected for
> > > > in-memory. Of course, we can run this initialization automatically,
> but
> > > it
> > > > is not always a good idea.
> > > >
> > > > If we are ok to have this restrictions for in-memory mode, then
> service
> > > > persistence makes sense.
> > > >
> > > > --AG
> > > >
> > > > 2018-04-11 22:36 GMT+03:00 Denis Magda :
> > > >
> > > >> Denis,
> > > >>
> > > >> I think that the service deployment state needs be persisted
> > > cluster-wide.
> > > >> I guess that our meta-store is capable of doing so. Alex G.,
> Vladimir,
> > > >> could you confirm?
> > > >>
> > > >> As for the split-brain scenarios, I would put them aside for now
> > > because,
> > > >> anyway, they have to be solved at lower levels (meta store,
> discovery,
> > > >> etc.).
> > > >>
> > > >> Also, I heard that presently we store a service configuration in the
> > > >> system
> > > >> cache that doesn't give us a way to deploy a new version of a
> service
> > > >> without undeployment of the previous one. Will this issue be
> addressed
> > > by
> > > >> the new deployment approach?
> > > >>
> > > >> --
> > > >> Denis
> > > >>
> > > >> On Wed, Apr 11, 2018 at 1:28 AM, Denis Mekhanikov <
> > > dmekhani...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Denis,
> > > >> >
> > > >> > Sounds reasonable. It's not clear, though, what should happen, if
> a
> > > >> joining
> > > >> > node has some services persisted, that are missing on other nodes.
> > > >> > Should we deploy them?
> > > >> > If we do so, it could lead to surprising behaviour. For example
> you
> > > >> could
> > > >> > kill a node, undeploy a service, then bring back an old node, and
> it
> > > >> would
> > > >> > make the service resurrect.
> > > >> > We could store some deployment counter along with the service
> > > 

[GitHub] ignite pull request #3817: IGNITE-8169: Adopt KMeans and remove FuzzyCMeans

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3817


---

[GitHub] ignite pull request #3835: IGNITE-7786 Add ping timeout and ping interval to...

[ivandasch]

GitHub user ivandasch opened a pull request:

https://github.com/apache/ignite/pull/3835

IGNITE-7786 Add ping timeout and ping interval to control.sh. Also,

make default values greater.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ivandasch/ignite ignite-7786

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3835.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3835


commit ac53a778abbb4b4d4b0d19d71bdff36593d0c3ac
Author: Ivan Daschinskiy 
Date:   2018-04-16T17:03:32Z

IGNITE-7786 Add ping timeout and ping interval to control.sh. Also,
make default values greater.




---

[jira] [Created] (IGNITE-8284) Performance SQL with distinct in subquery

[Pavel Vinokurov (JIRA)]

Pavel Vinokurov created IGNITE-8284:
---

 Summary: Performance SQL with distinct in subquery
 Key: IGNITE-8284
 URL: https://issues.apache.org/jira/browse/IGNITE-8284
 Project: Ignite
  Issue Type: New Feature
Reporter: Pavel Vinokurov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3816: IGNITE-8246: update print errors

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3816


---

Re: affinityRun/Call in C++

[Igor Sapego]

It depends on which functionality is needed, as there are
several versions of those methods. Some would require
implementing of Cluster API, which is not present in C++ also.

Best Regards,
Igor

On Mon, Apr 16, 2018 at 6:41 PM, Denis Magda  wrote:

> Do we target it for 2.6 release? It shouldn't be time-consuming to support
> it, right?
>
> --
> Denis
>
> On Mon, Apr 16, 2018 at 2:48 AM, Igor Sapego  wrote:
>
> > Guys,
> >
> > I've created a ticket: [1]
> >
> > [1] - https://issues.apache.org/jira/browse/IGNITE-8273
> >
> > Best Regards,
> > Igor
> >
> > On Wed, Apr 11, 2018 at 2:44 AM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Is there a ticket? Let's create one if not.
> > >
> > > -Val
> > >
> > > On Tue, Apr 10, 2018 at 6:17 AM, Vladimir Ozerov  >
> > > wrote:
> > >
> > > > Val,
> > > >
> > > > They are simply not implemented yet. I am not aware of concrete plans
> > to
> > > > support them.
> > > >
> > > > On Mon, Apr 9, 2018 at 11:33 PM, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com> wrote:
> > > >
> > > > > Guys,
> > > > >
> > > > > Is there a way to run collocated compute job in C++? I can't find
> > > > > affinityRun and affinityCall method in C++ compute API, am I
> missing
> > > > > something? If we really don't have them, is there any particular
> > reason
> > > > for
> > > > > this and/or plans to add them?
> > > > >
> > > > > -Val
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-8283) CPP: Implement 'varint' solution to be configurable via BinaryConfiguration

[Vyacheslav Daradur (JIRA)]

Vyacheslav Daradur created IGNITE-8283:
--

 Summary: CPP: Implement 'varint' solution to be configurable via 
BinaryConfiguration
 Key: IGNITE-8283
 URL: https://issues.apache.org/jira/browse/IGNITE-8283
 Project: Ignite
  Issue Type: Sub-task
Reporter: Vyacheslav Daradur
Assignee: Nikolay Izhikov
 Fix For: 2.6






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Mass TC Run-All during this weekend

[Dmitry Pavlov]

I wish this report have just 30 or 10 failures, so it would be very easy to
analyse these failures.

 I'll think how to present some statistic or summary of all these fails.

пн, 16 апр. 2018 г. в 18:40, Denis Magda :

> Hi Dmitriy,
>
> Though it's difficult to analyze the report, I guess the things are moving
> in a right direction. Probably, there is a clearer format you can come up
> with. Thanks for keep driving it.
>
> --
> Denis
>
> On Mon, Apr 16, 2018 at 3:04 AM, Dmitry Pavlov 
> wrote:
>
> > Hi Igniters,
> >
> >  New mass run results from this weekend: tests 309 suites 24
> >  It does not mean 97 tests were fixed (it can be muted, not reproduced
> > during these 30 runs, fixied, etc), but result is relatively positive.
> >
> >  Full report can be found in MTCGA wiki attachments:
> > https://cwiki.apache.org/confluence/pages/viewpageattachments.action?
> > pageId=73631266
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вс, 25 мар. 2018 г. в 15:50, Dmitry Pavlov :
> >
> > > Hi Igniters,
> > >
> > > We have the positive dynamics in the resolving MTCGA JIRA tasks, and I
> > > must mention the contributions of Aleksey Plekhanov, Nikolay Izhikov,
> > Peter
> > > Ivanov, Anton Kalashnikov, Alexey Goncharuk, Sergey Chugunov, Turik
> > > Campbell.
> > >
> > > Mass run-alls have been finished (
> > > https://ci.ignite.apache.org/viewType.html?buildTypeId=
> > IgniteTests24Java8_RunAll=%3Cdefault%3E=buildTypeStatusDiv ).
> > > Unfortunately current mass run shows more failures than previous.
> Current
> > > results: tests 406 suites 26 (Previous was: tests 371 suites 21)
> > >
> > > The explanation of this may be randomized nature of fails, and that
> some
> > > tests fail this time but did not fail in the past. Also it is possible
> > that
> > > new issues were introduced. I'll generate report on Monday and attach
> it
> > to
> > > https://cwiki.apache.org/confluence/pages/viewpageattachments.action?
> > pageId=73631266
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 23 мар. 2018 г. в 22:08, Denis Magda :
> > >
> > >> Ignite the Team City :)
> > >>
> > >> --
> > >> Denis
> > >>
> > >> On Fri, Mar 23, 2018 at 7:00 AM, Dmitry Pavlov  >
> > >> wrote:
> > >>
> > >> > Hi Igniters,
> > >> >
> > >> > To get the full picture of the tests now, I'm not going to, I'm
> going
> > to
> > >> > run a massive Run All this weekend.
> > >> >
> > >> > I'm going to start it at 20:00 MSK (17:00 UTC).
> > >> >
> > >> > Are there any objections? If you need to run your branches,
> rearrange
> > >> them
> > >> > at the beginning of the queue ..
> > >> >
> > >> > Sincerely,
> > >> > Dmitriy Pavlov
> > >> >
> > >>
> > >
> >
>

[GitHub] ignite pull request #3831: IGNITE-8280: fix NPE

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3831


---

Re: IGNITE-6879

[Роман Меерсон]

Denis,

module naming fixed

пн, 16 апр. 2018 г. в 19:24, Denis Magda :

> Roman,
>
> In my last reply, I voted for your approach - using "spring-data*_*2.0" for
> the new Spring version. However, please note that we use "_" symbol instead
> of "-" to separate a version from the package name.
>
> --
> Denis
>
> On Sun, Apr 15, 2018 at 8:17 AM, Роман Меерсон 
> wrote:
>
> > Hi all!
> >
> > So guys let’s make a decision. We leave code in current state as I
> suggest
> > or change module naming according Dmitry’s suggestion.
> > ср, 11 апр. 2018 г. в 1:54, Denis Magda :
> >
> > > Roman,
> > >
> > > Your suggestion sounds reasonable to me. Backing it up.
> > >
> > > --
> > > Denis
> > >
> > > On Tue, Apr 10, 2018 at 2:50 PM, Роман Меерсон 
> > > wrote:
> > >
> > > > Hi all!
> > > >
> > > > IMHO if we do so we'll produce big pain for everybody while migrating
> > on
> > > > new version, because ones should change method and others should
> change
> > > > their poms. This change would be backward incompatible so it probably
> > > > should follow with major version upgrade, but I'm not sure about it.
> > > >
> > > > Otherwise if we leave current state( spring-data for old and
> > > > spring-data-2.0 for new) we could support old users who probably use
> > > spring
> > > > data 1.0 (because spring data 2.0 release was not so long ago) and
> > > provide
> > > > new functionality for users who want to use new spring data.
> > > > In this case old users wouldn't change any in their code except
> ignite
> > > > version, and new users would include ignite in their Pom anyway and
> > could
> > > > choose which module of spring data to bring.
> > > >
> > > > After some time (probably on 3.0 release) we could change naming as
> > Denis
> > > > suggested.
> > > >
> > > > Anyway I leave this decision up to you, just tell me what is the way
> to
> > > > finish this PR.
> > > >
> > > > Regards, Roman.
> > > >
> > > > ср, 11 апр. 2018 г. в 1:35, Denis Magda :
> > > >
> > > > > In our Hibernate integration we define following two modules to
> > > > > distinguish incompatible versions:
> > > > >
> > > > >- ignite-hiberbate_4.2
> > > > >- ignite-hibernate_5.1
> > > > >
> > > > > In Spark we have:
> > > > >
> > > > >- ignite-spark
> > > > >- ignite-spark_2.10
> > > > >
> > > > > After thinking this over, I would do the following with Spring
> Data:
> > > > >
> > > > >- ignite-spring-data for the latest Sprind Data 2.0
> > > > >- ignite-spring-data_1.0
> > > > >
> > > > > What do you think?
> > > > >
> > > > > --
> > > > > Denis
> > > > >
> > > > > On Tue, Apr 10, 2018 at 3:50 AM, Dmitry Pavlov <
> > dpavlov@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Thank you, Roman.
> > > > >>
> > > > >> Igniters,
> > > > >>
> > > > >> IMO we should consider one more alternative - renaming of old
> module
> > > and
> > > > >> package names. Users, which prefer to stay on previous version
> will
> > be
> > > > >> requiered to update their pom's. In the same time users which are
> > > ready
> > > > to
> > > > >> migrate to spring data 2.0 will need to update methods naming.
> > > > >>
> > > > >> Denis M, what would you say?
> > > > >>
> > > > >> Sincerely,
> > > > >> Dmitriy Pavlov
> > > > >>
> > > > >> вт, 10 апр. 2018 г. в 11:27, Роман Меерсон  >:
> > > > >>
> > > > >>> Hi Dmitry!
> > > > >>>
> > > > >>> I`ve just commited new fix. I renamed package of new module to
> > > > >>> springdata20, it helps us to separate old implementation from new
> > and
> > > > also
> > > > >>> should fix all compilation errors.
> > > > >>>
> > > > >>> пн, 9 апр. 2018 г. в 23:54, Роман Меерсон  >:
> > > > >>>
> > > >  Ok, I'll check it, but I haven't face this problem.
> > > >  If I'll find same issue, what is the proper way? Renaming to
> > > something
> > > >  like Ignite2QueryGenerator or module removing?
> > > >  пн, 9 апр. 2018 г. в 23:40, Dmitry Pavlov <
> dpavlov@gmail.com
> > >:
> > > > 
> > > > > There are 2 classes IgniteQueryGenerator with same package
> name.
> > > > > Ignite in Idea can't compile.
> > > > >
> > > > >
> > > > > пн, 9 апр. 2018 г., 21:38 Роман Меерсон  >:
> > > > >
> > > > >> Hi Dmitry!
> > > > >
> > > > >
> > > > >> Could you specify where you find conflict? Because I don’t
> have
> > > any.
> > > > >> пн, 9 апр. 2018 г. в 21:09, Dmitry Pavlov <
> > dpavlov@gmail.com
> > > >:
> > > > >>
> > > > >>> Hi Denis,
> > > > >>>
> > > > >>> could we support just one version instead of leaving
> compatible
> > > > >>> module?
> > > > >>>
> > > > >>> Sincerely,
> > > > >>> Dmitriy Pavlov
> > > > >>>
> > > > >>> пн, 9 апр. 2018 г. в 20:08, Dmitry Pavlov <
> > dpavlov@gmail.com
> > > >:
> > > > >>>
> > > > 
> > > > 

Re: affinityRun/Call in C++

[Denis Magda]

Do we target it for 2.6 release? It shouldn't be time-consuming to support
it, right?

--
Denis

On Mon, Apr 16, 2018 at 2:48 AM, Igor Sapego  wrote:

> Guys,
>
> I've created a ticket: [1]
>
> [1] - https://issues.apache.org/jira/browse/IGNITE-8273
>
> Best Regards,
> Igor
>
> On Wed, Apr 11, 2018 at 2:44 AM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Is there a ticket? Let's create one if not.
> >
> > -Val
> >
> > On Tue, Apr 10, 2018 at 6:17 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Val,
> > >
> > > They are simply not implemented yet. I am not aware of concrete plans
> to
> > > support them.
> > >
> > > On Mon, Apr 9, 2018 at 11:33 PM, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Guys,
> > > >
> > > > Is there a way to run collocated compute job in C++? I can't find
> > > > affinityRun and affinityCall method in C++ compute API, am I missing
> > > > something? If we really don't have them, is there any particular
> reason
> > > for
> > > > this and/or plans to add them?
> > > >
> > > > -Val
> > > >
> > >
> >
>

Re: Mass TC Run-All during this weekend

[Denis Magda]

Hi Dmitriy,

Though it's difficult to analyze the report, I guess the things are moving
in a right direction. Probably, there is a clearer format you can come up
with. Thanks for keep driving it.

--
Denis

On Mon, Apr 16, 2018 at 3:04 AM, Dmitry Pavlov 
wrote:

> Hi Igniters,
>
>  New mass run results from this weekend: tests 309 suites 24
>  It does not mean 97 tests were fixed (it can be muted, not reproduced
> during these 30 runs, fixied, etc), but result is relatively positive.
>
>  Full report can be found in MTCGA wiki attachments:
> https://cwiki.apache.org/confluence/pages/viewpageattachments.action?
> pageId=73631266
>
> Sincerely,
> Dmitriy Pavlov
>
> вс, 25 мар. 2018 г. в 15:50, Dmitry Pavlov :
>
> > Hi Igniters,
> >
> > We have the positive dynamics in the resolving MTCGA JIRA tasks, and I
> > must mention the contributions of Aleksey Plekhanov, Nikolay Izhikov,
> Peter
> > Ivanov, Anton Kalashnikov, Alexey Goncharuk, Sergey Chugunov, Turik
> > Campbell.
> >
> > Mass run-alls have been finished (
> > https://ci.ignite.apache.org/viewType.html?buildTypeId=
> IgniteTests24Java8_RunAll=%3Cdefault%3E=buildTypeStatusDiv ).
> > Unfortunately current mass run shows more failures than previous. Current
> > results: tests 406 suites 26 (Previous was: tests 371 suites 21)
> >
> > The explanation of this may be randomized nature of fails, and that some
> > tests fail this time but did not fail in the past. Also it is possible
> that
> > new issues were introduced. I'll generate report on Monday and attach it
> to
> > https://cwiki.apache.org/confluence/pages/viewpageattachments.action?
> pageId=73631266
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 23 мар. 2018 г. в 22:08, Denis Magda :
> >
> >> Ignite the Team City :)
> >>
> >> --
> >> Denis
> >>
> >> On Fri, Mar 23, 2018 at 7:00 AM, Dmitry Pavlov 
> >> wrote:
> >>
> >> > Hi Igniters,
> >> >
> >> > To get the full picture of the tests now, I'm not going to, I'm going
> to
> >> > run a massive Run All this weekend.
> >> >
> >> > I'm going to start it at 20:00 MSK (17:00 UTC).
> >> >
> >> > Are there any objections? If you need to run your branches, rearrange
> >> them
> >> > at the beginning of the queue ..
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >>
> >
>

[GitHub] ignite pull request #3834: IGNITE-7640 done

[SharplEr]

GitHub user SharplEr opened a pull request:

https://github.com/apache/ignite/pull/3834

IGNITE-7640 done



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/SharplEr/ignite ignite-7640

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3834.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3834


commit dbcffce3b0cd90a13c9a8b70a344f240cfc72bea
Author: Alexander Menshikov 
Date:   2018-04-16T15:34:06Z

IGNITE-7640 done




---

Re: IGNITE-6879

[Denis Magda]

Roman,

In my last reply, I voted for your approach - using "spring-data*_*2.0" for
the new Spring version. However, please note that we use "_" symbol instead
of "-" to separate a version from the package name.

--
Denis

On Sun, Apr 15, 2018 at 8:17 AM, Роман Меерсон  wrote:

> Hi all!
>
> So guys let’s make a decision. We leave code in current state as I suggest
> or change module naming according Dmitry’s suggestion.
> ср, 11 апр. 2018 г. в 1:54, Denis Magda :
>
> > Roman,
> >
> > Your suggestion sounds reasonable to me. Backing it up.
> >
> > --
> > Denis
> >
> > On Tue, Apr 10, 2018 at 2:50 PM, Роман Меерсон 
> > wrote:
> >
> > > Hi all!
> > >
> > > IMHO if we do so we'll produce big pain for everybody while migrating
> on
> > > new version, because ones should change method and others should change
> > > their poms. This change would be backward incompatible so it probably
> > > should follow with major version upgrade, but I'm not sure about it.
> > >
> > > Otherwise if we leave current state( spring-data for old and
> > > spring-data-2.0 for new) we could support old users who probably use
> > spring
> > > data 1.0 (because spring data 2.0 release was not so long ago) and
> > provide
> > > new functionality for users who want to use new spring data.
> > > In this case old users wouldn't change any in their code except ignite
> > > version, and new users would include ignite in their Pom anyway and
> could
> > > choose which module of spring data to bring.
> > >
> > > After some time (probably on 3.0 release) we could change naming as
> Denis
> > > suggested.
> > >
> > > Anyway I leave this decision up to you, just tell me what is the way to
> > > finish this PR.
> > >
> > > Regards, Roman.
> > >
> > > ср, 11 апр. 2018 г. в 1:35, Denis Magda :
> > >
> > > > In our Hibernate integration we define following two modules to
> > > > distinguish incompatible versions:
> > > >
> > > >- ignite-hiberbate_4.2
> > > >- ignite-hibernate_5.1
> > > >
> > > > In Spark we have:
> > > >
> > > >- ignite-spark
> > > >- ignite-spark_2.10
> > > >
> > > > After thinking this over, I would do the following with Spring Data:
> > > >
> > > >- ignite-spring-data for the latest Sprind Data 2.0
> > > >- ignite-spring-data_1.0
> > > >
> > > > What do you think?
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Tue, Apr 10, 2018 at 3:50 AM, Dmitry Pavlov <
> dpavlov@gmail.com>
> > > > wrote:
> > > >
> > > >> Thank you, Roman.
> > > >>
> > > >> Igniters,
> > > >>
> > > >> IMO we should consider one more alternative - renaming of old module
> > and
> > > >> package names. Users, which prefer to stay on previous version will
> be
> > > >> requiered to update their pom's. In the same time users which are
> > ready
> > > to
> > > >> migrate to spring data 2.0 will need to update methods naming.
> > > >>
> > > >> Denis M, what would you say?
> > > >>
> > > >> Sincerely,
> > > >> Dmitriy Pavlov
> > > >>
> > > >> вт, 10 апр. 2018 г. в 11:27, Роман Меерсон :
> > > >>
> > > >>> Hi Dmitry!
> > > >>>
> > > >>> I`ve just commited new fix. I renamed package of new module to
> > > >>> springdata20, it helps us to separate old implementation from new
> and
> > > also
> > > >>> should fix all compilation errors.
> > > >>>
> > > >>> пн, 9 апр. 2018 г. в 23:54, Роман Меерсон :
> > > >>>
> > >  Ok, I'll check it, but I haven't face this problem.
> > >  If I'll find same issue, what is the proper way? Renaming to
> > something
> > >  like Ignite2QueryGenerator or module removing?
> > >  пн, 9 апр. 2018 г. в 23:40, Dmitry Pavlov  >:
> > > 
> > > > There are 2 classes IgniteQueryGenerator with same package name.
> > > > Ignite in Idea can't compile.
> > > >
> > > >
> > > > пн, 9 апр. 2018 г., 21:38 Роман Меерсон :
> > > >
> > > >> Hi Dmitry!
> > > >
> > > >
> > > >> Could you specify where you find conflict? Because I don’t have
> > any.
> > > >> пн, 9 апр. 2018 г. в 21:09, Dmitry Pavlov <
> dpavlov@gmail.com
> > >:
> > > >>
> > > >>> Hi Denis,
> > > >>>
> > > >>> could we support just one version instead of leaving compatible
> > > >>> module?
> > > >>>
> > > >>> Sincerely,
> > > >>> Dmitriy Pavlov
> > > >>>
> > > >>> пн, 9 апр. 2018 г. в 20:08, Dmitry Pavlov <
> dpavlov@gmail.com
> > >:
> > > >>>
> > > 
> > > 
> > >  пн, 9 апр. 2018 г. в 20:07, Dmitry Pavlov <
> > dpavlov@gmail.com
> > > >:
> > > 
> > > > Hi Roman,
> > > >
> > > > I've applied PR locally and I have class name conflict at
> least
> > > > for
> > > > org.apache.ignite.springdata.repository.query.
> > > IgniteQueryGenerator
> > > >
> > > > How could we solve it? Is it better to 

[GitHub] ignite pull request #3833: IGNITE-8282 Direct IO: support fdatasync, which d...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3833

IGNITE-8282 Direct IO: support fdatasync, which does not flush meta

 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8282-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3833.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3833


commit 8004d1e797fc6ccf9465797c026fb3884703e34a
Author: dpavlov 
Date:   2018-04-16T14:49:46Z

IGNITE-8282 Direct IO: support fdatasync, which does not flush modified 
metadata




---

[GitHub] ignite pull request #3832: IGNITE-8282 Direct IO: support fdatasync, which d...

[dspavlov]

Github user dspavlov closed the pull request at:

https://github.com/apache/ignite/pull/3832


---

[GitHub] ignite pull request #3832: IGNITE-8282 Direct IO: support fdatasync, which d...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3832

IGNITE-8282 Direct IO: support fdatasync, which does not flush modified 
metadata



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8282

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3832.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3832


commit f5fbd1a8951f73897aaadea2656ec35ede852762
Author: dpavlov 
Date:   2018-04-16T14:36:02Z

IGNITE-7182 Supported striped dirty page storage in skip list maps and 
quick sort with early cp pages write start

commit a9d19313a711765f702610b2d56488f58735fa70
Author: dpavlov 
Date:   2018-04-16T14:46:56Z

IGNITE-8282 Direct IO: support fdatasync, which does not flush modified 
metadata




---

[GitHub] ignite pull request #3831: IGNITE-8280: fix NPE

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/3831

IGNITE-8280: fix NPE



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8280

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3831.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3831


commit 45ae97c730d5548a2d288a4ca794abafb11cc9c1
Author: tledkov-gridgain 
Date:   2018-04-16T14:20:59Z

IGNITE-8280: fix NPE




---

[GitHub] ignite pull request #3243: IGNITE-7182: Checkpoint pages: optimize and paral...

[dspavlov]

Github user dspavlov closed the pull request at:

https://github.com/apache/ignite/pull/3243


---

[GitHub] ignite pull request #3830: IGNITE-7182 Supported striped dirty page storage ...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3830

IGNITE-7182 Supported striped dirty page storage in skip list maps  

And quick sort with early cp pages write start

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7182-buckets2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3830.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3830


commit f5fbd1a8951f73897aaadea2656ec35ede852762
Author: dpavlov 
Date:   2018-04-16T14:36:02Z

IGNITE-7182 Supported striped dirty page storage in skip list maps and 
quick sort with early cp pages write start




---

[jira] [Created] (IGNITE-8282) Direct IO: support fdatasync, which does not flush modified metadata

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-8282:
--

 Summary: Direct IO: support fdatasync, which does not flush 
modified metadata
 Key: IGNITE-8282
 URL: https://issues.apache.org/jira/browse/IGNITE-8282
 Project: Ignite
  Issue Type: Improvement
  Components: persistence
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov
 Fix For: 2.6


fsync(2) - Linux man page
fdatasync() is similar to fsync(), but does not flush modified metadata unless 
that metadata is needed in order to allow a subsequent data retrieval to be 
correctly handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8281) Add Docker image build for Apache Ignite Release

[Peter Ivanov (JIRA)]

Peter Ivanov created IGNITE-8281:


 Summary: Add Docker image build for Apache Ignite Release
 Key: IGNITE-8281
 URL: https://issues.apache.org/jira/browse/IGNITE-8281
 Project: Ignite
  Issue Type: Task
Reporter: Peter Ivanov
Assignee: Peter Ivanov


Similar to Apache Ignite Nightly Release, add Docker images (Apache Ignite and 
WebConsole) build to Apache Ignite Release build. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8280) NPE on some cases at the QueryEntity.getDecimalInfo

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-8280:


 Summary: NPE on some cases at the QueryEntity.getDecimalInfo
 Key: IGNITE-8280
 URL: https://issues.apache.org/jira/browse/IGNITE-8280
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.4
Reporter: Taras Ledkov
Assignee: Taras Ledkov
 Fix For: 2.5


{code}
Caused by: java.lang.NullPointerException
at java.util.Collections$UnmodifiableMap.(Collections.java:1446)
at java.util.Collections.unmodifiableMap(Collections.java:1433)
at org.apache.ignite.cache.QueryEntity.getDecimalInfo(QueryEntity.java:581)
at 
org.apache.ignite.internal.processors.query.QueryUtils.normalizeQueryEntity(QueryUtils.java:268)
at 
org.apache.ignite.internal.processors.query.QueryUtils.normalizeQueryEntities(QueryUtils.java:215)
at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.initializeConfigDefaults(GridCacheUtils.java:1688)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.initialize(GridCacheProcessor.java:270)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheChangeRequest(GridCacheProcessor.java:4305)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.dynamicStartCachesByStoredConf(GridCacheProcessor.java:2956)
at 
org.gridgain.grid.internal.processors.cache.database.snapshot.SnapshotRestoreFuture.startCaches(SnapshotRestoreFuture.java:371)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3829: IGNITE-6075 'CacheLoadingConcurrentGridStartSelfT...

[1vanan]

GitHub user 1vanan opened a pull request:

https://github.com/apache/ignite/pull/3829

IGNITE-6075 'CacheLoadingConcurrentGridStartSelfTest sometimes fails'



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/1vanan/ignite IGNITE-6075

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3829.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3829


commit 658f27c3c536729dc132067cbd299319bf7d4f67
Author: Fedotov 
Date:   2018-04-11T14:41:51Z

change iterations

commit 70c6b763ee394c5ff8854fb9cfe39daec139755d
Author: Fedotov 
Date:   2018-04-16T14:03:59Z

add remap count




---

[jira] [Created] (IGNITE-8279) Clients can't operate on services after deactivation

[Denis Mekhanikov (JIRA)]

Denis Mekhanikov created IGNITE-8279:


 Summary: Clients can't operate on services after deactivation
 Key: IGNITE-8279
 URL: https://issues.apache.org/jira/browse/IGNITE-8279
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Denis Mekhanikov
Assignee: Denis Mekhanikov
 Fix For: 2.6
 Attachments: ServiceDeploymentAfterDeactivationTest.java

When this cluster gets deactivated and activated back again, clients become 
incapable of service deployment. Calls to {{IgniteService.deploy*}} methods 
hang indefinitely, and no services are getting deployed to these clients.

After deactivation, {{ServiceEntriesListener}} stops being invoked in service 
cache changes.

Find attached test, reproducing this problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3828: Ignite 8154

[antkr]

GitHub user antkr opened a pull request:

https://github.com/apache/ignite/pull/3828

Ignite 8154



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8154

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3828.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3828


commit dc38ee5ca361a58907a1c97779538557687e5434
Author: Anton Kurbanov 
Date:   2018-04-10T20:47:55Z

IGNITE-8154 added exception listener to JmsStreamer

commit 8c580e65e0f9722d7f353e8beecfedc24828d164
Author: Anton Kurbanov 
Date:   2018-04-16T13:33:26Z

IGNITE-8154 default ExceptionListener behaviour changed, added test




---

[GitHub] ignite pull request #3827: IGNITE-8276 : Removed incorrect assertion, fixed ...

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/3827

IGNITE-8276 : Removed incorrect assertion, fixed check for forceKeysRequest.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8276

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3827.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3827


commit 99dd1b238a42e38cd2bd96482401d6ee3af22ebc
Author: Ilya Lantukh 
Date:   2018-04-06T10:49:10Z

ignite-8276 : Removed incorrect assertion, fixed check for forceKeysRequest.




---

[GitHub] ignite pull request #3826: IGNITE-8203 Correctly handles ClosedByInterrupt e...

[ivandasch]

GitHub user ivandasch opened a pull request:

https://github.com/apache/ignite/pull/3826

IGNITE-8203 Correctly handles ClosedByInterrupt exception while

partition initializes.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ivandasch/ignite ignite-8203

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3826.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3826


commit 53c9eb620bbc22e55e7fed7397b39f1744d3128e
Author: Ivan Daschinskiy 
Date:   2018-04-16T13:01:17Z

IGNITE-8203 Correctly handles ClosedByInterrupt exception while
partition initializes.




---

Re: IEP-18: Transparent Data Encryption

[Nikolay Izhikov]

Hello, Vladimir.

> community is not aware of concrete architecture and proposed public API

Concrete architecture is described in IEP-18 [1].
Please, tell me, what else you want to be written.

I think answers to all questions have to be addressed(and discussed with 
community!) when we crack into implementation.
Personally, I don't believe in any kind of more detailed design from eagle 
height.
So I want to move from one part of this feature to another and make concrete 
choices just before implementation.

Thoughts?

Public API proposition will be part of an implementation and, of course, will 
be discussed.
I don't think we have to do specific choices at the moment.

> We need to continue discussion and come up with detailed design.

Can you give me the check-list of detailed design?
As far as I can see many of other IEP(already implemented) doesn't include any 
details you mentioned here [2], [3].

Anyway, let's make this IEP some kind of "reference implementation" for other 
IEP's :).

> 1) How encryption is configured - public API is needed

Similarly to other SPI.

> 2) How encrypted cluster is started

I study links you provide.
I asked DB administrators I know.
Looks like we can implement automatic TDE activation.

> 3) How and where encrypted cache keys are stored

I want to follow your advice and store CEK's near cache.

> 4) Protocol of key's exchange between nodes
> 5) Detailed description of encryption flow - what algorithm is used? will> we 
> use CBC?
> 6) What changes would be required to WAL logic (at the very least -> 
> enumerate affected types of records)?

I don't have specific answers to this questions.
Do you think we must answer this questions before starting any implementation?

> 7) Would it be possible to enable/disable encryption on specific> cache/table 
> in runtime?

No, for the first implementation.
Cache has to be created as encrypted. 
If cache created as decrypted it can't become encrypted.

In the future we can extend encryption mechanism and add this feature to the 
product.

> 8) Would these changes affect compression design anyhow?

No.

[1] 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-18%3A+Transparent+Data+Encryption
[2] 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-4+Baseline+topology+for+caches
[3] 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-7%3A+Ignite+internal+problems+detection

В Пн, 16/04/2018 в 13:28 +0300, Vladimir Ozerov пишет:
> Hi Nikolay,
> 
> I noticed that some tickets have been created for this feature. Hopefully,
> you haven't started implementation yet, because at this point community is
> not aware of concrete architecture and proposed public API. We need to
> continue discussion and come up with detailed design. Of most importance we
> need to understand the following:
> 1) How encryption is configured - public API is needed (how to setup key
> providers, how to enable encryption for specific caches, etc)
> 2) How encrypted cluster is started - normally, I do not expect any
> additional manual actions from user once everything is configured.
> Especially, in presence of auto-activation capability.
> 3) How and where encrypted cache keys are stored
> 4) Protocol of key's exchange between nodes
> 5) Detailed description of encryption flow - what algorithm is used? will
> we use CBC?
> 6) What changes would be required to WAL logic (at the very least -
> enumerate affected types of records)?
> 7) Would it be possible to enable/disable encryption on specific
> cache/table in runtime?
> 8) Would these changes affect compression design anyhow?
> 
> Vladimir.
> 
> On Tue, Apr 10, 2018 at 5:30 PM, Vladimir Ozerov 
> wrote:
> 
> > Hi NIkolay,
> > 
> > Regarding system caches, rule of thumb here - do not use them. Keys should
> > be stored near cache.
> > 
> > As far as password:
> > 1) Oracle auto-login wallet [1]
> > 2) MySQL- password may be set inside configuration [2]
> > 
> > I do not think that any kind of prompts are needed here out of the box.
> > May be we could consider them in future, but at the moment it looks
> > redundant to me.
> > 
> > [1] https://docs.oracle.com/cd/E11882_01/network.112/
> > e40393/asotrans.htm#CHDCCHBH
> > [2] https://dev.mysql.com/doc/refman/5.7/en/keyring-
> > encrypted-file-plugin.html
> > 
> > On Mon, Apr 9, 2018 at 5:42 PM, Nikolay Izhikov 
> > wrote:
> > 
> > > Hello, Vladimir.
> > > 
> > > > 1) Why do you propose to store CEK in separate cache?
> > > 
> > > All CEKs data should be available on all cluster nodes.
> > > We want to use system cache to get data synchronization feature "for
> > > free".
> > > 
> > > > We consider storing any metadata in system caches as antipattern from
> > > 
> > > our previous experience.
> > > 
> > > Very interesting!
> > > Can you share your experience?
> > > I didn't know that.
> > > I think we can use any convinient storage for CEK.
> > > Current design doesn't couple to system caches, so we can change 

[jira] [Created] (IGNITE-8278) Nightly built package does not run

[Max Shonichev (JIRA)]

Max Shonichev created IGNITE-8278:
-

 Summary: Nightly built package does not run
 Key: IGNITE-8278
 URL: https://issues.apache.org/jira/browse/IGNITE-8278
 Project: Ignite
  Issue Type: Bug
Reporter: Max Shonichev


Trying to run build from 
http://172.25.1.150:8111/viewLog.html?buildId=1209266=Releases_NightlyRelease_RunApacheIgniteNightlyRelease=artifacts

results in error:

{noformat}
Exception in thread "main" java.lang.ExceptionInInitializerError
at 
org.apache.ignite.startup.cmdline.CommandLineStartup.(CommandLineStartup.java:99)
Caused by: java.lang.IllegalStateException: Failed to parse version: 
2.5.0.20180415-0
at 
org.apache.ignite.lang.IgniteProductVersion.fromString(IgniteProductVersion.java:314)
at 
org.apache.ignite.internal.IgniteVersionUtils.(IgniteVersionUtils.java:71)
{noformat}

Possibly this is due to empty build timestamp in ignite.properties.
{noformat}
ignite.version=2.5.0.20180415
ignite.build=0
ignite.revision=DEV
ignite.rel.date=15042018
ignite.update.status.params=ver=2.5.0.20180415=apache-ignite
ignite.update.notifier.enabled.by.default=true
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

RE: Efficiently determining if cache keys belong to the local server node

[Stanislav Lukyanov]

// Bcc’ing off dev@ignite list for now as it seems to be rather a user-space 
discussion.

Hi,

Let me take a step back first. It seems a bit like an XY problem 
(https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem),
so I’d like to clarify the goals before diving into your current solution.

AFAIU you want to process certain entries in your cache locally on the server 
that caches these entries. Is that correct?
Have you looked at affinityRun and affinityCall 
(https://apacheignite.readme.io/docs/collocate-compute-and-data)? If yes, why 
they don’t work for you?
One limitation with these methods is that they accept a single key to process. 
Can you process your keys one by one, or do you need to access multiple keys at 
once?

Thanks,
Stan 

From: Raymond Wilson
Sent: 15 апреля 2018 г. 10:55
To: u...@ignite.apache.org
Cc: dev@ignite.apache.org
Subject: Efficiently determining if cache keys belong to the local server node

I have a type of query that asks for potentially large numbers of
information elements to be computed. Each element has an affinity key that
maps it to a server node through an IAffinityFunction.



The way the question is asked means that a single query broadcast to the
compute projection (owning the cache containing the source data for the
request) contains the identities of all the pieces of information needed to
be processed.



Each server node then scans the elements requested and identifies which
ones are its responsibility according to the affinity key.



Calculating the partition ID from the affinity key is simple (I have an
affinity function set up and supplied to the cache configuration, or I
could use IAffinity.GetPartition()), so the question became: How do I know
the server node executing the query is responsible for that partition, and
so should process this element? IE: I need to derive the vector of primary
or backup  partitions that this node is responsible for.



I can query the partition map and return it, like this:



ICacheAffinity affinity = Cache.Ignite.GetAffinity(Cache.Name);

public Dictionary primaryPartitions =
affinity.GetPrimaryPartitions(Cache.Ignite.GetCluster().GetLocalNode()).ToDictionary(k
=> k, v => true);



This lets me do a dictionary lookup, but its less efficient that having a
complete partition map with simple array lookup semantics, like this:



ICacheAffinity affinity = Cache.Ignite.GetAffinity(Cache.Name);

bool[] partitionMap = new bool[affinity.Partitions];



foreach (int partition in
affinity.GetBackupPartitions(Cache.Ignite.GetCluster().GetLocalNode()))

partitionMap[partition] = true;



This is a nice lookup for the query to determine which elements are its
responsibility from the overall request.



I’m not sure of the performance profile of this approach if I end up doing
it a lot, so I’m considering caching this lookup and invalidate it if any
event occurs that could modify the key -> partition map.



Questions:



   1. How big is the penalty when determining the full partition map like
   this?
   2. If I decide to invalidate the cached map, what are all the events I’d
   need to listen to?
  1. Rebalancing events?:I found CacheRebalancingEvent, but I’m not
  sure if this gives visibility to the points in time when a rebalanced
  partition becomes active on the new node and so the partition map changes
  2. Topology change events? (eg: adding a new backup node without
  rebalancing (if that is a thing) I looked for an event like that but have
  not found it so far, though I do know the affinity function can
respond to
  this via AssignPartitions()
   3. How do I provide my own affinity key mapper to for keys to partition
   IDs, but allow Ignite to map the partitions to nodes. The IAffinityFunction
   implementation requires both steps to be implemented. I’d prefer not to
   have the partition -> server mapping responsibility as this requires
   persistent configuration on the nodes to ensure stable mapping.



Thanks,

Raymond.

[GitHub] ignite pull request #3825: IGNITE-8274 Changed invoking java in sqlline.sh

[oleg-ostanin]

GitHub user oleg-ostanin opened a pull request:

https://github.com/apache/ignite/pull/3825

IGNITE-8274 Changed invoking java in sqlline.sh



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8274

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3825.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3825


commit 9adf473d1d4e86e61d3589d388afec8e1a67382e
Author: oleg-ostanin 
Date:   2018-04-16T10:25:21Z

Changed invoking java in sqlline.sh




---

[jira] [Created] (IGNITE-8277) Add utilities to check and display cache info

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-8277:


 Summary: Add utilities to check and display cache info
 Key: IGNITE-8277
 URL: https://issues.apache.org/jira/browse/IGNITE-8277
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Goncharuk






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8276) Cache 2 frequent suite timeout is master

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-8276:
--

 Summary: Cache 2 frequent suite timeout is master
 Key: IGNITE-8276
 URL: https://issues.apache.org/jira/browse/IGNITE-8276
 Project: Ignite
  Issue Type: Test
Reporter: Dmitriy Pavlov
Assignee: Ilya Lantukh


https://ci.ignite.apache.org/viewLog.html?buildId=1207201=buildResultsDiv=IgniteTests24Java8_Cache2
https://ci.ignite.apache.org/viewLog.html?buildId=1207203=buildResultsDiv=IgniteTests24Java8_Cache4

Cache 2 timed out in ~ 33% of runs
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache2_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv


{noformat}
[12:41:29]W: [org.apache.ignite:ignite-core] [2018-04-15 
09:41:29,356][ERROR][sys-stripe-12-#70556%dht.IgniteCacheMultiTxLockSelfTest4%][G]
 Failed to execute runnable.
[12:41:29]W: [org.apache.ignite:ignite-core] 
java.lang.AssertionError: null values in update for key: KeyCacheObjectImpl 
[part=9, val=9, hasValBytes=true]
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3578)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2770)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:537)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:199)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processForceKeyResponse(GridDhtCacheAdapter.java:175)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$9.onMessage(GridDhtTransactionalCacheAdapter.java:175)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$9.onMessage(GridDhtTransactionalCacheAdapter.java:173)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1400)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$MessageHandler.apply(GridDhtCacheAdapter.java:1382)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511)
[12:41:29]W: [org.apache.ignite:ignite-core]at 
java.lang.Thread.run(Thread.java:748)
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IEP-18: Transparent Data Encryption

[Vladimir Ozerov]

Hi Nikolay,

I noticed that some tickets have been created for this feature. Hopefully,
you haven't started implementation yet, because at this point community is
not aware of concrete architecture and proposed public API. We need to
continue discussion and come up with detailed design. Of most importance we
need to understand the following:
1) How encryption is configured - public API is needed (how to setup key
providers, how to enable encryption for specific caches, etc)
2) How encrypted cluster is started - normally, I do not expect any
additional manual actions from user once everything is configured.
Especially, in presence of auto-activation capability.
3) How and where encrypted cache keys are stored
4) Protocol of key's exchange between nodes
5) Detailed description of encryption flow - what algorithm is used? will
we use CBC?
6) What changes would be required to WAL logic (at the very least -
enumerate affected types of records)?
7) Would it be possible to enable/disable encryption on specific
cache/table in runtime?
8) Would these changes affect compression design anyhow?

Vladimir.

On Tue, Apr 10, 2018 at 5:30 PM, Vladimir Ozerov 
wrote:

> Hi NIkolay,
>
> Regarding system caches, rule of thumb here - do not use them. Keys should
> be stored near cache.
>
> As far as password:
> 1) Oracle auto-login wallet [1]
> 2) MySQL- password may be set inside configuration [2]
>
> I do not think that any kind of prompts are needed here out of the box.
> May be we could consider them in future, but at the moment it looks
> redundant to me.
>
> [1] https://docs.oracle.com/cd/E11882_01/network.112/
> e40393/asotrans.htm#CHDCCHBH
> [2] https://dev.mysql.com/doc/refman/5.7/en/keyring-
> encrypted-file-plugin.html
>
> On Mon, Apr 9, 2018 at 5:42 PM, Nikolay Izhikov 
> wrote:
>
>> Hello, Vladimir.
>>
>> > 1) Why do you propose to store CEK in separate cache?
>>
>> All CEKs data should be available on all cluster nodes.
>> We want to use system cache to get data synchronization feature "for
>> free".
>>
>> > We consider storing any metadata in system caches as antipattern from
>> our previous experience.
>>
>> Very interesting!
>> Can you share your experience?
>> I didn't know that.
>> I think we can use any convinient storage for CEK.
>> Current design doesn't couple to system caches, so we can change this
>> part at any moment.
>>
>> > 2) I do not think that decryption process should require any
>> "administrator" role and passwords.
>> > Instead, we can have a kind of pluggable interface which will provide
>> decrypted MEK on demand when it is needed.
>> > This should be pre-configured in advance on server node(s).
>> > AFAIK this is how a number of other vendors work.
>>
>> Can't agree with you, for now.
>> Can you please send me some info about implementation you refer to?
>> We study following papers to make current design:
>>
>> 1. ORACLE [1]
>>
>> To enable encryption one has to execute following statement:
>>
>> `ALTER SYSTEM SET ENCRYPTION KEY IDENTIFIED BY *clear_text_password*`
>> Or after database server restart - `ALTER SYSTEM SET WALLET OPEN
>> IDENTIFIED BY *clear_text_password*`
>> So yes, administrator is involved into server restart.
>> And no, it's not all automatic.
>>
>> 2. MSSQL [4] - AFAIK uses some Windows OS internal key storage to write
>> and read master key.
>> DB server process should have administrator(system) priveleges to access
>> that storage.
>>
>> > Instead, we can have a kind of pluggable interface which will provide
>> decrypted MEK on demand when it is needed.
>>
>> We, for sure, will have pluggable interface for providing MEK.
>> But, from my point of view, default implementation should use some
>> default java features.
>> I think we should use java key store.
>> It requires to have a clear text password to be loaded [3]
>>
>> [1] https://docs.oracle.com/cd/B19306_01/network.102/b14268/asot
>> rans.htm#BABJJAIG
>> [2] https://technet.microsoft.com/en-us/library/bb934049(v=sql.110).aspx
>> [3] https://docs.oracle.com/javase/8/docs/api/java/security/
>> KeyStore.html#load-java.io.InputStream-char:A-
>> [4] https://technet.microsoft.com/en-us/library/bb934049(v=sql.110).aspx
>>
>> > 3) CEK decryption should not be tied to MEK decryption.
>>
>> MEK will be available from the moment it obtained to the node stop.
>> So, we can decrypt existing or encrypt newly created CEK whenever it
>> necessary.
>>
>> > 4) I do not think that SSL should be a strict requirement. It is up to
>> the> user to asses the risks.
>>
>> I'm fully OK with it.
>> Let's remove that restriction.
>>
>> В Пн, 09/04/2018 в 11:35 +0300, Vladimir Ozerov пишет:
>> > Hi Nikolay,
>> >
>> > First of all thank you for excellent summary. Two-tiered key management
>> is
>> > well respected technique and makes perfect sense to me. However, several
>> > questions regarding architecture arises:
>>
>> > 3) CEK decryption should not be tied to MEK decryption. Main reason -
>> 

Re: Mass TC Run-All during this weekend

[Dmitry Pavlov]

Hi Igniters,

 New mass run results from this weekend: tests 309 suites 24
 It does not mean 97 tests were fixed (it can be muted, not reproduced
during these 30 runs, fixied, etc), but result is relatively positive.

 Full report can be found in MTCGA wiki attachments:
https://cwiki.apache.org/confluence/pages/viewpageattachments.action?pageId=73631266

Sincerely,
Dmitriy Pavlov

вс, 25 мар. 2018 г. в 15:50, Dmitry Pavlov :

> Hi Igniters,
>
> We have the positive dynamics in the resolving MTCGA JIRA tasks, and I
> must mention the contributions of Aleksey Plekhanov, Nikolay Izhikov, Peter
> Ivanov, Anton Kalashnikov, Alexey Goncharuk, Sergey Chugunov, Turik
> Campbell.
>
> Mass run-alls have been finished (
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAll=%3Cdefault%3E=buildTypeStatusDiv
>  ).
> Unfortunately current mass run shows more failures than previous. Current
> results: tests 406 suites 26 (Previous was: tests 371 suites 21)
>
> The explanation of this may be randomized nature of fails, and that some
> tests fail this time but did not fail in the past. Also it is possible that
> new issues were introduced. I'll generate report on Monday and attach it to
> https://cwiki.apache.org/confluence/pages/viewpageattachments.action?pageId=73631266
>
> Sincerely,
> Dmitriy Pavlov
>
> пт, 23 мар. 2018 г. в 22:08, Denis Magda :
>
>> Ignite the Team City :)
>>
>> --
>> Denis
>>
>> On Fri, Mar 23, 2018 at 7:00 AM, Dmitry Pavlov 
>> wrote:
>>
>> > Hi Igniters,
>> >
>> > To get the full picture of the tests now, I'm not going to, I'm going to
>> > run a massive Run All this weekend.
>> >
>> > I'm going to start it at 20:00 MSK (17:00 UTC).
>> >
>> > Are there any objections? If you need to run your branches, rearrange
>> them
>> > at the beginning of the queue ..
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>>
>

[jira] [Created] (IGNITE-8275) PDS (Compatibility): FoldersReuseCompatibilityTest

[Peter Ivanov (JIRA)]

Peter Ivanov created IGNITE-8275:


 Summary: PDS (Compatibility): FoldersReuseCompatibilityTest
 Key: IGNITE-8275
 URL: https://issues.apache.org/jira/browse/IGNITE-8275
 Project: Ignite
  Issue Type: Bug
Reporter: Peter Ivanov
Assignee: Dmitriy Govorukhin


Some weird behaviour of *IgniteCompatibilityBasicTestSuite* test suite:
* {{mvn test}} run reports 8 successful tests (see 
https://ci.ignite.apache.org/viewLog.html?buildId=1211334=buildResultsDiv=IgniteTests24Java8_PdsCompatibility)
* {{mvn surefire:test}} run reports 2 failures (see 
https://ci.ignite.apache.org/viewLog.html?buildId=1211336=buildResultsDiv=IgniteTests24Java8_PdsCompatibility)
{code}
[2018-04-16 09:41:29,823][ERROR][main][root] Test failed.
junit.framework.AssertionFailedError: Directory [binary_meta, 
d0d49035_c611_4836_8956_8fdf220d7f46] is expected to exist 
[/data/teamcity/work/c182b70f2dfa6507/work/binary_meta/d0d49035_c611_4836_8956_8fdf220d7f46]
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.assertTrue(Assert.java:22)
at junit.framework.TestCase.assertTrue(TestCase.java:192)
at 
org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertDirectoryExist(FoldersReuseCompatibilityTest.java:258)
at 
org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertPdsDirsDefaultExist(FoldersReuseCompatibilityTest.java:231)
at 
org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.runFoldersReuse(FoldersReuseCompatibilityTest.java:128)
at 
org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_2(FoldersReuseCompatibilityTest.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:176)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2080)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:140)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1995)
at java.lang.Thread.run(Thread.java:745)
[org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_2]
 junit.framework.AssertionFailedError: Directory [binary_meta, 
d0d49035_c611_4836_8956_8fdf220d7f46] is expected to exist 
[/data/teamcity/work/c182b70f2dfa6507/work/binary_meta/d0d49035_c611_4836_8956_8fdf220d7f46]
[org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_2]
 junit.framework.AssertionFailedError: Directory [binary_meta, 
d0d49035_c611_4836_8956_8fdf220d7f46] is expected to exist 
[/data/teamcity/work/c182b70f2dfa6507/work/binary_meta/d0d49035_c611_4836_8956_8fdf220d7f46]
at 
org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertDirectoryExist(FoldersReuseCompatibilityTest.java:258)
at 
org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.assertPdsDirsDefaultExist(FoldersReuseCompatibilityTest.java:231)
at 
org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.runFoldersReuse(FoldersReuseCompatibilityTest.java:128)
at 
org.apache.ignite.compatibility.persistence.FoldersReuseCompatibilityTest.testFoldersReuseCompatibility_2_2(FoldersReuseCompatibilityTest.java:97)
{code} 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8274) Wrong invoking of java in sqlline.sh

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-8274:
--

 Summary: Wrong invoking of java in sqlline.sh
 Key: IGNITE-8274
 URL: https://issues.apache.org/jira/browse/IGNITE-8274
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Konstantinov
Assignee: Oleg Ostanin


Currently java invoked w\o using JAVA_HOME variable
Need to be fixed using $JAVA_HOME/bin/java as in others ignite sh scripts



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8273) CPP: Implement AffinityRun and AffinityCall Compute methods

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-8273:
---

 Summary: CPP: Implement AffinityRun and AffinityCall Compute 
methods
 Key: IGNITE-8273
 URL: https://issues.apache.org/jira/browse/IGNITE-8273
 Project: Ignite
  Issue Type: Task
  Components: platforms
Affects Versions: 2.4
Reporter: Igor Sapego


We need to implement {{AffinityRun}} and {{AffinityCall}} features for C++ 
thick client.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: affinityRun/Call in C++

[Igor Sapego]

Guys,

I've created a ticket: [1]

[1] - https://issues.apache.org/jira/browse/IGNITE-8273

Best Regards,
Igor

On Wed, Apr 11, 2018 at 2:44 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Is there a ticket? Let's create one if not.
>
> -Val
>
> On Tue, Apr 10, 2018 at 6:17 AM, Vladimir Ozerov 
> wrote:
>
> > Val,
> >
> > They are simply not implemented yet. I am not aware of concrete plans to
> > support them.
> >
> > On Mon, Apr 9, 2018 at 11:33 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Guys,
> > >
> > > Is there a way to run collocated compute job in C++? I can't find
> > > affinityRun and affinityCall method in C++ compute API, am I missing
> > > something? If we really don't have them, is there any particular reason
> > for
> > > this and/or plans to add them?
> > >
> > > -Val
> > >
> >
>

[GitHub] ignite pull request #3824: IGNITE-7778 User's authentication data must be pe...

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/3824

IGNITE-7778  User's authentication data must be persisted on cluster with 
disabled persistence



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7778

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3824.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3824


commit 806b8cc0928bff85bf25d75ae807483b811d33c3
Author: tledkov-gridgain 
Date:   2018-04-13T13:46:32Z

IGNITE-7778: add test

commit 220aadd7c8309f39c9208fce2b2d291bda0b5891
Author: tledkov-gridgain 
Date:   2018-04-13T16:04:42Z

IGNITE-7778: save the progress




---

[GitHub] ignite pull request #3822: IGNITE-8097 Java thin client: throw handshake exc...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3822


---

Re: Apache Ignite nightly release builds

[Petr Ivanov]

Hi, Igniters!


I’m glad to inform that starting today Apache Ignite Nightly Builds provide 
Nuget and Maven stagings at MyGet [1].

Instructions on connecting to these feeds are here:
 — Nuget [2]
 — Maven [3]


Enjoy.


[1] https://www.myget.org/gallery/apache-ignite-nightly 

[2] http://docs.myget.org/docs/walkthrough/getting-started-with-nuget 

[3] http://docs.myget.org/docs/walkthrough/getting-started-with-maven 


> On 24 Mar 2018, at 11:42, Dmitriy Setrakyan  wrote:
> 
> Thanks, Denis. It should be added to the download page, I updated the ticket.
> 
> On Sat, Mar 24, 2018 at 5:48 AM, Denis Magda  > wrote:
> Created a JIRA ticket for that:
> https://issues.apache.org/jira/browse/IGNITE-8040 
> 
> 
> --
> Denis
> 
> On Fri, Mar 23, 2018 at 1:27 AM, Dmitriy Setrakyan  >
> wrote:
> 
> > Awesome! Finally instead of asking our users to build from the master, we
> > can provide a link to the nightly build instead.
> >
> > Denis, can you please add these links to the website?
> >
> > D.
> >
> > On Thu, Mar 22, 2018 at 1:27 PM, Petr Ivanov  > > wrote:
> >
> >> It works, thanks!
> >>
> >>
> >> Here is updated links for Artifacts and Changes respectively with silent
> >> guest login (can be added to bookmarks):
> >> * https://ci.ignite.apache.org/viewLog.html?buildId=lastSucces 
> >> 
> >> sful=Releases_NightlyRelease_RunApacheIgnite
> >> NightlyRelease=artifacts=1
> >> * https://ci.ignite.apache.org/viewLog.html?buildId=lastSucces 
> >> 
> >> sful=Releases_NightlyRelease_RunApacheIgnite
> >> NightlyRelease=buildChangesDiv=1
> >>
> >>
> >>
> >> > On 22 Mar 2018, at 13:06, Vitaliy Osipov  >> > > wrote:
> >> >
> >> > 
> >>
> >>
> >
> 

[GitHub] ignite pull request #3795: IGNITE-8129: fix test: setup default SSL context ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3795


---

[jira] [Created] (IGNITE-8272) Add documentation for normalization preprocessor (release 2.5)

[Anton Dmitriev (JIRA)]

Anton Dmitriev created IGNITE-8272:
--

 Summary: Add documentation for normalization preprocessor (release 
2.5)
 Key: IGNITE-8272
 URL: https://issues.apache.org/jira/browse/IGNITE-8272
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.5
Reporter: Anton Dmitriev
Assignee: Anton Dmitriev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8271) Add documentation for partition based dataset (release 2.5)

[Anton Dmitriev (JIRA)]

Anton Dmitriev created IGNITE-8271:
--

 Summary: Add documentation for partition based dataset (release 
2.5)
 Key: IGNITE-8271
 URL: https://issues.apache.org/jira/browse/IGNITE-8271
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.5
Reporter: Anton Dmitriev
Assignee: Anton Dmitriev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8270) Add documentation for MLP (release 2.5)

[Anton Dmitriev (JIRA)]

Anton Dmitriev created IGNITE-8270:
--

 Summary: Add documentation for MLP (release 2.5)
 Key: IGNITE-8270
 URL: https://issues.apache.org/jira/browse/IGNITE-8270
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.5
Reporter: Anton Dmitriev
Assignee: Anton Dmitriev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8269) Add documentation to decision tree (release 2.5)

[Anton Dmitriev (JIRA)]

Anton Dmitriev created IGNITE-8269:
--

 Summary: Add documentation to decision tree (release 2.5)
 Key: IGNITE-8269
 URL: https://issues.apache.org/jira/browse/IGNITE-8269
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.5
Reporter: Anton Dmitriev
Assignee: Anton Dmitriev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8268) Add documentation for linear regression (release 2.5)

[Anton Dmitriev (JIRA)]

Anton Dmitriev created IGNITE-8268:
--

 Summary: Add documentation for linear regression (release 2.5)
 Key: IGNITE-8268
 URL: https://issues.apache.org/jira/browse/IGNITE-8268
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.5
Reporter: Anton Dmitriev
Assignee: Anton Dmitriev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache Ignite RPM packaging (Stage II)

[Petr Ivanov]

> On 15 Apr 2018, at 10:19, Ilya Kasnacheev  wrote:
> 
> Hello!
> 
> With existing binary archive, user can move directories from
> apache-ignite-fabric/libs/optional to apache-ignite-fabric/libs to activate
> them.
> 
> But with RPM, user should not contemplate moving directories from
> /usr/lib/apache-ignite/optional to /usr/lib/apache-ignite.

I always thought of that option as COPYING optional libs, not MOVING.


> 
> 
> User lacks permissions for that operation and it would defeat the purpose
> of having a RPM (imagine trying to upgrade Ignite RPM version with a setup
> like that). "Turning distribution into Slackware" should not be an offering.

Can’t imagine use case where Apache Ignite software is installed by one user, 
and run by another, without sudo/root permissions.
Yet, ignite user’s login shell is disabled indeed and without sudo/root 
permissions optional libs cannot be copied.
Also — the symlinks is a good idea, but I’ve already thought of it and I’m 
planning addition of special utility for enabling / disabling optional libs 
(like a2enable/a2disable) in next development iteration.


> 
> How it could work: Imagine we create /var/lib/ignite, use it as
> IGNITE_HOME, add symlinks from files in /usr/lib to /var/lib/ignite. This
> way, we can add and remove symlinks to modules in optional/, and thus
> enable and disable them as user sees fit.

By linux file hierarchy convention, home dir should be in /usr/lib. /var/lib/ 
is currently used for working files (MySQL alike).


> 
> This also answers the problem of "what's IGNITE_HOME for visor launched
> from /usr/bin”

That is addressed in separate task and will be fixed with minor startup script 
redesign with universal location-independent solution.


> 
> But obviously this will require dedication and effort.

No problems with efforts after final design is discussed an agreed.


> 
> 
> -- 
> Ilya Kasnacheev
> 
> 2018-04-14 8:03 GMT+03:00 Peter Ivanov :
> 
>> Current packages design (after installation) does not differ from binary
>> archive - everything works (except necessity to run service instead
>> ignite.sh) just the same way, including libs/optional.
>> 
>> Also, there can be issues with system JDK version by default, but every
>> problem will be in journalctl and/or  /var/log, and package has strong
>> dependency on any implementation of JDK 1.8.
>> 
>> 
>> I am lacking enough feedback about Apache Ignite from packages from real
>> users, don’t know real use cases so still "moving in the dark".
>> 
>> 
>> On Fri, 13 Apr 2018 at 22:18, Denis Magda  wrote:
>> 
>>> Ilya,
>>> 
>>> Thanks for your inputs. The reason why we decided to split Ignite into
>>> several packages mimics the reason why Java community introduced modular
>>> subsystem for JDK. That's all about size. Ignite distribution is too big,
>>> and we're trying to separate it into several components so that people
>> can
>>> install only the features they need.
>>> 
>>> The point of a package is to ship something into root file system that
>> can
 be used from root file system. If cpp files require compilation we
>> should
 not ship them, or ship them to 'examples'. Ditto with benchmarks. If
 there's no mechanism to add optional libs to Ignite classpath, we
>> should
 not ship optional libs. Moreover, some of 'optional' modules such as
>> yarn
 don't make sense here because they're not supposed to be used with
 standalone Ignite.
>>> 
>>> 
>>> Agree that we need to ship the code that is ready to be run. As for the
>>> classpath thing, if an optional package is installed into the root (core)
>>> package directory, then its jars have to be added to "ignite/libs"
>> folder.
>>> After that, the one needs to restart a cluster node, nd it will add the
>>> just installed optional libs to the classpath. *Petr*, does it work this
>>> way or can be implemented this way to address Ilya's concerns?
>>> 
>>> --
>>> Denis
>>> 
>>> On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com>
>>> wrote:
>>> 
 2018-04-13 7:42 GMT+03:00 Peter Ivanov :
 
> On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com
> 
> wrote:
> 
>> 
>> Moreover I did not find a way to start service if default installed
>>> JVM
> is
>> Java 7 :( I understand it's EOL, still this is something that hit
>> me.
> 
> 
> Apache Ignite >=2.4 does not support Java 7 - it is said in
 documentation,
> DEVNOTES and even in startup scripts.
> 
> I have Java 8 too, but I could not get service from package to start
 Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
 matter). Is it possible to specify it while running packaged Ignite?
 
 
> 
>> 
>> apache-ignite-libs is a totally unexpected package name.
>>> apache-ignite
> core
>> doesn't depend on it.