Re: IGNITE-7107 Apache Ignite RPM packages

[Petr Ivanov]

Hi, Denis.


I was glad to contribute.

Concerning DEB package — for 2.4 release I’m planning to introduce instructions 
and spec files for conversion RPM package to DEB, substantially reducing 
efforts for now because of file structure and install scripts inheritance.
Later, of course, I’m going to prepare fully synchronised source-based package 
creation process, packages from which are meant for inclusion into official 
repositories.



> On 28 Dec 2017, at 22:53, Denis Magda  wrote:
> 
> Peter, thanks for this Christmas/New Year gift for the community! :)
> 
> I’ll be back soon putting together a documentation for this capability.
> 
> BTW, what’s the plan in regards DEB packages?
> https://issues.apache.org/jira/browse/IGNITE-7108 
> 
> 
> Do you consider fitting them in AI 2.4 release?
> 
> —
> Denis
> 
>> On Dec 28, 2017, at 5:23 AM, Sergey Kozlov  wrote:
>> 
>> Guys
>> 
>> Thanks for your efforts to make it available in 2.4!
>> 
>> On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura  wrote:
>> 
>>> Guys,
>>> 
>>> I've merged change to master branch. Thanks a lot!
>>> 
>>> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov  wrote:
 Fixed remarks and nitpicks.
 
 Also, I purpose to delay service autostart implementation until we’ll
>>> get some first-hand usability feedback.
 Added notes to DEVNOTES.txt about how to start service.
 
 
> On 28 Dec 2017, at 14:32, Ilya Kasnacheev 
>>> wrote:
> 
> Hello once more!
> 
> Two more nitpicks:
> 
> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
> work since a) visor wants to write to /usr/share, and b) we moved it to
> examples for that.
> Please remove that alternative for now, create a ticket.
> 
> rpm -ql apache-ignite | grep \\.java
> some yardstick and ml sources are there. I don't know who is there to
>>> blame
> - Ignite release process likely, not RPM building - but they shouldn't
>>> be
> here I guess?
> 
> Regards!
> 
> --
> Ilya Kasnacheev
> 
> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev :
> 
>> Hello again!
>> 
>> I have reviewed the modified patch. All the things in my critical list
>> were fixed. I have just two minor remarks:
>> 
>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>> works just as well from there as it was from our distribution. visor
>> doesn't, hence it stays in examples.
>> - I'm a bit confused that we no longer auto-start service after
>>> package is
>> installed. You have to do
>> sudo systemctl start apache-ign...@default-config.xml
>> manually. I'm used to packages that start the thing they install
>> immediately. Of course we can just document that clearly on downloads
>>> pages
>> so people won't get lost. What do you all think? Should Ignite
>>> auto-start
>> with default config? Is it possible to do so, but make possible to
>>> turn it
>> off.
>> 
>> So from my side I recommend this pull request for inclusion after 1) is
>> done and 2) is discussed.
>> 
>> Also, would be nice if you create additional JIRA tickets for issues
>>> in my
>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>> working) to be implemented in future releases.
>> 
>> Regards,
>> 
>> --
>> Ilya Kasnacheev
>> 
>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov :
>> 
>>> Removed replacement for default-config.xml.
>>> Also reimplemented service to be able to run multiple instances of
>>> Ignite.
>>> 
>>> See updated PR [1].
>>> 
>>> 
>>> [1] https://github.com/apache/ignite/pull/3171 <
>>> https://github.com/apache/ignite/pull/3171>
>>> 
>>> 
>>> 
 On 25 Dec 2017, at 18:57, Pavel Tupitsyn 
>>> wrote:
 
 PDS and discovery settings are unrelated to the packaging task.
 Andrey is right, just use existing default config.
 
 On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov 
>>> wrote:
 
> Can we change default configuration file then?
> So that both binary and package deliveries contained PDS turned on
>>> by
> default?
> 
> And what about Multicast Discovery?
> 
> 
>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan >>> 
> wrote:
>> 
>> I agree with Andrey. The product should be identical regardless of
>>> how
> you
>> download and install it.
>> 
>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura 
>>> wrote:
>> 
>>> Hi,
>>> 
>>> I think we should provide the same default configuration for all
>>> binary builds. From my point of view RPM/DEB/etc packages should
>>> use
>>> default configuration file like to standard binary release.
>>> 
>>> On Mon

[jira] [Created] (IGNITE-7338) can get value by entry.getValue but cann't get value by entry.getKey

[dean (JIRA)]

dean created IGNITE-7338:


 Summary: can get value by entry.getValue but cann't get value by 
entry.getKey
 Key: IGNITE-7338
 URL: https://issues.apache.org/jira/browse/IGNITE-7338
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.9
Reporter: dean
Priority: Critical






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7337) Spark Data Frames: support saving a data frame in Ignite

[Valentin Kulichenko (JIRA)]

Valentin Kulichenko created IGNITE-7337:
---

 Summary: Spark Data Frames: support saving a data frame in Ignite
 Key: IGNITE-7337
 URL: https://issues.apache.org/jira/browse/IGNITE-7337
 Project: Ignite
  Issue Type: New Feature
  Components: spark
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Assignee: Nikolay Izhikov
Priority: Critical
 Fix For: 2.4


Once Ignite data source for Spark is implemented, we need to add an ability to 
store a data frame in Ignite. Most likely if should be enough to provide 
implementation for the following traits:
* {{InsertableRelation}}
* {{CreatableRelationProvider}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7336) Improve peer class loading documentation

[Valentin Kulichenko (JIRA)]

Valentin Kulichenko created IGNITE-7336:
---

 Summary: Improve peer class loading documentation
 Key: IGNITE-7336
 URL: https://issues.apache.org/jira/browse/IGNITE-7336
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Assignee: Prachi Garg


[Zero Deployment|https://apacheignite.readme.io/docs/zero-deployment] page 
should be updated with the following:
* Applicability of the feature (where it can and can't be used).
* Warning about anonymous classes and lambdas.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: ATOMIC caches consistency - IEP-12

[Dmitriy Setrakyan]

On Thu, Dec 28, 2017 at 9:12 AM, Yakov Zhdanov 
wrote:

> Dmitry,
>
> This will not solve the problem:
>
> 1. new primary election may be affinity function dependent
>

Yes, of course.


> 2. If you want to introduce order in updating backup copies then you lose
> parallelism. This will increase ops latency.
>

I did not mean waiting till an update is finished. I suggested waiting an
ask for the message is received. In my view, this will significantly reduce
the possibility of out-of-sync  data.


>
> My suggestion should not affect current performance on stable and on
> unstable topologies.
>
> Btw, I have just got an idea - if we need to rebalance partition to more
> than 1 node then we need to scan it only once. I believe now we do it each
> time from very beginning.
>

I am not sure what this really means, but you should file a ticket for it
with a proper description.

D.

[jira] [Created] (IGNITE-7335) DatabaseMetaData getTables error

[Maxwell Pettit (JIRA)]

Maxwell Pettit created IGNITE-7335:
--

 Summary: DatabaseMetaData getTables error
 Key: IGNITE-7335
 URL: https://issues.apache.org/jira/browse/IGNITE-7335
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.3
 Environment: Default ignite cluster with a table created via SQL from 
the JDBC driver
Reporter: Maxwell Pettit


Exception when calling the java.sql.DatabaseMetaData.getTables(String catalog, 
String schemaPattern, String tableNamePattern, String[] types) method.  
Exception is listed below:

Caused by: java.sql.SQLException: Failed to communicate with Ignite cluster.
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:681)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinDatabaseMetadata.getTables(JdbcThinDatabaseMetadata.java:740)
at 
org.apache.commons.dbcp.DelegatingDatabaseMetaData.getTables(DelegatingDatabaseMetaData.java:604)
... 20 more
Caused by: java.io.IOException: Failed to read incoming message (not enough 
data).
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.read(JdbcThinTcpIo.java:406)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.read(JdbcThinTcpIo.java:384)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinTcpIo.sendRequest(JdbcThinTcpIo.java:327)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:668)
... 26 more



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: IGNITE-7107 Apache Ignite RPM packages

[Denis Magda]

Peter, thanks for this Christmas/New Year gift for the community! :)

I’ll be back soon putting together a documentation for this capability.

BTW, what’s the plan in regards DEB packages?
https://issues.apache.org/jira/browse/IGNITE-7108 


Do you consider fitting them in AI 2.4 release?

—
Denis

> On Dec 28, 2017, at 5:23 AM, Sergey Kozlov  wrote:
> 
> Guys
> 
> Thanks for your efforts to make it available in 2.4!
> 
> On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura  wrote:
> 
>> Guys,
>> 
>> I've merged change to master branch. Thanks a lot!
>> 
>> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov  wrote:
>>> Fixed remarks and nitpicks.
>>> 
>>> Also, I purpose to delay service autostart implementation until we’ll
>> get some first-hand usability feedback.
>>> Added notes to DEVNOTES.txt about how to start service.
>>> 
>>> 
 On 28 Dec 2017, at 14:32, Ilya Kasnacheev 
>> wrote:
 
 Hello once more!
 
 Two more nitpicks:
 
 right now we install /usr/bin/ignitevisorcmd alternative, but it does't
 work since a) visor wants to write to /usr/share, and b) we moved it to
 examples for that.
 Please remove that alternative for now, create a ticket.
 
 rpm -ql apache-ignite | grep \\.java
 some yardstick and ml sources are there. I don't know who is there to
>> blame
 - Ignite release process likely, not RPM building - but they shouldn't
>> be
 here I guess?
 
 Regards!
 
 --
 Ilya Kasnacheev
 
 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev :
 
> Hello again!
> 
> I have reviewed the modified patch. All the things in my critical list
> were fixed. I have just two minor remarks:
> 
> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
> works just as well from there as it was from our distribution. visor
> doesn't, hence it stays in examples.
> - I'm a bit confused that we no longer auto-start service after
>> package is
> installed. You have to do
> sudo systemctl start apache-ign...@default-config.xml
> manually. I'm used to packages that start the thing they install
> immediately. Of course we can just document that clearly on downloads
>> pages
> so people won't get lost. What do you all think? Should Ignite
>> auto-start
> with default config? Is it possible to do so, but make possible to
>> turn it
> off.
> 
> So from my side I recommend this pull request for inclusion after 1) is
> done and 2) is discussed.
> 
> Also, would be nice if you create additional JIRA tickets for issues
>> in my
> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
> working) to be implemented in future releases.
> 
> Regards,
> 
> --
> Ilya Kasnacheev
> 
> 2017-12-26 15:25 GMT+03:00 Petr Ivanov :
> 
>> Removed replacement for default-config.xml.
>> Also reimplemented service to be able to run multiple instances of
>> Ignite.
>> 
>> See updated PR [1].
>> 
>> 
>> [1] https://github.com/apache/ignite/pull/3171 <
>> https://github.com/apache/ignite/pull/3171>
>> 
>> 
>> 
>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn 
>> wrote:
>>> 
>>> PDS and discovery settings are unrelated to the packaging task.
>>> Andrey is right, just use existing default config.
>>> 
>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov 
>> wrote:
>>> 
 Can we change default configuration file then?
 So that both binary and package deliveries contained PDS turned on
>> by
 default?
 
 And what about Multicast Discovery?
 
 
> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan >> 
 wrote:
> 
> I agree with Andrey. The product should be identical regardless of
>> how
 you
> download and install it.
> 
> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura 
>> wrote:
> 
>> Hi,
>> 
>> I think we should provide the same default configuration for all
>> binary builds. From my point of view RPM/DEB/etc packages should
>> use
>> default configuration file like to standard binary release.
>> 
>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>  wrote:
>>> Hello Igniters!
>>> 
>>> What's your take on enabling PDS in 2.4 in default config? This
>> way
>> we
>>> could enable it in RPM build with easy heart.
>>> 
>>> The rest of your answers seem spot on. I'll happily review your
>> amended
>>> patch when it is ready (later this week?)
>>> 
>>> --
>>> Ilya Kasnacheev
>>> 
>>> 2017-12-22 18:27 GMT+03:00 vveider :
>>> 
 Ilya Kasnacheev wrote
> I have noticed that both spec and DEVNOTES are made to use t

Re: Handling slashes in cache names

[Dmitriy Setrakyan]

On Thu, Dec 28, 2017 at 9:22 AM, Stanislav Lukyanov 
wrote:

> Hi all ,
>
> I’ve implemented an approach of encoding unsafe characters in the cache
> names for persistent storage directories. You can find it at
> https://github.com/gridgain/apache-ignite/tree/ignite-7264.
> How it works now is: 1) all characters outside of the [a-zA-Z0-9_-] class
> are replaced with their hex value (seems to be the easiest way);


I would surround such replacements with "_", e.g. "myCacheName_somesymbol_".


> 2) a hash of the cache name is added at the end of the name to avoid
> case-insensitive collisions.
> There is still a tiny chance of hitting two cache names that are equal
> ignoring case which also have the same hash, but that’s really unlikely.
>

Here I am confused. I think the cache names should be case insensitive at
all times. I seriously doubt enforcing this rule would cause problems. If
we enforce this rule at cache creation time, then we would not have to add
a hashcode at the end.


>
> It seems that there are no complications with this approach.
> The cache name to directory mapping is like
>   mycache -> cache-mycache-f19fd83d
>   my/cool/cache -> cache-my2fcool2fcache
>

As mentioned above, I would prefer "cache-my_2f_cool_2f_cache"


>   my!@#$%^&()cache -> cache-my21402324255e262829cache-84ba3e99
>
> Turns out the persistence is not the only place that doesn’t like special
> symbols in cache names – I also got an exception from MBean registration
> when creating a cache with ‘*’ or ‘?’. Filed https://issues.apache.org/
> jira/browse/IGNITE-7334 for that.
>
> Please let me know if you have any comments.
>
> Thanks,
> Stan
>
> From: Stanislav Lukyanov
> Sent: 25 декабря 2017 г. 18:09
> To: dev@ignite.apache.org
> Subject: Handling slashes in cache names
>
> Hi all,
>
> I’m looking into https://issues.apache.org/jira/browse/IGNITE-7264, and I
> need some guidance on what’s the best way to approach it.
>
> The problem is that cache names are not restricted, but if persistence is
> enabled the cache needs to have a corresponding directory on the file
> system (“cache-…”) which can’t be created if the cache name contains
> certain characters (or a reserved system name).
>
> A straightforward approach would be to check if a cache name is allowed on
> the local system (e.g. via `Paths.get(name)`) and fail to create cache if
> it isn’t, but I’m a bit concerned with the consistency of the behavior (the
> same cache name be allowed on one system and not on another).
> I think a better way would be to replace special characters (say, all
> non-alphanumeric characters) with underscores in file names (not changing
> the cache configuration). Would this be OK? Are there any risks I’m not
> considering?
>
> WDYT?
>
> Thanks,
> Stan
>
>

Re: Removing "fabric" from Ignite binary package name

[Denis Magda]

Petr, thanks for solving it!

Hope that Anton V. or some other build master will double-check the changes and 
merge them.

—
Denis

> On Dec 28, 2017, at 8:29 AM, Petr Ivanov  wrote:
> 
> IGNITE-7251 is done, needs review and some additional tests. See PR #3315 [1].
> 
> 
> [1] https://github.com/apache/ignite/pull/3315 
> 
> 
> 
> 
>> On 20 Dec 2017, at 23:15, Denis Magda  wrote:
>> 
>> Petr, thanks, such a swift turnaround! 
>> 
>> Have you found the one who can asses and review the changes?
>> 
>> Maintainers label might be helpful. Just ping them directly:
>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers
>>  
>> 
>> 
>> 
>> —
>> Denis
>> 
>>> On Dec 20, 2017, at 12:24 AM, Petr Ivanov  wrote:
>>> 
>>> Assigned myself — done the same work while preparing RPM package.
>>> But for fixing DEVNOTES.txt waiting for review and merge of IGNITE-7107 [1].
>>> 
>>> 
>>> [1] https://issues.apache.org/jira/browse/IGNITE-7107
>>> 
>>> 
>>> 
 On 19 Dec 2017, at 22:55, Denis Magda  wrote:
 
 All the bids were accepted and the verdict is executed:
 https://issues.apache.org/jira/browse/IGNITE-7251 
 
 
 Who is ready to pick this up?
 
 —
 Denis
 
> On Dec 19, 2017, at 5:35 AM, Anton Vinogradov  
> wrote:
> 
> +1б фо шур
> 
> On Tue, Dec 19, 2017 at 9:59 AM, Vladimir Ozerov 
> wrote:
> 
>> +1б вуаштшеудн
>> 
>> On Tue, Dec 19, 2017 at 2:34 AM, Valentin Kulichenko <
>> valentin.kuliche...@gmail.com> wrote:
>> 
>>> +1
>>> 
>>> On Mon, Dec 18, 2017 at 12:24 PM, Dmitriy Setrakyan <
>> dsetrak...@apache.org
 
>>> wrote:
>>> 
 +1 (completely agree)
 
 On Mon, Dec 18, 2017 at 10:21 AM, Denis Magda 
>> wrote:
 
> Igniters,
> 
> Apache Ignite binary releases still include “fabric” word in their
>>> names:
> https://ignite.apache.org/download.cgi#binaries <
> https://ignite.apache.org/download.cgi#binaries>
> 
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> 
> Proposal! Remove “fabric” from the name and have the binary releases
 named
> as - apache-ignite-{version}-bin.
> 
> If we’re in consensus then let’s make the change in 2.4.
> 
> —
> Denis
> 
 
>>> 
>> 
 
>>> 
>> 
> 

[GitHub] ignite pull request #3318: IGNITE-7264: Caches with forward slash "/" in nam...

[slukyano]

GitHub user slukyano opened a pull request:

https://github.com/apache/ignite/pull/3318

IGNITE-7264: Caches with forward slash "/" in names cause problems for PDS



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7264

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3318.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3318


commit fcacf73d2db54865fa3ff1387e04dfc2fe6ca4b8
Author: Stanislav Lukyanov 
Date:   2017-12-28T09:26:10Z

IGNITE-7264: removed unused method

commit f796b61b8f25c32022c6a3041e46deea2002ad0a
Author: Stanislav Lukyanov 
Date:   2017-12-28T14:35:06Z

IGNITE-7264: added escaping of unsafe characters in persistent storage names

commit 62d7c38f4cc3c561420ae9f0289acf646011a04a
Author: Stanislav Lukyanov 
Date:   2017-12-28T16:13:36Z

IGNITE-7264: restored afterTest action in IgnitePdsExoticCacheNamesTest

commit f7bbcc63d91953d8188566442a36661ad4819ced
Author: Stanislav Lukyanov 
Date:   2017-12-28T17:42:17Z

IGNITE-7264: added handling of long cache names




---

[GitHub] ignite pull request #3317: IGNITE-7334: added escaping of special characters...

[slukyano]

GitHub user slukyano opened a pull request:

https://github.com/apache/ignite/pull/3317

IGNITE-7334: added escaping of special characters to MBean registrati…

…on code

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7334

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3317.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3317


commit 8650dddb717a32a05071fda0053ac89029bf4a13
Author: Stanislav Lukyanov 
Date:   2017-12-28T17:13:08Z

IGNITE-7334: added escaping of special characters to MBean registration code




---

RE: Handling slashes in cache names

[Stanislav Lukyanov]

Hi all ,

I’ve implemented an approach of encoding unsafe characters in the cache names 
for persistent storage directories. You can find it at 
https://github.com/gridgain/apache-ignite/tree/ignite-7264.
How it works now is: 1) all characters outside of the [a-zA-Z0-9_-] class are 
replaced with their hex value (seems to be the easiest way); 2) a hash of the 
cache name is added at the end of the name to avoid case-insensitive collisions.
There is still a tiny chance of hitting two cache names that are equal ignoring 
case which also have the same hash, but that’s really unlikely.

It seems that there are no complications with this approach.
The cache name to directory mapping is like
  mycache -> cache-mycache-f19fd83d
  my/cool/cache -> cache-my2fcool2fcache
  my!@#$%^&()cache -> cache-my21402324255e262829cache-84ba3e99

Turns out the persistence is not the only place that doesn’t like special 
symbols in cache names – I also got an exception from MBean registration when 
creating a cache with ‘*’ or ‘?’. Filed 
https://issues.apache.org/jira/browse/IGNITE-7334 for that.

Please let me know if you have any comments.

Thanks,
Stan

From: Stanislav Lukyanov
Sent: 25 декабря 2017 г. 18:09
To: dev@ignite.apache.org 
Subject: Handling slashes in cache names

Hi all,

I’m looking into https://issues.apache.org/jira/browse/IGNITE-7264, and I need 
some guidance on what’s the best way to approach it.

The problem is that cache names are not restricted, but if persistence is 
enabled the cache needs to have a corresponding directory on the file system 
(“cache-…”) which can’t be created if the cache name contains certain 
characters (or a reserved system name).

A straightforward approach would be to check if a cache name is allowed on the 
local system (e.g. via `Paths.get(name)`) and fail to create cache if it isn’t, 
but I’m a bit concerned with the consistency of the behavior (the same cache 
name be allowed on one system and not on another).
I think a better way would be to replace special characters (say, all 
non-alphanumeric characters) with underscores in file names (not changing the 
cache configuration). Would this be OK? Are there any risks I’m not considering?

WDYT?

Thanks,
Stan

Re: ATOMIC caches consistency - IEP-12

[Yakov Zhdanov]

Dmitry,

This will not solve the problem:

1. new primary election may be affinity function dependent
2. If you want to introduce order in updating backup copies then you lose
parallelism. This will increase ops latency.

My suggestion should not affect current performance on stable and on
unstable topologies.

Btw, I have just got an idea - if we need to rebalance partition to more
than 1 node then we need to scan it only once. I believe now we do it each
time from very beginning.

Yakov Zhdanov,
www.gridgain.com

2017-12-27 6:14 GMT+03:00 Dmitriy Setrakyan :

> Yakov,
>
> As one of the prevention mechanisms for the scenario you describe, would it
> be possible to send the update message to backup nodes in the order in
> which they would become primary nodes? For example, if backup1 node is the
> next in chain to become the primary node, then the current primary node
> should send the update message to backup1 before it sends it to backup2.
>
> In this case, if the primary and client nodes fail, then backup1 node will
> become primary and will have the latest version of the data, no? Will this
> solve the situation you describe, at least partially?
>
> D.
>
> On Mon, Dec 25, 2017 at 7:23 AM, Yakov Zhdanov 
> wrote:
>
> > Hello!
> >
> > I tried to summarize some ideas on how to make ATOMIC caches stay
> > consistent in case of topology changes.
> >
> > Imagine partitioned ATOMIC cache with 2 backups configured (primary + 2
> > backup copies of the partition). One of possible scenarios to get to
> > primary and backups divergence is the following - update initiating node
> > sends update operation to primary node, primary node propagates update
> to 1
> > of 2 backups and then dies. If initiating node crashes as well (and
> > therefore cannot retry the operation) then in current implementation
> system
> > comes to a situation when 2 copies of the partition present in cluster
> may
> > be different. Note that both situations possible - new primary contains
> the
> > latest update and backup does not and vice versa. New backup will be
> > elected according to configured affinity and will rebalance the partition
> > from random owner, but copies may not be consistent due to described
> above.
> >
> > This problem does not affect TRANSACTIONAL caches as 2PC protocol deals
> > with scenarios of the kind very well.
> >
> > Here is the link to IEP -
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 12+Make+ATOMIC+Caches+Consistent+Again
> >
> > Sam, Alex G, Vladimir, please share your thoughts.
> >
> > --Yakov
> >
>

[jira] [Created] (IGNITE-7334) MBeans fail to be created for caches with special symbols in names

[Stanislav Lukyanov (JIRA)]

Stanislav Lukyanov created IGNITE-7334:
--

 Summary: MBeans fail to be created for caches with special symbols 
in names
 Key: IGNITE-7334
 URL: https://issues.apache.org/jira/browse/IGNITE-7334
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Stanislav Lukyanov
Assignee: Stanislav Lukyanov


If a cache contains a certain symbols, e.g. an asterisk, its creation will fail 
due to an exception during MBean creation. This is because of insufficient 
escaping in the shared MBean registration code in IgniteUtils.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Removing "fabric" from Ignite binary package name

[Petr Ivanov]

IGNITE-7251 is done, needs review and some additional tests. See PR #3315 [1].


[1] https://github.com/apache/ignite/pull/3315 




> On 20 Dec 2017, at 23:15, Denis Magda  wrote:
> 
> Petr, thanks, such a swift turnaround! 
> 
> Have you found the one who can asses and review the changes?
> 
> Maintainers label might be helpful. Just ping them directly:
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers
>  
> 
> 
> 
> —
> Denis
> 
>> On Dec 20, 2017, at 12:24 AM, Petr Ivanov  wrote:
>> 
>> Assigned myself — done the same work while preparing RPM package.
>> But for fixing DEVNOTES.txt waiting for review and merge of IGNITE-7107 [1].
>> 
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-7107
>> 
>> 
>> 
>>> On 19 Dec 2017, at 22:55, Denis Magda  wrote:
>>> 
>>> All the bids were accepted and the verdict is executed:
>>> https://issues.apache.org/jira/browse/IGNITE-7251 
>>> 
>>> 
>>> Who is ready to pick this up?
>>> 
>>> —
>>> Denis
>>> 
 On Dec 19, 2017, at 5:35 AM, Anton Vinogradov  
 wrote:
 
 +1б фо шур
 
 On Tue, Dec 19, 2017 at 9:59 AM, Vladimir Ozerov 
 wrote:
 
> +1б вуаштшеудн
> 
> On Tue, Dec 19, 2017 at 2:34 AM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> 
>> +1
>> 
>> On Mon, Dec 18, 2017 at 12:24 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org
>>> 
>> wrote:
>> 
>>> +1 (completely agree)
>>> 
>>> On Mon, Dec 18, 2017 at 10:21 AM, Denis Magda 
> wrote:
>>> 
 Igniters,
 
 Apache Ignite binary releases still include “fabric” word in their
>> names:
 https://ignite.apache.org/download.cgi#binaries <
 https://ignite.apache.org/download.cgi#binaries>
 
 For instance, this is a full name of the previous release -
 apache-ignite-fabric-2.3.0-bin.
 It’s a little oversight on our side because the project has not been
 positioned as a fabric for a while.
 
 Proposal! Remove “fabric” from the name and have the binary releases
>>> named
 as - apache-ignite-{version}-bin.
 
 If we’re in consensus then let’s make the change in 2.4.
 
 —
 Denis
 
>>> 
>> 
> 
>>> 
>> 
> 

[GitHub] ignite pull request #3316: IGNITE-7332: Add test suite for ML examples

[dmitrievanthony]

GitHub user dmitrievanthony opened a pull request:

https://github.com/apache/ignite/pull/3316

IGNITE-7332: Add test suite for ML examples



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGNITE-7332

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3316.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3316


commit f4c852ffb131221757e9f5e6774d6c807e2d9c39
Author: dmitrievanthony 
Date:   2017-12-28T16:17:38Z

IGNITE-7332 Add test suite for ML tests.




---

[GitHub] ignite pull request #3315: IGNITE-7251 Remove term "fabric" from Ignite deli...

[vveider]

GitHub user vveider opened a pull request:

https://github.com/apache/ignite/pull/3315

IGNITE-7251 Remove term "fabric" from Ignite deliverables



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7251

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3315.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3315


commit 02d44464e30877dcb61831ffb1ac93c2284f756a
Author: Ivanov Petr 
Date:   2017-12-28T16:07:09Z

IGNITE-7251 Remove term "fabric" from Ignite deliverables




---

[GitHub] ignite pull request #3308: IGNITE-5217: Add Gradient Descent and QR-based tr...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3308


---

[jira] [Created] (IGNITE-7333) SQL: Documentation for ALTER TABLE DROP COLUMN statement

[Sergey Kalashnikov (JIRA)]

Sergey Kalashnikov created IGNITE-7333:
--

 Summary: SQL: Documentation for ALTER TABLE DROP COLUMN statement
 Key: IGNITE-7333
 URL: https://issues.apache.org/jira/browse/IGNITE-7333
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.4
Reporter: Sergey Kalashnikov
Assignee: Sergey Kalashnikov
 Fix For: 2.4


Add a documentation for ALTER TABLE DROP COLUMN statement.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #3258: IGNITE-5949: Added limited support for DDL ALTER ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3258


---

[GitHub] ignite pull request #3314: IGNITE-7214 performance measurement for FCM and K...

[oignatenko]

GitHub user oignatenko opened a pull request:

https://github.com/apache/ignite/pull/3314

IGNITE-7214 performance measurement for FCM and KNN algorithms



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7214

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3314.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3314


commit b328d6b2856c7998e6a3de2f8344ddefaad81196
Author: Oleg Ignatenko 
Date:   2017-12-25T18:14:44Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- kNN benchmarks
-- verified with diffs overview, clean rebuild, trial execution of the 
benchmarks and studying results

commit 7c4b06dc2eff5c68c8bd0e05d82e98af8d9f02ab
Author: Oleg Ignatenko 
Date:   2017-12-26T07:10:00Z

Merge branch 'master-ml' into ignite-7214-preliminary

commit 9fa1c42c76e26037d5e4eab16bcbb195c5ece51d
Author: Oleg Ignatenko 
Date:   2017-12-26T09:49:42Z

Merge branch 'master-ml' into ignite-7214-preliminary

commit 7acc2db76434868bed5436aaed5243684d0f394c
Author: Oleg Ignatenko 
Date:   2017-12-26T10:23:35Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- kNN examples cleanup
-- verified with diffs overview, clean rebuild, trial execution of the 
benchmarks and studying results

commit ce5c4be654bb2e1f7ac7193ecaaf2538d1570ad0
Author: Oleg Ignatenko 
Date:   2017-12-26T10:51:30Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- corrected data for IgniteKNNRegressionBenchmark
-- verified with diffs overview

commit 5ffdf8d951a9af0fa4391e17a62cd2ca764bd8e9
Author: Oleg Ignatenko 
Date:   2017-12-26T15:21:35Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- preliminary cleanup for FCM benchmark
-- verified with diffs overview

commit 3901a2df34908c7f9df37932940920510989fb34
Author: Oleg Ignatenko 
Date:   2017-12-26T16:03:10Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- drafted distributed FCM benchmark
-- verified with diffs overview

commit 9148e50fbe61f5c963e2c433d3a45c9af936e9fb
Author: Oleg Ignatenko 
Date:   2017-12-26T16:05:21Z

Merge branch 'master-ml' into ignite-7214-preliminary

commit b0d9c6cb58716f28913f53d3e5780824758fe558
Author: Oleg Ignatenko 
Date:   2017-12-26T16:24:00Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- preliminary cleanup for FCM benchmark
-- verified with diffs overview

commit e626d194400bbadae31e2f6da0410a8bddabb5bf
Author: Oleg Ignatenko 
Date:   2017-12-26T16:34:00Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- preliminary cleanup for FCM benchmark
-- verified with diffs overview

commit 7cfaed9d4d5ea407c91a256100767b05bc942059
Author: Oleg Ignatenko 
Date:   2017-12-27T07:22:23Z

Merge branch 'master-ml' into ignite-7214-preliminary

commit 353e8ab70294c599f262278c54b04c55fd2c349c
Author: Oleg Ignatenko 
Date:   2017-12-27T11:55:31Z

Merge branch 'master-ml' into ignite-7214-preliminary

commit f16a37e5a088df23a6b9b716731b14018791c534
Author: Oleg Ignatenko 
Date:   2017-12-27T12:32:32Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- preliminary cleanup for FCM benchmark // added missing example for local 
FCM
-- verified with diffs overview

commit 50bb3d24c6001dee8351fdb8c8d68ebdd19d57bf
Author: Oleg Ignatenko 
Date:   2017-12-27T12:44:05Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- drafted local FCM benchmark
-- verified with diffs overview

commit 14cea0e89de17e625f526bf08338fea65d9d6a5c
Author: Oleg Ignatenko 
Date:   2017-12-27T13:05:32Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- FCM benchmarks
-- verified with diffs overview, clean rebuild, trial execution of the 
benchmarks and studying results

commit 4f71ff9d88d4f016fcbe52e49104d17687191f3f
Author: Oleg Ignatenko 
Date:   2017-12-28T14:34:34Z

Merge branch 'master-ml' into ignite-7214

# Conflicts:
#   
modules/ml/src/main/java/org/apache/ignite/ml/structures/LabeledDataset.java

commit efbfda49a040ec290239e00a29fbc6818361e4be
Author: Oleg Ignatenko 
Date:   2017-12-28T14:43:21Z

IGNITE-7214 performance measurement for FCM and KNN algorithms
- minor cleanup
-- verified with diffs overview




---

[jira] [Created] (IGNITE-7332) Create selftest suite for ml examples

[Yury Babak (JIRA)]

Yury Babak created IGNITE-7332:
--

 Summary: Create selftest suite for ml examples
 Key: IGNITE-7332
 URL: https://issues.apache.org/jira/browse/IGNITE-7332
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Yury Babak
 Fix For: 2.5


We want to add self test suite for our examples like we have for java8 examples.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7331) create MLP examples

[Oleg Ignatenko (JIRA)]

Oleg Ignatenko created IGNITE-7331:
--

 Summary: create MLP examples
 Key: IGNITE-7331
 URL: https://issues.apache.org/jira/browse/IGNITE-7331
 Project: Ignite
  Issue Type: Sub-task
  Components: examples, ml
Reporter: Oleg Ignatenko


create examples for local and distributed MLP in order to help project users 
see how to use these features



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #3313: Ignite-2.4-java8

[agura]

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/3313

Ignite-2.4-java8



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4-java8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3313.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3313


commit 7b66609de9da4705926fb0a7845d69240eb0dee2
Author: Ivanov Petr 
Date:   2017-12-15T14:35:35Z

IGNITE-7203 Java 8 by default
 * got rid of java-1.8 related profiles
 * remove ml profile and included ml build into main reactor
 * added master properties to maven-compiler-plugin for whole project to be 
compiled only by java-1.8
 * refactored met properties + gathered them in parent module

commit b20f6ab8cb22528d7717229c1a7acc586cda44bc
Author: Ivanov Petr 
Date:   2017-12-18T07:25:19Z

IGNITE-7203 Java 8 by default
IGNITE-7216 Refactor examples project
 * refactored examples module and ml part of other modules: merged java8 
and java packages

commit 3177f28d5651f4ad910e6d54d5e8a225b9664c6b
Author: Ivanov Petr 
Date:   2017-12-18T15:19:02Z

IGNITE-7203 Java 8 by default
IGNITE-7216 Refactor examples project
 * fixed notes from review

commit ab8d197d8ed3c430e7720750db6c00aaf845b249
Author: Pavel Tupitsyn 
Date:   2017-12-26T09:11:41Z

.NET: Fix ExecutableTest

commit f4b0bdc72ba3f62f9125843680ff3ae78098deed
Author: Pavel Tupitsyn 
Date:   2017-12-26T09:26:23Z

.NET: Fix TestStopFromJava

commit b361ac4f6edfaa7d0645a9352145b84bd02745b1
Author: devozerov 
Date:   2017-12-26T11:31:07Z

Attempty to fix AOP failure.

commit 1390677806c9a28aba6720566260b31cdb58b086
Author: Ivanov Petr 
Date:   2017-12-26T12:40:15Z

IGNITE-7203 Java 8 by default
 * replaced aspectj dependency with newest one for attempt to fix AOP tests

commit aa2546aa55a9c22253206add2839917faef47027
Author: Pavel Tupitsyn 
Date:   2017-12-26T14:20:44Z

.NET: Another attempt to fix WindowsServiceTest

commit 017f63b4cb36a62234b8c406b2ebe085549d5628
Author: Pavel Tupitsyn 
Date:   2017-12-27T07:22:22Z

.NET: Another attempt to fix WindowsServiceTest

commit 83b9d21edc71c8f263a093c19be60042dfca36c1
Author: Pavel Tupitsyn 
Date:   2017-12-27T08:24:29Z

.NET: Another attempt to fix WindowsServiceTest

commit a93f2625c08b4c116c5a5df21c0e277f3821ad5f
Author: Pavel Tupitsyn 
Date:   2017-12-27T09:28:30Z

.NET: Another attempt to fix WindowsServiceTest, adding logger, no idea 
what the problem is

commit 2cf50943fb7451f7d5916fc824716b07c911093b
Author: Pavel Tupitsyn 
Date:   2017-12-27T11:35:10Z

.NET: Fix WindowsServiceTest - pass JVM path to the service

commit da9e76a5f093f745c51a0c55b6f8fa13dfc63ae9
Author: Ivanov Petr 
Date:   2017-12-27T15:20:40Z

IGNITE-7203 Java 8 by default
 * further improved AOP test behaviour by moving javaagent call into pom.xml

commit 602830163960b64e370f55b7a1269bb2fab513ba
Author: Ivanov Petr 
Date:   2017-12-28T13:03:40Z

Merge remote-tracking branch 'ignite/master' into ignite-7203

commit 129e956e9c17e86bcee939816da8d17e75ca954c
Author: Andrey Gura 
Date:   2017-12-27T16:14:22Z

ignite-6736 Unsafe.monitorEnter/Exit usages are removed

commit 85c59ab8b25dbd6f1b4b29a0279f6c0a0f81ce27
Author: Andrey Gura 
Date:   2017-12-28T13:34:31Z

Merge branch 'ignite-7203' into ignite-2.4-java8




---

[GitHub] ignite pull request #3312: Ignite-2.4-java8

[agura]

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/3312

Ignite-2.4-java8



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agura/incubator-ignite ignite-2.4-java8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3312.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3312


commit 7b66609de9da4705926fb0a7845d69240eb0dee2
Author: Ivanov Petr 
Date:   2017-12-15T14:35:35Z

IGNITE-7203 Java 8 by default
 * got rid of java-1.8 related profiles
 * remove ml profile and included ml build into main reactor
 * added master properties to maven-compiler-plugin for whole project to be 
compiled only by java-1.8
 * refactored met properties + gathered them in parent module

commit b20f6ab8cb22528d7717229c1a7acc586cda44bc
Author: Ivanov Petr 
Date:   2017-12-18T07:25:19Z

IGNITE-7203 Java 8 by default
IGNITE-7216 Refactor examples project
 * refactored examples module and ml part of other modules: merged java8 
and java packages

commit 3177f28d5651f4ad910e6d54d5e8a225b9664c6b
Author: Ivanov Petr 
Date:   2017-12-18T15:19:02Z

IGNITE-7203 Java 8 by default
IGNITE-7216 Refactor examples project
 * fixed notes from review

commit ab8d197d8ed3c430e7720750db6c00aaf845b249
Author: Pavel Tupitsyn 
Date:   2017-12-26T09:11:41Z

.NET: Fix ExecutableTest

commit f4b0bdc72ba3f62f9125843680ff3ae78098deed
Author: Pavel Tupitsyn 
Date:   2017-12-26T09:26:23Z

.NET: Fix TestStopFromJava

commit b361ac4f6edfaa7d0645a9352145b84bd02745b1
Author: devozerov 
Date:   2017-12-26T11:31:07Z

Attempty to fix AOP failure.

commit 1390677806c9a28aba6720566260b31cdb58b086
Author: Ivanov Petr 
Date:   2017-12-26T12:40:15Z

IGNITE-7203 Java 8 by default
 * replaced aspectj dependency with newest one for attempt to fix AOP tests

commit aa2546aa55a9c22253206add2839917faef47027
Author: Pavel Tupitsyn 
Date:   2017-12-26T14:20:44Z

.NET: Another attempt to fix WindowsServiceTest

commit 017f63b4cb36a62234b8c406b2ebe085549d5628
Author: Pavel Tupitsyn 
Date:   2017-12-27T07:22:22Z

.NET: Another attempt to fix WindowsServiceTest

commit 83b9d21edc71c8f263a093c19be60042dfca36c1
Author: Pavel Tupitsyn 
Date:   2017-12-27T08:24:29Z

.NET: Another attempt to fix WindowsServiceTest

commit a93f2625c08b4c116c5a5df21c0e277f3821ad5f
Author: Pavel Tupitsyn 
Date:   2017-12-27T09:28:30Z

.NET: Another attempt to fix WindowsServiceTest, adding logger, no idea 
what the problem is

commit 2cf50943fb7451f7d5916fc824716b07c911093b
Author: Pavel Tupitsyn 
Date:   2017-12-27T11:35:10Z

.NET: Fix WindowsServiceTest - pass JVM path to the service

commit da9e76a5f093f745c51a0c55b6f8fa13dfc63ae9
Author: Ivanov Petr 
Date:   2017-12-27T15:20:40Z

IGNITE-7203 Java 8 by default
 * further improved AOP test behaviour by moving javaagent call into pom.xml

commit 602830163960b64e370f55b7a1269bb2fab513ba
Author: Ivanov Petr 
Date:   2017-12-28T13:03:40Z

Merge remote-tracking branch 'ignite/master' into ignite-7203

commit 129e956e9c17e86bcee939816da8d17e75ca954c
Author: Andrey Gura 
Date:   2017-12-27T16:14:22Z

ignite-6736 Unsafe.monitorEnter/Exit usages are removed

commit 85c59ab8b25dbd6f1b4b29a0279f6c0a0f81ce27
Author: Andrey Gura 
Date:   2017-12-28T13:34:31Z

Merge branch 'ignite-7203' into ignite-2.4-java8




---

[GitHub] ignite pull request #3306: ignite-6736 Unsafe.monitorEnter/Exit usages are r...

[agura]

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3306


---

[GitHub] ignite pull request #3311: Ignite-2.4-java8

[agura]

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3311


---

[GitHub] ignite pull request #3311: Ignite-2.4-java8

[agura]

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/3311

Ignite-2.4-java8



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4-java8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3311.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3311


commit 7b66609de9da4705926fb0a7845d69240eb0dee2
Author: Ivanov Petr 
Date:   2017-12-15T14:35:35Z

IGNITE-7203 Java 8 by default
 * got rid of java-1.8 related profiles
 * remove ml profile and included ml build into main reactor
 * added master properties to maven-compiler-plugin for whole project to be 
compiled only by java-1.8
 * refactored met properties + gathered them in parent module

commit b20f6ab8cb22528d7717229c1a7acc586cda44bc
Author: Ivanov Petr 
Date:   2017-12-18T07:25:19Z

IGNITE-7203 Java 8 by default
IGNITE-7216 Refactor examples project
 * refactored examples module and ml part of other modules: merged java8 
and java packages

commit 3177f28d5651f4ad910e6d54d5e8a225b9664c6b
Author: Ivanov Petr 
Date:   2017-12-18T15:19:02Z

IGNITE-7203 Java 8 by default
IGNITE-7216 Refactor examples project
 * fixed notes from review

commit ab8d197d8ed3c430e7720750db6c00aaf845b249
Author: Pavel Tupitsyn 
Date:   2017-12-26T09:11:41Z

.NET: Fix ExecutableTest

commit f4b0bdc72ba3f62f9125843680ff3ae78098deed
Author: Pavel Tupitsyn 
Date:   2017-12-26T09:26:23Z

.NET: Fix TestStopFromJava

commit b361ac4f6edfaa7d0645a9352145b84bd02745b1
Author: devozerov 
Date:   2017-12-26T11:31:07Z

Attempty to fix AOP failure.

commit 1390677806c9a28aba6720566260b31cdb58b086
Author: Ivanov Petr 
Date:   2017-12-26T12:40:15Z

IGNITE-7203 Java 8 by default
 * replaced aspectj dependency with newest one for attempt to fix AOP tests

commit aa2546aa55a9c22253206add2839917faef47027
Author: Pavel Tupitsyn 
Date:   2017-12-26T14:20:44Z

.NET: Another attempt to fix WindowsServiceTest

commit 017f63b4cb36a62234b8c406b2ebe085549d5628
Author: Pavel Tupitsyn 
Date:   2017-12-27T07:22:22Z

.NET: Another attempt to fix WindowsServiceTest

commit 83b9d21edc71c8f263a093c19be60042dfca36c1
Author: Pavel Tupitsyn 
Date:   2017-12-27T08:24:29Z

.NET: Another attempt to fix WindowsServiceTest

commit a93f2625c08b4c116c5a5df21c0e277f3821ad5f
Author: Pavel Tupitsyn 
Date:   2017-12-27T09:28:30Z

.NET: Another attempt to fix WindowsServiceTest, adding logger, no idea 
what the problem is

commit 2cf50943fb7451f7d5916fc824716b07c911093b
Author: Pavel Tupitsyn 
Date:   2017-12-27T11:35:10Z

.NET: Fix WindowsServiceTest - pass JVM path to the service

commit da9e76a5f093f745c51a0c55b6f8fa13dfc63ae9
Author: Ivanov Petr 
Date:   2017-12-27T15:20:40Z

IGNITE-7203 Java 8 by default
 * further improved AOP test behaviour by moving javaagent call into pom.xml

commit 602830163960b64e370f55b7a1269bb2fab513ba
Author: Ivanov Petr 
Date:   2017-12-28T13:03:40Z

Merge remote-tracking branch 'ignite/master' into ignite-7203

commit 129e956e9c17e86bcee939816da8d17e75ca954c
Author: Andrey Gura 
Date:   2017-12-27T16:14:22Z

ignite-6736 Unsafe.monitorEnter/Exit usages are removed

commit 85c59ab8b25dbd6f1b4b29a0279f6c0a0f81ce27
Author: Andrey Gura 
Date:   2017-12-28T13:34:31Z

Merge branch 'ignite-7203' into ignite-2.4-java8




---

[GitHub] ignite pull request #3303: IGNITE-7205: Dataset API

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3303


---

[GitHub] ignite pull request #3240: IGNITE-7203 Java 8 by default

[vveider]

GitHub user vveider reopened a pull request:

https://github.com/apache/ignite/pull/3240

IGNITE-7203 Java 8 by default



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7203

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3240.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3240


commit 7b66609de9da4705926fb0a7845d69240eb0dee2
Author: Ivanov Petr 
Date:   2017-12-15T14:35:35Z

IGNITE-7203 Java 8 by default
 * got rid of java-1.8 related profiles
 * remove ml profile and included ml build into main reactor
 * added master properties to maven-compiler-plugin for whole project to be 
compiled only by java-1.8
 * refactored met properties + gathered them in parent module

commit b20f6ab8cb22528d7717229c1a7acc586cda44bc
Author: Ivanov Petr 
Date:   2017-12-18T07:25:19Z

IGNITE-7203 Java 8 by default
IGNITE-7216 Refactor examples project
 * refactored examples module and ml part of other modules: merged java8 
and java packages

commit 3177f28d5651f4ad910e6d54d5e8a225b9664c6b
Author: Ivanov Petr 
Date:   2017-12-18T15:19:02Z

IGNITE-7203 Java 8 by default
IGNITE-7216 Refactor examples project
 * fixed notes from review

commit ab8d197d8ed3c430e7720750db6c00aaf845b249
Author: Pavel Tupitsyn 
Date:   2017-12-26T09:11:41Z

.NET: Fix ExecutableTest

commit f4b0bdc72ba3f62f9125843680ff3ae78098deed
Author: Pavel Tupitsyn 
Date:   2017-12-26T09:26:23Z

.NET: Fix TestStopFromJava

commit b361ac4f6edfaa7d0645a9352145b84bd02745b1
Author: devozerov 
Date:   2017-12-26T11:31:07Z

Attempty to fix AOP failure.

commit 1390677806c9a28aba6720566260b31cdb58b086
Author: Ivanov Petr 
Date:   2017-12-26T12:40:15Z

IGNITE-7203 Java 8 by default
 * replaced aspectj dependency with newest one for attempt to fix AOP tests

commit aa2546aa55a9c22253206add2839917faef47027
Author: Pavel Tupitsyn 
Date:   2017-12-26T14:20:44Z

.NET: Another attempt to fix WindowsServiceTest

commit 017f63b4cb36a62234b8c406b2ebe085549d5628
Author: Pavel Tupitsyn 
Date:   2017-12-27T07:22:22Z

.NET: Another attempt to fix WindowsServiceTest

commit 83b9d21edc71c8f263a093c19be60042dfca36c1
Author: Pavel Tupitsyn 
Date:   2017-12-27T08:24:29Z

.NET: Another attempt to fix WindowsServiceTest

commit a93f2625c08b4c116c5a5df21c0e277f3821ad5f
Author: Pavel Tupitsyn 
Date:   2017-12-27T09:28:30Z

.NET: Another attempt to fix WindowsServiceTest, adding logger, no idea 
what the problem is

commit 2cf50943fb7451f7d5916fc824716b07c911093b
Author: Pavel Tupitsyn 
Date:   2017-12-27T11:35:10Z

.NET: Fix WindowsServiceTest - pass JVM path to the service

commit da9e76a5f093f745c51a0c55b6f8fa13dfc63ae9
Author: Ivanov Petr 
Date:   2017-12-27T15:20:40Z

IGNITE-7203 Java 8 by default
 * further improved AOP test behaviour by moving javaagent call into pom.xml

commit 602830163960b64e370f55b7a1269bb2fab513ba
Author: Ivanov Petr 
Date:   2017-12-28T13:03:40Z

Merge remote-tracking branch 'ignite/master' into ignite-7203




---

[GitHub] ignite pull request #3171: IGNITE-7107 Apache Ignite RPM packages

[vveider]

Github user vveider closed the pull request at:

https://github.com/apache/ignite/pull/3171


---

[GitHub] ignite pull request #3240: IGNITE-7203 Java 8 by default

[vveider]

Github user vveider closed the pull request at:

https://github.com/apache/ignite/pull/3240


---

Re: IGNITE-7107 Apache Ignite RPM packages

[Sergey Kozlov]

Guys

Thanks for your efforts to make it available in 2.4!

On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura  wrote:

> Guys,
>
> I've merged change to master branch. Thanks a lot!
>
> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov  wrote:
> > Fixed remarks and nitpicks.
> >
> > Also, I purpose to delay service autostart implementation until we’ll
> get some first-hand usability feedback.
> > Added notes to DEVNOTES.txt about how to start service.
> >
> >
> >> On 28 Dec 2017, at 14:32, Ilya Kasnacheev 
> wrote:
> >>
> >> Hello once more!
> >>
> >> Two more nitpicks:
> >>
> >> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
> >> work since a) visor wants to write to /usr/share, and b) we moved it to
> >> examples for that.
> >> Please remove that alternative for now, create a ticket.
> >>
> >> rpm -ql apache-ignite | grep \\.java
> >> some yardstick and ml sources are there. I don't know who is there to
> blame
> >> - Ignite release process likely, not RPM building - but they shouldn't
> be
> >> here I guess?
> >>
> >> Regards!
> >>
> >> --
> >> Ilya Kasnacheev
> >>
> >> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev :
> >>
> >>> Hello again!
> >>>
> >>> I have reviewed the modified patch. All the things in my critical list
> >>> were fixed. I have just two minor remarks:
> >>>
> >>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
> >>> works just as well from there as it was from our distribution. visor
> >>> doesn't, hence it stays in examples.
> >>> - I'm a bit confused that we no longer auto-start service after
> package is
> >>> installed. You have to do
> >>> sudo systemctl start apache-ign...@default-config.xml
> >>> manually. I'm used to packages that start the thing they install
> >>> immediately. Of course we can just document that clearly on downloads
> pages
> >>> so people won't get lost. What do you all think? Should Ignite
> auto-start
> >>> with default config? Is it possible to do so, but make possible to
> turn it
> >>> off.
> >>>
> >>> So from my side I recommend this pull request for inclusion after 1) is
> >>> done and 2) is discussed.
> >>>
> >>> Also, would be nice if you create additional JIRA tickets for issues
> in my
> >>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
> >>> working) to be implemented in future releases.
> >>>
> >>> Regards,
> >>>
> >>> --
> >>> Ilya Kasnacheev
> >>>
> >>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov :
> >>>
>  Removed replacement for default-config.xml.
>  Also reimplemented service to be able to run multiple instances of
> Ignite.
> 
>  See updated PR [1].
> 
> 
>  [1] https://github.com/apache/ignite/pull/3171 <
>  https://github.com/apache/ignite/pull/3171>
> 
> 
> 
> > On 25 Dec 2017, at 18:57, Pavel Tupitsyn 
> wrote:
> >
> > PDS and discovery settings are unrelated to the packaging task.
> > Andrey is right, just use existing default config.
> >
> > On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov 
>  wrote:
> >
> >> Can we change default configuration file then?
> >> So that both binary and package deliveries contained PDS turned on
> by
> >> default?
> >>
> >> And what about Multicast Discovery?
> >>
> >>
> >>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan  >
> >> wrote:
> >>>
> >>> I agree with Andrey. The product should be identical regardless of
> how
> >> you
> >>> download and install it.
> >>>
> >>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura 
>  wrote:
> >>>
>  Hi,
> 
>  I think we should provide the same default configuration for all
>  binary builds. From my point of view RPM/DEB/etc packages should
> use
>  default configuration file like to standard binary release.
> 
>  On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>   wrote:
> > Hello Igniters!
> >
> > What's your take on enabling PDS in 2.4 in default config? This
> way
>  we
> > could enable it in RPM build with easy heart.
> >
> > The rest of your answers seem spot on. I'll happily review your
>  amended
> > patch when it is ready (later this week?)
> >
> > --
> > Ilya Kasnacheev
> >
> > 2017-12-22 18:27 GMT+03:00 vveider :
> >
> >> Ilya Kasnacheev wrote
> >>> I have noticed that both spec and DEVNOTES are made to use tabs
>  alongside
> >>> with spaces. I thought we are avoiding tabs? Please confirm
> that
>  usage of
> >>> tabs is desired in this case.
> >>
> >> My fault - got used to editing such files in default vim. Fixed
> and
>  updated
> >> vim settings to indent with spaces.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >>> Maybe it's my CentOS but initially build will fail with the
>  followi

[GitHub] ignite pull request #3305: ignite-6745 URLClassLoader.getURLs() usages are r...

[agura]

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3305


---

Re: IGNITE-7107 Apache Ignite RPM packages

[Andrey Gura]

Guys,

I've merged change to master branch. Thanks a lot!

On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov  wrote:
> Fixed remarks and nitpicks.
>
> Also, I purpose to delay service autostart implementation until we’ll get 
> some first-hand usability feedback.
> Added notes to DEVNOTES.txt about how to start service.
>
>
>> On 28 Dec 2017, at 14:32, Ilya Kasnacheev  wrote:
>>
>> Hello once more!
>>
>> Two more nitpicks:
>>
>> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
>> work since a) visor wants to write to /usr/share, and b) we moved it to
>> examples for that.
>> Please remove that alternative for now, create a ticket.
>>
>> rpm -ql apache-ignite | grep \\.java
>> some yardstick and ml sources are there. I don't know who is there to blame
>> - Ignite release process likely, not RPM building - but they shouldn't be
>> here I guess?
>>
>> Regards!
>>
>> --
>> Ilya Kasnacheev
>>
>> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev :
>>
>>> Hello again!
>>>
>>> I have reviewed the modified patch. All the things in my critical list
>>> were fixed. I have just two minor remarks:
>>>
>>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>>> works just as well from there as it was from our distribution. visor
>>> doesn't, hence it stays in examples.
>>> - I'm a bit confused that we no longer auto-start service after package is
>>> installed. You have to do
>>> sudo systemctl start apache-ign...@default-config.xml
>>> manually. I'm used to packages that start the thing they install
>>> immediately. Of course we can just document that clearly on downloads pages
>>> so people won't get lost. What do you all think? Should Ignite auto-start
>>> with default config? Is it possible to do so, but make possible to turn it
>>> off.
>>>
>>> So from my side I recommend this pull request for inclusion after 1) is
>>> done and 2) is discussed.
>>>
>>> Also, would be nice if you create additional JIRA tickets for issues in my
>>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>>> working) to be implemented in future releases.
>>>
>>> Regards,
>>>
>>> --
>>> Ilya Kasnacheev
>>>
>>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov :
>>>
 Removed replacement for default-config.xml.
 Also reimplemented service to be able to run multiple instances of Ignite.

 See updated PR [1].


 [1] https://github.com/apache/ignite/pull/3171 <
 https://github.com/apache/ignite/pull/3171>



> On 25 Dec 2017, at 18:57, Pavel Tupitsyn  wrote:
>
> PDS and discovery settings are unrelated to the packaging task.
> Andrey is right, just use existing default config.
>
> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov 
 wrote:
>
>> Can we change default configuration file then?
>> So that both binary and package deliveries contained PDS turned on by
>> default?
>>
>> And what about Multicast Discovery?
>>
>>
>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan 
>> wrote:
>>>
>>> I agree with Andrey. The product should be identical regardless of how
>> you
>>> download and install it.
>>>
>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura 
 wrote:
>>>
 Hi,

 I think we should provide the same default configuration for all
 binary builds. From my point of view RPM/DEB/etc packages should use
 default configuration file like to standard binary release.

 On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
  wrote:
> Hello Igniters!
>
> What's your take on enabling PDS in 2.4 in default config? This way
 we
> could enable it in RPM build with easy heart.
>
> The rest of your answers seem spot on. I'll happily review your
 amended
> patch when it is ready (later this week?)
>
> --
> Ilya Kasnacheev
>
> 2017-12-22 18:27 GMT+03:00 vveider :
>
>> Ilya Kasnacheev wrote
>>> I have noticed that both spec and DEVNOTES are made to use tabs
 alongside
>>> with spaces. I thought we are avoiding tabs? Please confirm that
 usage of
>>> tabs is desired in this case.
>>
>> My fault - got used to editing such files in default vim. Fixed and
 updated
>> vim settings to indent with spaces.
>>
>>
>> Ilya Kasnacheev wrote
>>> Maybe it's my CentOS but initially build will fail with the
 following
>>> message, which I had to fix in spec
>>> + cd 'apache-ignite-*'
>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
 file
>> or
>>> directory
>>
>> Strange error - shell expansion is not working. Anyway, replaced
 with
 more
>> universal variant.
>>
>>
>> Ilya Kasnacheev wrote
>>> We absolutel

Re: IGNITE-7107 Apache Ignite RPM packages

[Petr Ivanov]

Fixed remarks and nitpicks.

Also, I purpose to delay service autostart implementation until we’ll get some 
first-hand usability feedback.
Added notes to DEVNOTES.txt about how to start service.


> On 28 Dec 2017, at 14:32, Ilya Kasnacheev  wrote:
> 
> Hello once more!
> 
> Two more nitpicks:
> 
> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
> work since a) visor wants to write to /usr/share, and b) we moved it to
> examples for that.
> Please remove that alternative for now, create a ticket.
> 
> rpm -ql apache-ignite | grep \\.java
> some yardstick and ml sources are there. I don't know who is there to blame
> - Ignite release process likely, not RPM building - but they shouldn't be
> here I guess?
> 
> Regards!
> 
> -- 
> Ilya Kasnacheev
> 
> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev :
> 
>> Hello again!
>> 
>> I have reviewed the modified patch. All the things in my critical list
>> were fixed. I have just two minor remarks:
>> 
>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>> works just as well from there as it was from our distribution. visor
>> doesn't, hence it stays in examples.
>> - I'm a bit confused that we no longer auto-start service after package is
>> installed. You have to do
>> sudo systemctl start apache-ign...@default-config.xml
>> manually. I'm used to packages that start the thing they install
>> immediately. Of course we can just document that clearly on downloads pages
>> so people won't get lost. What do you all think? Should Ignite auto-start
>> with default config? Is it possible to do so, but make possible to turn it
>> off.
>> 
>> So from my side I recommend this pull request for inclusion after 1) is
>> done and 2) is discussed.
>> 
>> Also, would be nice if you create additional JIRA tickets for issues in my
>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>> working) to be implemented in future releases.
>> 
>> Regards,
>> 
>> --
>> Ilya Kasnacheev
>> 
>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov :
>> 
>>> Removed replacement for default-config.xml.
>>> Also reimplemented service to be able to run multiple instances of Ignite.
>>> 
>>> See updated PR [1].
>>> 
>>> 
>>> [1] https://github.com/apache/ignite/pull/3171 <
>>> https://github.com/apache/ignite/pull/3171>
>>> 
>>> 
>>> 
 On 25 Dec 2017, at 18:57, Pavel Tupitsyn  wrote:
 
 PDS and discovery settings are unrelated to the packaging task.
 Andrey is right, just use existing default config.
 
 On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov 
>>> wrote:
 
> Can we change default configuration file then?
> So that both binary and package deliveries contained PDS turned on by
> default?
> 
> And what about Multicast Discovery?
> 
> 
>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan 
> wrote:
>> 
>> I agree with Andrey. The product should be identical regardless of how
> you
>> download and install it.
>> 
>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura 
>>> wrote:
>> 
>>> Hi,
>>> 
>>> I think we should provide the same default configuration for all
>>> binary builds. From my point of view RPM/DEB/etc packages should use
>>> default configuration file like to standard binary release.
>>> 
>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>  wrote:
 Hello Igniters!
 
 What's your take on enabling PDS in 2.4 in default config? This way
>>> we
 could enable it in RPM build with easy heart.
 
 The rest of your answers seem spot on. I'll happily review your
>>> amended
 patch when it is ready (later this week?)
 
 --
 Ilya Kasnacheev
 
 2017-12-22 18:27 GMT+03:00 vveider :
 
> Ilya Kasnacheev wrote
>> I have noticed that both spec and DEVNOTES are made to use tabs
>>> alongside
>> with spaces. I thought we are avoiding tabs? Please confirm that
>>> usage of
>> tabs is desired in this case.
> 
> My fault - got used to editing such files in default vim. Fixed and
>>> updated
> vim settings to indent with spaces.
> 
> 
> Ilya Kasnacheev wrote
>> Maybe it's my CentOS but initially build will fail with the
>>> following
>> message, which I had to fix in spec
>> + cd 'apache-ignite-*'
>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>> file
> or
>> directory
> 
> Strange error - shell expansion is not working. Anyway, replaced
>>> with
>>> more
> universal variant.
> 
> 
> Ilya Kasnacheev wrote
>> We absolutely should not inline configuration files (such as
>> default-config.xml) into build scripts. We should just copy over
>> default-config.xml from config/ to /etc, with dependencies if
>>> needed.
> 

[jira] [Created] (IGNITE-7329) .NET: Thin client: SSL

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-7329:
--

 Summary: .NET: Thin client: SSL
 Key: IGNITE-7329
 URL: https://issues.apache.org/jira/browse/IGNITE-7329
 Project: Ignite
  Issue Type: Improvement
  Components: platforms, thin client
Affects Versions: 2.4
Reporter: Pavel Tupitsyn


Allow secure .NET thin client connection.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7330) When client connects during cluster activation process it hangs on obtaining cache proxy

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-7330:
---

 Summary: When client connects during cluster activation process it 
hangs on obtaining cache proxy
 Key: IGNITE-7330
 URL: https://issues.apache.org/jira/browse/IGNITE-7330
 Project: Ignite
  Issue Type: Bug
Reporter: Sergey Chugunov
Priority: Critical
 Fix For: 2.4


The test below reproduces the issue:

{noformat}
public void testClientJoinWhenActivationInProgress() throws Exception {
Ignite srv = startGrids(5);

srv.active(true);

srv.createCaches(Arrays.asList(cacheConfigurations1()));

Map cacheData = new LinkedHashMap<>();

for (int i = 1; i <= 100; i++) {
for (CacheConfiguration ccfg : cacheConfigurations1()) {
srv.cache(ccfg.getName()).put(-i, i);

cacheData.put(-i, i);
}
}

stopAllGrids();

srv = startGrids(5);

final CountDownLatch clientStartLatch = new CountDownLatch(1);

IgniteInternalFuture clStartFut = GridTestUtils.runAsync(new Runnable() 
{
@Override public void run() {
try {
clientStartLatch.await();

Thread.sleep(10);

client = true;

Ignite cl = startGrid("client0");

IgniteCache atomicCache = 
cl.cache(CACHE_NAME_PREFIX + '0');
IgniteCache txCache = 
cl.cache(CACHE_NAME_PREFIX + '1');

assertEquals(100, atomicCache.size());
assertEquals(100, txCache.size());
}
catch (Exception e) {
log.error("Error occurred", e);
}
}
}, "client-starter-thread");

clientStartLatch.countDown();
srv.active(true);

clStartFut.get();
}
{noformat}

Expected behavior: test finishes successfully.

Actual behavior: test hangs on waiting for client start future to complete 
while "client-started-thread" will be hanging on obtaining a reference to the 
first cache.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7328) Improve Labeled Dataset loading from txt file

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7328:


 Summary: Improve Labeled Dataset loading from txt file
 Key: IGNITE-7328
 URL: https://issues.apache.org/jira/browse/IGNITE-7328
 Project: Ignite
  Issue Type: New Feature
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev


1. Wouldn't it be better to parse rows in-place (not to save them as strings at 
first)? In current implementation we will be needed to keep a dataset in memory 
twice and it might be a problem for big datasets.

2. What about the case when a dataset contains not only a numerical data? Do we 
consider this case or for such purposes some other "DatasetLoader" will be used?

3. Just an idea, in case we don't want to fall on bad data (99% of cases) would 
be great to understand the quality of loaded dataset such as number of missed 
rows/values.

4. Does a situation when a row doesn't contain required number of columns 
should be considered as "bad data" and don't break parsing with 
IndexOutOfBoundException?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7327) Add CSV loading to Labeled Dataset with Loader

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7327:


 Summary: Add CSV loading to Labeled Dataset with Loader 
 Key: IGNITE-7327
 URL: https://issues.apache.org/jira/browse/IGNITE-7327
 Project: Ignite
  Issue Type: New Feature
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev


Comment from [~dmitrievanthony]
Lots of datasets (from Kaggle for example) are supplied in CSV format with 
header line. In connection with it does it make sense to:

Use some CSV parsing (it's a bit more complicated than just splitting by comma)?
Add ability to use first header line as a source for so called feature names?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7326) Fix ignitevisorcmd | sqlline scripts to be able to run from /usr/bin installed as symbolic links

[Peter Ivanov (JIRA)]

Peter Ivanov created IGNITE-7326:


 Summary: Fix ignitevisorcmd | sqlline scripts to be able to run 
from /usr/bin installed as symbolic links
 Key: IGNITE-7326
 URL: https://issues.apache.org/jira/browse/IGNITE-7326
 Project: Ignite
  Issue Type: Bug
Reporter: Peter Ivanov
Assignee: Peter Ivanov
 Fix For: 2.5


Currenlty, {{ignitevisorcmd.sh}} and {{sqlline.sh}} being installed into 
{{/usr/bin}} will fail to run because of:
* their unawarelessness of theirs real location;
* necessity to write to {{$\{IGNITE_HOME}/work}} which can have different 
permissions and owner (in packages, for example).

It is required to rewrite these scripts to be able to run from anywhere by 
theirs symbolic linka and with some temporary dir ({{/tmp}} for example) as 
workdir.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7325) Remove ExternalizeTest from all test and change on ExternalizableTest

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7325:


 Summary: Remove ExternalizeTest from all test and change on 
ExternalizableTest
 Key: IGNITE-7325
 URL: https://issues.apache.org/jira/browse/IGNITE-7325
 Project: Ignite
  Issue Type: New Feature
  Components: ml
Reporter: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: How to handle CacheLocalStore on clients node?

[ALEKSEY KUZNETSOV]

CacheLocalStore annotation means CacheStore is not "global" database, but
each node has its independent db. If CacheStore isn't annotated with
CacheLocalStore, then you can observe situation when data is persisted by
node, which is neither primary, nor backup for the key.(i.e. we have
transactional cache with near disabled).

ср, 27 дек. 2017 г. в 13:02, Vyacheslav Daradur :

> Hi, Igniters!
>
> I’m working on an implementation of 3rd party persistence solution.
>
> The solution is annotated by CacheLocalStore annotation.
>
> As far as I know, CacheLocalStore annotation means that server node
> persists its own primary and backup partitions (please correct me if
> I’m wrong).
>
> How about client nodes? It’s possible to define CacheStoreFactory in
> CacheConfiguration which creates CacheStoreStore even on clients node.
>
> But as far as I can see ‘read-through’ and ‘write-through’ methods
> aren’t called on clients nodes in case of using CacheLocalStore. It
> may be a problem because of CacheLocalStore which may require the
> creation of some temporary resources which are not needed on clients
> nodes.
>
> How must this situation be handled?
> I see some ways:
> 1) Ignite should validate this situation at cache creation on client node
> 2) CacheStoreFactory should throw some exception in case of creation
> CacheLocalStore on client nodes
> 3) CacheStoreFactory should create some stub of CacheLocalStore to do
> nothing
>
> Any thoughts?
>
>
> --
> Best Regards, Vyacheslav D.
>
-- 

*Best Regards,*

*Kuznetsov Aleksey*

Re: IGNITE-7107 Apache Ignite RPM packages

[Ilya Kasnacheev]

Hello once more!

Two more nitpicks:

right now we install /usr/bin/ignitevisorcmd alternative, but it does't
work since a) visor wants to write to /usr/share, and b) we moved it to
examples for that.
Please remove that alternative for now, create a ticket.

rpm -ql apache-ignite | grep \\.java
some yardstick and ml sources are there. I don't know who is there to blame
- Ignite release process likely, not RPM building - but they shouldn't be
here I guess?

Regards!

-- 
Ilya Kasnacheev

2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev :

> Hello again!
>
> I have reviewed the modified patch. All the things in my critical list
> were fixed. I have just two minor remarks:
>
> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
> works just as well from there as it was from our distribution. visor
> doesn't, hence it stays in examples.
> - I'm a bit confused that we no longer auto-start service after package is
> installed. You have to do
> sudo systemctl start apache-ign...@default-config.xml
> manually. I'm used to packages that start the thing they install
> immediately. Of course we can just document that clearly on downloads pages
> so people won't get lost. What do you all think? Should Ignite auto-start
> with default config? Is it possible to do so, but make possible to turn it
> off.
>
> So from my side I recommend this pull request for inclusion after 1) is
> done and 2) is discussed.
>
> Also, would be nice if you create additional JIRA tickets for issues in my
> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
> working) to be implemented in future releases.
>
> Regards,
>
> --
> Ilya Kasnacheev
>
> 2017-12-26 15:25 GMT+03:00 Petr Ivanov :
>
>> Removed replacement for default-config.xml.
>> Also reimplemented service to be able to run multiple instances of Ignite.
>>
>> See updated PR [1].
>>
>>
>> [1] https://github.com/apache/ignite/pull/3171 <
>> https://github.com/apache/ignite/pull/3171>
>>
>>
>>
>> > On 25 Dec 2017, at 18:57, Pavel Tupitsyn  wrote:
>> >
>> > PDS and discovery settings are unrelated to the packaging task.
>> > Andrey is right, just use existing default config.
>> >
>> > On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov 
>> wrote:
>> >
>> >> Can we change default configuration file then?
>> >> So that both binary and package deliveries contained PDS turned on by
>> >> default?
>> >>
>> >> And what about Multicast Discovery?
>> >>
>> >>
>> >>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan 
>> >> wrote:
>> >>>
>> >>> I agree with Andrey. The product should be identical regardless of how
>> >> you
>> >>> download and install it.
>> >>>
>> >>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura 
>> wrote:
>> >>>
>>  Hi,
>> 
>>  I think we should provide the same default configuration for all
>>  binary builds. From my point of view RPM/DEB/etc packages should use
>>  default configuration file like to standard binary release.
>> 
>>  On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>   wrote:
>> > Hello Igniters!
>> >
>> > What's your take on enabling PDS in 2.4 in default config? This way
>> we
>> > could enable it in RPM build with easy heart.
>> >
>> > The rest of your answers seem spot on. I'll happily review your
>> amended
>> > patch when it is ready (later this week?)
>> >
>> > --
>> > Ilya Kasnacheev
>> >
>> > 2017-12-22 18:27 GMT+03:00 vveider :
>> >
>> >> Ilya Kasnacheev wrote
>> >>> I have noticed that both spec and DEVNOTES are made to use tabs
>>  alongside
>> >>> with spaces. I thought we are avoiding tabs? Please confirm that
>>  usage of
>> >>> tabs is desired in this case.
>> >>
>> >> My fault - got used to editing such files in default vim. Fixed and
>>  updated
>> >> vim settings to indent with spaces.
>> >>
>> >>
>> >> Ilya Kasnacheev wrote
>> >>> Maybe it's my CentOS but initially build will fail with the
>> following
>> >>> message, which I had to fix in spec
>> >>> + cd 'apache-ignite-*'
>> >>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>> file
>> >> or
>> >>> directory
>> >>
>> >> Strange error - shell expansion is not working. Anyway, replaced
>> with
>>  more
>> >> universal variant.
>> >>
>> >>
>> >> Ilya Kasnacheev wrote
>> >>> We absolutely should not inline configuration files (such as
>> >>> default-config.xml) into build scripts. We should just copy over
>> >>> default-config.xml from config/ to /etc, with dependencies if
>> needed.
>> >>
>> >> Extracted corresponding sections into separate files.
>> >>
>> >>
>> >> Ilya Kasnacheev wrote
>> >>> I'm not sure PDS should be ON by default. Better provide it in
>>  examples
>> >>> but disable by default.
>> >>
>> >> AFAIK, PDS is actively pushing forward and has to be enabled by
>> >> default
>>  so
>> >> that Ignite users sta

Re: IGNITE-7107 Apache Ignite RPM packages

[Ilya Kasnacheev]

Hello again!

I have reviewed the modified patch. All the things in my critical list were
fixed. I have just two minor remarks:

- Please move sqlline.sh back from examples to /usr/share/a-i/bin. It works
just as well from there as it was from our distribution. visor doesn't,
hence it stays in examples.
- I'm a bit confused that we no longer auto-start service after package is
installed. You have to do
sudo systemctl start apache-ign...@default-config.xml
manually. I'm used to packages that start the thing they install
immediately. Of course we can just document that clearly on downloads pages
so people won't get lost. What do you all think? Should Ignite auto-start
with default config? Is it possible to do so, but make possible to turn it
off.

So from my side I recommend this pull request for inclusion after 1) is
done and 2) is discussed.

Also, would be nice if you create additional JIRA tickets for issues in my
minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
working) to be implemented in future releases.

Regards,

-- 
Ilya Kasnacheev

2017-12-26 15:25 GMT+03:00 Petr Ivanov :

> Removed replacement for default-config.xml.
> Also reimplemented service to be able to run multiple instances of Ignite.
>
> See updated PR [1].
>
>
> [1] https://github.com/apache/ignite/pull/3171  ignite/pull/3171>
>
>
>
> > On 25 Dec 2017, at 18:57, Pavel Tupitsyn  wrote:
> >
> > PDS and discovery settings are unrelated to the packaging task.
> > Andrey is right, just use existing default config.
> >
> > On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov 
> wrote:
> >
> >> Can we change default configuration file then?
> >> So that both binary and package deliveries contained PDS turned on by
> >> default?
> >>
> >> And what about Multicast Discovery?
> >>
> >>
> >>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan 
> >> wrote:
> >>>
> >>> I agree with Andrey. The product should be identical regardless of how
> >> you
> >>> download and install it.
> >>>
> >>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura  wrote:
> >>>
>  Hi,
> 
>  I think we should provide the same default configuration for all
>  binary builds. From my point of view RPM/DEB/etc packages should use
>  default configuration file like to standard binary release.
> 
>  On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>   wrote:
> > Hello Igniters!
> >
> > What's your take on enabling PDS in 2.4 in default config? This way
> we
> > could enable it in RPM build with easy heart.
> >
> > The rest of your answers seem spot on. I'll happily review your
> amended
> > patch when it is ready (later this week?)
> >
> > --
> > Ilya Kasnacheev
> >
> > 2017-12-22 18:27 GMT+03:00 vveider :
> >
> >> Ilya Kasnacheev wrote
> >>> I have noticed that both spec and DEVNOTES are made to use tabs
>  alongside
> >>> with spaces. I thought we are avoiding tabs? Please confirm that
>  usage of
> >>> tabs is desired in this case.
> >>
> >> My fault - got used to editing such files in default vim. Fixed and
>  updated
> >> vim settings to indent with spaces.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >>> Maybe it's my CentOS but initially build will fail with the
> following
> >>> message, which I had to fix in spec
> >>> + cd 'apache-ignite-*'
> >>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file
> >> or
> >>> directory
> >>
> >> Strange error - shell expansion is not working. Anyway, replaced
> with
>  more
> >> universal variant.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >>> We absolutely should not inline configuration files (such as
> >>> default-config.xml) into build scripts. We should just copy over
> >>> default-config.xml from config/ to /etc, with dependencies if
> needed.
> >>
> >> Extracted corresponding sections into separate files.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >>> I'm not sure PDS should be ON by default. Better provide it in
>  examples
> >>> but disable by default.
> >>
> >> AFAIK, PDS is actively pushing forward and has to be enabled by
> >> default
>  so
> >> that Ignite users start working with PDS from the very beginning.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >>> I cannot comment on firewall rules validity, but firewall and
> service
> >>> configurations should be stored in sources, as files, supplied with
> >>> release (somewhere in packages/) and installed from there
> >>
> >> Unfortunately, I did not find any other way to open ports and
> >> multicast,
> >> except applying direct iptables rules though firewall-cmd -- so
> there
>  can
> >> be
> >> no separate service firewall rules file (as can be found here:
> >> /usr/lib/firewalld/services). Applied rules from package go straight
> >> to
> >> /etc/firewa

Re: Example for OP_PUT_BINARY_TYPE

[Pavel Tupitsyn]

Hi Prachi,

1) hash code of the field name
2) When "Is enum" is false, do not write anything, skip "Enum field count"
and "Enum values"
3) Schema is a set of field name hashes. Typically objects have a single
schema. Schema id is calculated as FNV1 hash, see
BinaryUtils.updateSchemaId.

On Thu, Dec 28, 2017 at 12:57 AM, Prachi Garg  wrote:

> Pavel,
>
> I am trying to come up with an example for OP_PUT_BINARY_TYPE = 3003.
> However, few things are not clear to me from the wiki doc -
>
> 1. In BinaryField, What is Field id?
> 2. If there is no enum, what should I put for 'Enum values'?
> 3. What is Schema?
>
> -P
>

Re: Thin Client documentation: OP_QUERY_SQL_CURSOR_GET_PAGE

[Pavel Tupitsyn]

Hi Prachi,

Code looks correct to me.
- Can you try on the latest master branch?
- Have you tried to debug the server side and see what's going on there?

Thanks,
Pavel

On Thu, Dec 28, 2017 at 3:06 AM, Prachi Garg  wrote:

> Pavel,
>
> I have 2 records in my cache, and when I run an example for
> OP_QUERY_SQL=2002 with cursor page size 1, I get the following result
> (expected)
>
> len: 83
> resReqId: 1
> status code: 0
> cursorId: 1
> rowCount: 1
> moreResults: true
>
> Then, I run the OP_QUERY_SQL_CURSOR_GET_PAGE = 2003, using the following
> code:
>
> DataOutputStream out = new DataOutputStream(socket.getOutputStream());
>
> // Message length
> writeIntLittleEndian(18, out);
>
> // Op code = QUERY_SQL_CURSOR_GET_PAGE
> writeShortLittleEndian(2003, out);
>
> // Request id (can be anything)
> long reqId = 1;
> writeLongLittleEndian(reqId, out);
>
> // Cursor Id
> writeLongLittleEndian(1, out);
>
> // Send request
> out.flush();
>
> // Read result
> DataInputStream in = new DataInputStream(socket.getInputStream());
>
> // Response length
> final int len = readIntLittleEndian(in);
> System.out.println("len: " + len);
>
> // Request id
> long resReqId = readLongLittleEndian(in);
> System.out.println("resReqId: " + resReqId);
>
> // Success
> int statusCode = readIntLittleEndian(in);
> System.out.println("status code: " + statusCode);
>
> int rowCount = readIntLittleEndian(in);
> System.out.println("rowCount: " + rowCount);
>
> // Read entries (as user objects)
> for (int i = 0; i < rowCount; i++){
> // ...
> }
>
> boolean moreResults = readBooleanLittleEndian(in);
> System.out.println("moreResults: " + moreResults);
>
> but the result is not what is expected:
>
> len: 1
> resReqId: 44096429228032
> status code: 721512192
> rowCount: 1672392448
> moreResults: true
>
> Also, there is no error on server side.
>
> Anything wrong in the OP_QUERY_SQL_CURSOR_GET_PAGE request that I am
> sending?
>
> -Prachi
>

[jira] [Created] (IGNITE-7324) AssertionError Entry cannot become obsolete when DHT local candidate is added in deadlock detection suite

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7324:


 Summary: AssertionError Entry cannot become obsolete when DHT 
local candidate is added in deadlock detection suite
 Key: IGNITE-7324
 URL: https://issues.apache.org/jira/browse/IGNITE-7324
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
Reporter: Alexey Goncharuk
Assignee: Alexey Goncharuk
 Fix For: 2.4


I observed this assertion in deadlock detection suite on TC:
{code}
[12:11:30]W: [org.apache.ignite:ignite-core] 
java.lang.AssertionError: Entry cannot become obsolete when DHT local candidate 
is added [e=GridDhtCacheEntry [rdrs=[], part=39, 
super=GridDistributedCacheEntry [super=GridCacheMapEntry 
[key=KeyCacheObjectImpl [part=39, val=39, hasValBytes=true], val=null, 
startVer=1514452286919, ver=GridCacheVersion [topVer=125932251, 
order=1514452286919, nodeOrder=2], hash=39, extras=GridCacheObsoleteEntryExtras 
[obsoleteVer=GridCacheVersion [topVer=2147483647, order=0, nodeOrder=0]], 
flags=2]]], 
ex=org.apache.ignite.internal.processors.cache.GridCacheEntryRemovedException]
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.map(GridDhtLockFuture.java:902)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.onOwnerChanged(GridDhtLockFuture.java:655)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheMvccManager.notifyOwnerChanged(GridCacheMvccManager.java:226)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheMvccManager.access$200(GridCacheMvccManager.java:80)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheMvccManager$3.onOwnerChanged(GridCacheMvccManager.java:163)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.checkOwnerChanged(GridCacheMapEntry.java:3709)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.GridDistributedCacheEntry.removeLock(GridDistributedCacheEntry.java:400)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheEntry.removeLock(GridDhtCacheEntry.java:329)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.GridDistributedCacheEntry.txUnlock(GridDistributedCacheEntry.java:665)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.unlockMultiple(IgniteTxManager.java:1672)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.rollbackTx(IgniteTxManager.java:1287)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userRollback(IgniteTxLocalAdapter.java:980)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3080)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:418)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.rollbackNearTxLocalAsync(GridNearTxLocal.java:3316)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.access$4900(GridNearTxLocal.java:122)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$26.run(GridNearTxLocal.java:4129)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6673)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
[12:11:30]W: [org.apache.ignite:ignite-core]at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[12:11:30]W: [org.apach

Re: Create cache using thin protocol

[Pavel Tupitsyn]

Changes merged to master, doc updated:
https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol

Note that OP_CACHE_GET_CONFIGURATION still returns full config,
but OP_CACHE_CREATE_WITH_CONFIGURATION now allows partial config in
key-value form.

Thanks,
Pavel

On Fri, Dec 22, 2017 at 8:25 PM, Denis Magda  wrote:

> Pavel, thanks for a quick turnaround. Please ping Prachi when you merge
> the improvement so that she can update the readme doc.
>
> —
> Denis
>
> > On Dec 22, 2017, at 4:43 AM, Pavel Tupitsyn 
> wrote:
> >
> > Ticket created: https://issues.apache.org/jira/browse/IGNITE-7288
> > I'll implement this next week.
> >
> > On Fri, Dec 22, 2017 at 10:06 AM, Pavel Tupitsyn 
> > wrote:
> >
> >> Hi Denis,
> >>
> >> I think we can assign an id to each parameter. It is not difficult to
> add.
> >>
> >>> not to tell about this operation
> >> Hiding stuff is not a good approach. Any decent engineer will have a
> look
> >> at the server code and see it.
> >>
> >> Pavel
> >>
> >> On Fri, Dec 22, 2017 at 3:45 AM, Denis Magda  wrote:
> >>
> >>> Hi Pavel,
> >>>
> >>> If it’s supposed to be used this way then it’s better not to tell about
> >>> this operation at all until it’s simplified from a user standpoint.
> >>>
> >>> My suggestion would be to design the operation so that I can pass only
> >>> those configuration parameters that need to be different from the
> defaults.
> >>> How difficult is this and can the improvement go into 2.4?
> >>>
> >>> —
> >>> Denis
> >>>
>  On Dec 20, 2017, at 10:48 PM, Pavel Tupitsyn 
> >>> wrote:
> 
>  Hi Prachi,
> 
>  Yes, you have to provide full configuration. The error is caused by
>  incomplete request message.
> 
>  Alternatively, define cache templates on server (configs with * in
> them)
>  and then use OP_CACHE_CREATE_WITH_NAME from thin client.
> 
>  Thanks,
>  Pavel
> 
>  On Thu, Dec 21, 2017 at 3:47 AM, Prachi Garg 
> >>> wrote:
> 
> > Pavel,
> >
> > I tried to create a cache with configuration, using the thin
> protocol.
> >
> > 1. Do I need to provide every configuration as mentioned in the wiki,
> >>> or I
> > can just provide a few and the rest can be default?
> >
> > 2. Using the below code:
> >
> > DataOutputStream out = new DataOutputStream(socket.
> getOutputStream());
> >
> > // Message length
> > writeIntLittleEndian(22, out);
> >
> > // Op code = OP_CACHE_CREATE_WITH_CONFIGURATION
> > writeShortLittleEndian(1053, out);
> >
> > // Request id (can be anything)
> > long reqId = 1;
> > writeLongLittleEndian(reqId, out);
> >
> > // CacheAtomicityMode
> > writeIntLittleEndian(0, out);
> >
> > // Backups
> > writeIntLittleEndian(2, out);
> >
> > // CacheMode
> > writeIntLittleEndian(2, out);
> >
> > // Read result
> > DataInputStream in = new DataInputStream(socket.getInputStream());
> >
> > // Response length
> > final int len = readIntLittleEndian(in);
> > System.out.println("len: " + len);
> >
> > // Request id
> > long resReqId = readLongLittleEndian(in);
> > System.out.println("resReqId: " + resReqId);
> >
> > // Success
> > int statusCode = readIntLittleEndian(in);
> > System.out.println("status code: " + statusCode);
> >
> >
> > I get the following error:
> >
> > [2017-12-20 16:43:32,800][ERROR][client-co
> >>> nnector-#45][ClientListenerNioListener] Failed to parse client
> request.
> > class org.apache.ignite.binary.BinaryObjectException: Not enough
> data
> >>> to read the value [position=22, requiredBytes=4, remainingBytes=0]
> > at org.apache.ignite.internal.binary.streams.BinaryAbstractInpu
> >>> tStream.ensureEnoughData(BinaryAbstractInputStream.java:305)
> > at org.apache.ignite.internal.binary.streams.BinaryAbstractInpu
> >>> tStream.readInt(BinaryAbstractInputStream.java:127)
> > at org.apache.ignite.internal.binary.BinaryReaderExImpl.readInt
> >>> (BinaryReaderExImpl.java:743)
> > at org.apache.ignite.internal.processors.platform.client.cache.
> >>> ClientCacheConfigurationSerializer.read(ClientCacheConfigura
> >>> tionSerializer.java:125)
> > at org.apache.ignite.internal.processors.platform.client.cache.
> >>> ClientCacheCreateWithConfigurationRequest.(ClientCache
> >>> CreateWithConfigurationRequest.java:45)
> > at org.apache.ignite.internal.processors.platform.client.Client
> >>> MessageParser.decode(ClientMessageParser.java:333)
> > at org.apache.ignite.internal.processors.platform.client.Client
> >>> MessageParser.decode(ClientMessageParser.java:220)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNio
> >>> Listener.onMessage(ClientListenerNioListener.java:119)
> > at org.apache.ignite.internal.processors.odbc.ClientListenerNio
> >>> Listener.onMessage(ClientListenerNioListener.

[GitHub] ignite pull request #3286: IGNITE-7288 Thin client: partial cache configurat...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3286


---

[GitHub] ignite pull request #3310: IGNITE-7239 - In case of not serializable cache u...

[dkarachentsev]

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/3310

IGNITE-7239 - In case of not serializable cache update response, futu…

…re on node requester will never complete

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7239

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3310.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3310


commit e0a1ec657b6e638d4ed28fc3d4db4ff552929cff
Author: dkarachentsev 
Date:   2017-12-28T09:32:06Z

IGNITE-7239 - In case of not serializable cache update response, future on 
node requester will never complete




---

[GitHub] ignite pull request #3309: IGNITE-6456: Added opportunity to enable/disable ...

[dolphin1414]

GitHub user dolphin1414 opened a pull request:

https://github.com/apache/ignite/pull/3309

IGNITE-6456: Added opportunity to enable/disable different clients.

Added flags to ClientConnectorConfiguration which enable/disable different 
clients. Tests implemented.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6456

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3309


commit d41ae61b8605df7c4f69af38c4b6f53f0f115cf2
Author: rkondakov 
Date:   2017-12-28T09:42:32Z

IGNITE-6456: Added flags to ClientConnectorConfiguration which 
enable/disable different clients.




---

[GitHub] ignite pull request #3308: IGNITE-5217: Add Gradient Descent and QR-based tr...

[dmitrievanthony]

GitHub user dmitrievanthony opened a pull request:

https://github.com/apache/ignite/pull/3308

IGNITE-5217: Add Gradient Descent and QR-based trainers for Linear 
Regression



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5217

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3308.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3308


commit e95d8b6154f7ae1dc9b2e97a69b1abda145b0679
Author: dmitrievanthony 
Date:   2017-12-22T15:08:00Z

IGNITE-5217 Added initial version of Gradient Descent trainer of OLS 
regression

commit 1d67f26b1e85159e8734ec21d16bee0693ef0272
Author: Антон Дмитриев 
Date:   2017-12-24T15:09:43Z

Merge branch 'master' of https://github.com/apache/ignite into ignite-5217

commit bd76022b11f7a3b0f9df5e1a85daebacfe094f8e
Author: dmitrievanthony 
Date:   2017-12-25T05:39:13Z

IGNITE-5217 Move gradient descent to optimization package, add test
datasets, add BarzilaiBorwein gradient descent updater.

commit 254dc7881f7bd7aaecc44f03c1ab3b7754045560
Author: dmitrievanthony 
Date:   2017-12-25T09:40:42Z

IGNITE-5217 Add artificial regression datasets and refactor SGD.

commit 0b588c1241cc4d8fbb825d5cda745f5c6dbdeb9a
Author: dmitrievanthony 
Date:   2017-12-25T09:58:06Z

IGNITE-5217 Improve code style in accordance with guidelines.

commit 22fb769f8f578e180d178397ecae853d4c3baf70
Author: dmitrievanthony 
Date:   2017-12-25T10:29:52Z

IGNITE-5217 Make LinearRegressionModel exportable.

commit 7c0f8a4e59ec6fa83363ffbfe59d773163ef19dd
Author: dmitrievanthony 
Date:   2017-12-25T10:57:49Z

IGNITE-5217 Add simple linear regression trainer based on QR decomposition.

commit f2a9d34afb09b95d66c70a3bdd72617671910517
Author: dmitrievanthony 
Date:   2017-12-25T13:08:04Z

IGNITE-5217 Linear regression tests refactoring.

commit 92e64e9364c2c42073e6ac782feb00b86fcd5366
Author: dmitrievanthony 
Date:   2017-12-26T07:05:34Z

Merge remote-tracking branch 'origin/master' into ignite-5217

commit f29d385184735fd2a71f6753c2bbd4af133154f6
Author: dmitrievanthony 
Date:   2017-12-26T15:18:24Z

IGNITE-5217 Add distributed matrices support to linear regression.

commit ed6cd77b3926899c17ce71982b27af0cd129b6aa
Author: dmitrievanthony 
Date:   2017-12-27T17:38:11Z

IGNITE-5217 Add distributed gradient calculator.

commit 053d7511e6070a66063c5a324c6e9b81c0d3a649
Author: dmitrievanthony 
Date:   2017-12-28T08:48:34Z

IGNITE-5217 Add SparseDistributedMatrixMapReducer.

commit 0792bee4d5e2fa0b7eba20ee436c55c92114e25f
Author: dmitrievanthony 
Date:   2017-12-28T09:13:29Z

IGNITE-5217 Revert accidental changes.

commit 06fb44ce155db3a39be2b530253a2e319bbd87fd
Author: dmitrievanthony 
Date:   2017-12-28T09:19:11Z

IGNITE-5217 Revert accidental changes.




---