Re: Adding sqlline tool to Apache Ignite project

2017-10-10 Thread Denis Magda 
Oleg,

Looks good to me. Please consider the notes left in the ticket. I want us to 
prepare a script for Windows, review the language for help notice and errors, 
put together documentation. Prachi will be able to help with the editing and 
documentation.

—
Denis

> On Oct 9, 2017, at 10:13 AM, Oleg Ostanin  wrote:
> 
> New build with fixed argument parsing:
> https://ci.ignite.apache.org/viewLog.html?buildId=882282=artifacts=IgniteRelease_XxxFromMirrorIgniteRelease3PrepareVote#!1rrb2,1esn4zrslm4po,-h8h0hn9vvvxp
> 
> On Mon, Oct 9, 2017 at 5:38 PM, Denis Magda  wrote:
> 
>> I think it’s a must have for the ticket resolution.
>> 
>> Denis
>> 
>> On Monday, October 9, 2017, Anton Vinogradov 
>> wrote:
>> 
>>> Any plans to have ignitesql.bat?
>>> 
>>> On Mon, Oct 9, 2017 at 5:29 PM, Oleg Ostanin >> > wrote:
>>> 
 Another build with sqlline included:
 https://ci.ignite.apache.org/viewLog.html?buildId=881120;
 tab=artifacts=IgniteRelease_XxxFromMirrorIgniteRelease3Pre
 pareVote#!1rrb2,-wpvx2aopzexz,1esn4zrslm4po,-h8h0hn9vvvxp
 
 On Sun, Oct 8, 2017 at 5:11 PM, Denis Magda >> > wrote:
 
> No more doubts on my side. +1 for Vladimir’s suggestion.
> 
> Denis
> 
> On Saturday, October 7, 2017, Dmitriy Setrakyan <
>> dsetrak...@apache.org
>>> >
> wrote:
> 
>> I now tend to agree with Vladimir. We should always require that
>> some
>> address is specified. The help menu should clearly state how to
>>> connect
> to
>> a localhost.
>> 
>> D.
>> 
>> On Sat, Oct 7, 2017 at 12:44 AM, Vladimir Ozerov <
>>> voze...@gridgain.com 
>> >
>> wrote:
>> 
>>> Denis,
>>> 
>>> Default Ignite configuration uses multicast, this is why you do
>> not
> need
>> to
>>> change anything. Ignite node is always both a server (listens)
>> and
>>> a
>> client
>>> (connects).
>>> 
>>> This will not work for ignitesql, as this is a client. And in
>> real
>>> deployments it will connect to remote nodes, not local. So the
 earlier
> we
>>> explain user how to do this, the better. This is why it should
>> not
 work
>> out
>>> of the box connecting to 127.0.0.1. No magic for users please.
>>> 
>>> This is what user will see (draft):
 ./ignitesql.sh
 Please specify the host: ignitesql.sh [host]; type --help for
>>> more
>>> information.
 ./ignitesql.sh 192.168.12.55
 Connected successfully.
>>> 
>>> Again, specifying parameters manually is not poor UX. This is
 excellent
>> UX,
>>> as user learns on his own how to connect to a node in 1 minute.
>>> Most
>>> command line tools work this way.
>>> 
>>> сб, 7 окт. 2017 г. в 7:12, Dmitriy Setrakyan <
>>> dsetrak...@apache.org 
>> >:
>>> 
 How does the binding happen? Can we bind to everything, like we
>>> do
 in
 Ignite?
 
 On Fri, Oct 6, 2017 at 2:51 PM, Denis Magda >> 
>> > wrote:
 
> Thought over 127.0.0.1 as a default host once again. The bad
 thing
>>> about
> it is that the user gets a lengthy exception stack trace if
 Ignite
>> is
 not
> running locally and not a small error message.
> 
> What are the other opinions on this? Do we want to follow
> Vladimir’s
> suggestion forcing to set the host name/IP (port is optional)
>>> for
> the
 sake
> of usability or leaver 127.0.0.1 as default?
> 
> —
> Denis
> 
>> On Oct 6, 2017, at 12:21 PM, Denis Magda <
>> dma...@apache.org
>>> 
>> > wrote:
>> 
>>> But, we need to support “help” (-h, -help) argument
>> listing
 all
>> the
> parameters accepted by the tools.
>> 
>> Meant accepted by the ignitesql script only such as host
>>> name.
>> 
>> —
>> Denis
>> 
>>> On Oct 6, 2017, at 12:20 PM, Denis Magda <
>> dma...@apache.org
>>> 
>> > wrote:
>>> 
>>> Really nice, could click through the getting started [1]
>> in
>>> a
>>> minute!
>>> 
>>> +1 to rename the script to “ignitesql”. Vladimir’s point
>>> makes
>> total
> sense.
>>> 
>>> However, tend to disagree that the host has to be
>> requested
 all
>> the
> times. We never request a configuration or host name for
 ignite.sh,
>>> visor
> or web agent scripts. I would follow this approach that’s
 excellent
>> for
 dev
> time.
>>> 
>>> But, we

Re: Ignite ML news

2017-10-10 Thread Denis Magda 
Hi Yuri,

Sounds impressive!

The only suggestion I have for you guys is to copy the dev list more for ML 
related discussions. This will let the community to keep track of your 
activities on a daily basis and do proposal to algorithms you’ve been working 
on.

Agreed?

—
Denis
 
> On Oct 10, 2017, at 11:58 AM, Yury Babak  wrote:
> 
> Hi Igniters!
> 
> I want to provide some updates about current state of ml module, so let me
> introduce that we already have:
> 
> * OLS lin regression(not so efficient in distributed case, but Alexey
> Zinoviev already working to fix that, IGNITE-6222).
> * K-means (IGNITE-5113).
> * Decision tree(almost done by Artem Malykh, IGNITE-5218).
> * Suitable for distributed calculations block matrices(IGNITE-5791).
> * BLAS support (IGNITE-5777).
> * And some minors bugfixes/improvements.
> 
> As new features now we worked on the several algorithms:
> 
> * FCM (Ilya Nozhkin, IGNITE-5246) - Fuzzy c-means, will be the first fuzzy
> algoritm in our ML module.
> * Log regression (Vladisav Jelisavcic, IGNITE-5059) - logistic regression
> and regularization mechanism.
> * SVM (Oleg Ignatenko, IGNTIE-6585) - Support vector machine, another
> popular classification algorithm.
> 
> Also we start works related with neural networks. And here we working on two
> big ares. 
> 
> The first area is implementation our own distributed neural networks over
> Ignite(IGNITE-6386). We will start with CNN(convolutional neural network).
> 
> And the second area is integration with existing libs and using them as
> backend wrapped on our API. Here we looks on Encog and Deeplearning4j libs.
> 
> Also we want to add API for group model trainings. Using this API we will
> able to train and choose best model. We even could choose between different
> models, like k-mean and decision tree for example.
> 
> Any questions or suggestions are welcome.
> 
> Regards,
> Yury
> 
> 
> 
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Ignite ML news

2017-10-10 Thread Yury Babak 
Hi Igniters!

I want to provide some updates about current state of ml module, so let me
introduce that we already have:

* OLS lin regression(not so efficient in distributed case, but Alexey
Zinoviev already working to fix that, IGNITE-6222).
* K-means (IGNITE-5113).
* Decision tree(almost done by Artem Malykh, IGNITE-5218).
* Suitable for distributed calculations block matrices(IGNITE-5791).
* BLAS support (IGNITE-5777).
* And some minors bugfixes/improvements.

As new features now we worked on the several algorithms:

* FCM (Ilya Nozhkin, IGNITE-5246) - Fuzzy c-means, will be the first fuzzy
algoritm in our ML module.
* Log regression (Vladisav Jelisavcic, IGNITE-5059) - logistic regression
and regularization mechanism.
* SVM (Oleg Ignatenko, IGNTIE-6585) - Support vector machine, another
popular classification algorithm.

Also we start works related with neural networks. And here we working on two
big ares. 

The first area is implementation our own distributed neural networks over
Ignite(IGNITE-6386). We will start with CNN(convolutional neural network).

And the second area is integration with existing libs and using them as
backend wrapped on our API. Here we looks on Encog and Deeplearning4j libs.

Also we want to add API for group model trainings. Using this API we will
able to train and choose best model. We even could choose between different
models, like k-mean and decision tree for example.

Any questions or suggestions are welcome.

Regards,
Yury



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-6594) Add lazy flag to the list of the ODBC connection string attributes.

2017-10-10 Thread Igor Sapego (JIRA) 
Igor Sapego created IGNITE-6594:
---

 Summary: Add lazy flag to the list of the ODBC connection string 
attributes.
 Key: IGNITE-6594
 URL: https://issues.apache.org/jira/browse/IGNITE-6594
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.2
Reporter: Igor Sapego
 Fix For: 2.3


Need to add {{lazy}} flag to the list of the ODBC connection string attributes 
here: https://apacheignite.readme.io/docs/connecting-string



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6593) Document C and SQL types, supported by ODBC driver.

2017-10-10 Thread Igor Sapego (JIRA) 
Igor Sapego created IGNITE-6593:
---

 Summary: Document C and SQL types, supported by ODBC driver.
 Key: IGNITE-6593
 URL: https://issues.apache.org/jira/browse/IGNITE-6593
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.2
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.3


Need to list all the available C and SQL types, supported by ODBC driver.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6592) Describe Ignite C++ pointer reading and writing semantics

2017-10-10 Thread Igor Sapego (JIRA) 
Igor Sapego created IGNITE-6592:
---

 Summary: Describe Ignite C++ pointer reading and writing semantics
 Key: IGNITE-6592
 URL: https://issues.apache.org/jira/browse/IGNITE-6592
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.2
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.3


Need to describe how user can read and write nullable values using pointer 
semantic in Ignite C++:
{code}
// One can write null value like this:
writer.WriteObject(nullptr);
// And read it like this:
std::unique_ptr nullableVal = reader.ReadObject();
if (nullableVal) {
  // Processing...
}
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2460: IGNITE-5790 SQL: Append UUID to names of Ignite i...

2017-10-10 Thread alamar 
Github user alamar closed the pull request at:

https://github.com/apache/ignite/pull/2460


---

[GitHub] ignite pull request #2484: Backport IGNITE-3196

2017-10-10 Thread alamar 
Github user alamar closed the pull request at:

https://github.com/apache/ignite/pull/2484


---

[GitHub] ignite pull request #2825: IGNITE-6588

2017-10-10 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2825


---

[jira] [Created] (IGNITE-6591) JdbcThinErrorsSelfTest.testBatchUpdateException() should be removed, as batch updates already supported

2017-10-10 Thread Evgenii Zhuravlev (JIRA) 
Evgenii Zhuravlev created IGNITE-6591:
-

 Summary: JdbcThinErrorsSelfTest.testBatchUpdateException() should 
be removed, as batch updates already supported
 Key: IGNITE-6591
 URL: https://issues.apache.org/jira/browse/IGNITE-6591
 Project: Ignite
  Issue Type: Bug
Reporter: Evgenii Zhuravlev
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2826: IGNITE-6140 JDBC thin: implement DataSource inter...

2017-10-10 Thread tledkov-gridgain 
GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/2826

IGNITE-6140  JDBC thin: implement DataSource interface



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6140

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2826.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2826


commit d3753728fcab2e1794f98bec1135d67343d80cd8
Author: tledkov-gridgain 
Date:   2017-10-06T13:23:09Z

IGNITE-6140: save the progress

commit a57e1491c724fbd77e975ac12707a35870a49983
Author: tledkov-gridgain 
Date:   2017-10-09T09:10:02Z

IGNITE-6140: save the progress

commit 720c003ecc9f58938cabb3b0a07cb222cd99abe4
Author: tledkov-gridgain 
Date:   2017-10-09T12:02:15Z

Merge branch '_master' into ignite-6140

commit 548ed8f3f63626d3fa0276a419188ab4815ceca5
Author: tledkov-gridgain 
Date:   2017-10-09T13:16:48Z

IGNITE-6140: save the progress

commit 7b04997fa15ad2be9fd462cf29302d1c57486e2c
Author: tledkov-gridgain 
Date:   2017-10-10T12:01:54Z

Merge branch '_master' into ignite-6140

commit 634d7e6dd1477e0ab2d60b1b0d49d5984a444924
Author: tledkov-gridgain 
Date:   2017-10-10T13:47:57Z

IGNITE-6140 JDBC thin: implement DataSource interface

commit b83ff657ac20523d8c6bac4cf8a3522e554a0822
Author: tledkov-gridgain 
Date:   2017-10-10T13:48:43Z

IGNITE-6140: add test to suite




---

[jira] [Created] (IGNITE-6590) TCP port in bin/control.sh differs from default

2017-10-10 Thread Ilya Kasnacheev (JIRA) 
Ilya Kasnacheev created IGNITE-6590:
---

 Summary: TCP port in bin/control.sh differs from default
 Key: IGNITE-6590
 URL: https://issues.apache.org/jira/browse/IGNITE-6590
 Project: Ignite
  Issue Type: Bug
  Components: clients
Affects Versions: 2.2
Reporter: Ilya Kasnacheev


{code}
% bin/ignite.sh -v
>>> Local ports: TCP:8081 TCP:10800 TCP:11211 TCP:47100 TCP:47500 

% bin/control.sh --host 127.0.0.1
окт 10, 2017 3:01:26 PM org.apache.ignite.internal.client.impl.GridClientImpl 

WARNING: Failed to initialize topology on client start. Will retry in 
background.
Caused by: class 
org.apache.ignite.internal.client.GridServerUnreachableException: Failed to 
connect to any of the servers in list: [/127.0.0.1:11212]
{code}

11212 != 11211. But it's very hard to spot visually. Please fix control.sh to 
use correct port by default.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6589) Encountered incompatible class loaders for cache

2017-10-10 Thread Vladislav Pyatkov (JIRA) 
Vladislav Pyatkov created IGNITE-6589:
-

 Summary: Encountered incompatible class loaders for cache
 Key: IGNITE-6589
 URL: https://issues.apache.org/jira/browse/IGNITE-6589
 Project: Ignite
  Issue Type: Bug
Reporter: Vladislav Pyatkov


By unknown reasons DeploymentManager forces to use objects with compatible 
classloader.
{noformat}
class org.apache.ignite.IgniteCheckedException: Encountered incompatible class 
loaders for cache [class1=org.apache.ignite.tests.p2p.cache.Person, 
class2=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap]
at 
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.registerClass(GridCacheDeploymentManager.java:642)
at 
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.registerClass(GridCacheDeploymentManager.java:586)
at 
org.apache.ignite.internal.processors.cache.GridCacheMessage.prepareObject(GridCacheMessage.java:223)
at 
org.apache.ignite.internal.processors.cache.GridCacheMessage.marshalInvokeArguments(GridCacheMessage.java:444)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateInvokeRequest.prepareMarshal(GridNearAtomicSingleUpdateInvokeRequest.java:192)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onSend(GridCacheIoManager.java:1120)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1154)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1205)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:311)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2825: IGNITE-6588

2017-10-10 Thread devozerov 
GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/2825

IGNITE-6588



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6588

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2825.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2825


commit 1723fbdf3d8ed9c3e81f4388eedc0c64653299c1
Author: devozerov 
Date:   2017-10-10T12:00:01Z

Done.




---

[jira] [Created] (IGNITE-6588) SQL: optimize segment resolution in GridH2IndexBase when index is not segmented

2017-10-10 Thread Vladimir Ozerov (JIRA) 
Vladimir Ozerov created IGNITE-6588:
---

 Summary: SQL: optimize segment resolution in GridH2IndexBase when 
index is not segmented
 Key: IGNITE-6588
 URL: https://issues.apache.org/jira/browse/IGNITE-6588
 Project: Ignite
  Issue Type: Task
  Components: sql
Affects Versions: 2.1
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: 2.3


Currently we go through relatively slow path to determine index segment - 
unwrap key, resolved partition, etc..

All of this is not needed when index is not segmented.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6587) Ignite watchdog service

2017-10-10 Thread Alexey Goncharuk (JIRA) 
Alexey Goncharuk created IGNITE-6587:


 Summary: Ignite watchdog service
 Key: IGNITE-6587
 URL: https://issues.apache.org/jira/browse/IGNITE-6587
 Project: Ignite
  Issue Type: Improvement
  Components: general
Affects Versions: 2.2
Reporter: Alexey Goncharuk
 Fix For: 2.4


We need to come up with a 'watchdog service' to monitor for Ignite node local 
health and kill the process under some critical conditions.
For example, if one of the mission-critical Ignite threads die, the Ignite node 
must be stopped.
At the first glance, the list of critical threads is:
All TCP discovery threads
All communication NIO threads (acceptor and workers)
Exchange worker
Striped pool threads
Timeout Worker
Checkpointer 
WAL archiver

The mechanism should support pluggable components so that self-check can be 
extended via plugins.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Identifying grid transactions

2017-10-10 Thread Yakov Zhdanov 
Val, I really like your idea to implement common mechanism for all APIs.
Can you please file a ticket?

--Yakov

Re: Download links on ignite.apache.org not always working

2017-10-10 Thread Ilya Kasnacheev 
Dmitriy,

Yes, I still see this mirror in "Selected mirror:" box and it's usually
default for me. However it looks that it is up today and I'm able to
download Ignite from it.

I wonder what's long term status of this mirror though.

Regards,

-- 
Ilya Kasnacheev

2017-10-09 21:08 GMT+03:00 Dmitriy Setrakyan :

> Ilya,
>
> I do not see this mirror in the list. Do you still see it? If yes, we can
> try filing an INFRA issue in JIRA.
>
> D.
>
> On Mon, Oct 9, 2017 at 10:00 AM, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello Igniters,
>>
>> It came to my attention that I am offered to download
>> http://mirror.linux-ia64.org/apache//ignite/2.2.0/apache-ign
>> ite-fabric-2.2.0-bin.zip
>> This links look fishy to me, and it doesn't work anyway, unable to
>> connect.
>>
>> Can we please at least kick mirror.linux-ia64.org from the list of
>> mirrors?
>>
>> This is from
>> https://ignite.apache.org/download.cgi
>>
>> Regards,
>>
>> --
>> Ilya Kasnacheev
>>
>
>

[GitHub] ignite pull request #2824: Ignite 1.9.7 b1

2017-10-10 Thread ntikhonov 
GitHub user ntikhonov opened a pull request:

https://github.com/apache/ignite/pull/2824

Ignite 1.9.7 b1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.9.7-b1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2824.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2824


commit 72882126c047b937bd5ed93c85e28262530e4977
Author: Vasiliy Sisko 
Date:   2017-03-30T04:08:10Z

IGNITE-4838 Fixed internal task detection logic. Added tests.
(cherry picked from commit ba68c6c)

commit 0d3d93cd4eeb589f1b6a11b48e429defad01c82f
Author: dkarachentsev 
Date:   2017-05-18T16:11:08Z

IGNITE-4842 Now containsKey() respects isReadFromBackup() flag.

(cherry picked from commit d84fd29)

commit 4bf9123a6cff3b1fd17e8772bb0496108fce3f5e
Author: dkarachentsev 
Date:   2017-05-18T16:13:47Z

Merge remote-tracking branch 'origin/ignite-1.8.7' into ignite-1.8.7

commit 2a818d36395dd1af23acf444adf396b2e2edbede
Author: Konstantin Dudkov 
Date:   2017-05-22T13:28:07Z

Fixed "IGNITE-4205 CassandraCacheStore should start IiteThread threads in 
loadCache() method"

Signed-off-by: nikolay_tikhonov 

commit 04fadd4a499239176ba21c390d93e30809abb4c1
Author: dkarachentsev 
Date:   2017-05-23T12:42:20Z

IGNITE-5223 Allow use local binary metadata cache if it's possible

commit b2040b7a95e421609bcf7ae05b56dc623310b409
Author: dkarachentsev 
Date:   2017-05-23T13:14:08Z

IGNITE-5259 Minor serialization fix

commit b77428d12658b3ab2cdd43ca61ed71d329e83283
Author: sboikov 
Date:   2017-01-10T13:59:17Z

Do not evict removed entries, otherwise removes can be lost.

(cherry picked from commit 55ac6e7)

commit 29187ef6b663eafe67eaaaf38e4c09fc244ac7aa
Author: dkarachentsev 
Date:   2017-05-24T14:31:27Z

Do not evict removed entries, otherwise removes can be lost.

Rollback due to test failings.

commit 442aac2507210d39b7f30ab8f8d9a3dbe2610cae
Author: Andrey V. Mashenkov 
Date:   2017-05-24T15:32:11Z

IGNITE-5225: Fix NPE caused by changes in IGNITE-4577.

(cherry picked from commit d463840)

commit b1736c0bd87d6cfb65f9ef422241e0f1aba04c8d
Author: Andrey V. Mashenkov 
Date:   2017-05-24T15:48:52Z

Fixed thread pools incorrect shutdown.

(cherry picked from commit 66cef22)

commit 15d94b432fdfe458a826df6ad3c30a0408a93f49
Author: Andrey V. Mashenkov 
Date:   2017-05-25T11:27:08Z

Backport of IGNITE-4336: Manual rebalance can't be requested twice.
(cherry picked from commit 9a691c4)

commit 3a12fee29625de8d75a291e39b7d52c5f5111fb4
Author: Andrey V. Mashenkov 
Date:   2017-05-25T16:38:59Z

Minors fix segmented indices snapshots.

commit 7362d3c692c28245b193658d727b20caa62ffd38
Author: dkarachentsev 
Date:   2017-05-25T17:13:01Z

Merge compilation fix

commit 26072dffb8f5b28693731f8367872a8e1e6dfe7e
Author: agura 
Date:   2017-05-18T16:40:09Z

ignite-5203 Simple BLOB support added

commit e105b4e5c9c04f966fa4ffcff0e49dc253f4f050
Author: Andrey V. Mashenkov 
Date:   2017-05-30T13:40:24Z

Merge remote-tracking branch 'origin/ignite-1.7.11' into ignite-1.8.7

# Conflicts:
#   
modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/datasource/DataSource.java
#   
modules/cassandra/store/src/test/java/org/apache/ignite/tests/IgnitePersistentStoreTest.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/service/GridServiceProcessorMultiNodeConfigSelfTest.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/service/GridServiceProcessorMultiNodeSelfTest.java
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteBinaryObjectsTestSuite.java

Merge remote-tracking branch 'origin/ignite-1.7.11' into ignite-1.8.7

# Conflicts:
#   docs/RELEASE_NOTES.txt
#   docs/community/RELEASE_NOTES.txt
#   
modules/compatibility/src/test/java/org/gridgain/grid/compatibility/tests/GridCompatibilityAbstractTest.java
#   modules/core/src/main/resources/gridgain.properties

commit d77a134fffee431cd7fa0bae2349419bc97ec1cf
Author: dkarachentsev

[jira] [Created] (IGNITE-6586) Spring bean as ignite service

2017-10-10 Thread Denis Koblov (JIRA) 
Denis Koblov created IGNITE-6586:


 Summary: Spring bean as ignite service
 Key: IGNITE-6586
 URL: https://issues.apache.org/jira/browse/IGNITE-6586
 Project: Ignite
  Issue Type: Wish
Reporter: Denis Koblov


Hello.
I use a spring-ignite in my application. When using ServiceGrid I noticed a 
feature.
I created the following class for the ServiceGrid:

{code:java}
public class SimpleService implements Service, ApplicationContextAware {
private ApplicationContext applicationContext;

@Override
public void setApplicationContext(ApplicationContext applicationContext) 
throws BeansException {
this.applicationContext = applicationContext;
}

@Override
public void init(ServiceContext serviceContext) throws Exception {
if (applicationContext == null) {
System.out.println("ApplicationContext is null");
}
}

@Override
public void execute(ServiceContext serviceContext) throws Exception {

}

@Override
public void cancel(ServiceContext serviceContext) {

}
}
{code}


{code:java}






















































{code}

After start the application i see following text on the output  console:



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2501: IGNITE-5280: SparseDistributedMatrix refactoring

2017-10-10 Thread ybabak 
Github user ybabak closed the pull request at:

https://github.com/apache/ignite/pull/2501


---

Re: Logical Cache Documented

2017-10-10 Thread Alexey Goncharuk 
Denis,

There is an overhead for each separate fsync syscall which may become
significant with a large number of files, so having fewer files will
perform better in general. As for the heap structures overhead, this is
true that on large topologies having several cache groups will
significantly improve heap usage.

2017-10-04 0:24 GMT+03:00 Denis Magda :

> Vladimir,
>
> Thanks for the explanation and see inline
>
> > On Oct 3, 2017, at 12:57 PM, Vladimir Ozerov 
> wrote:
> >
> > Denis,
> >
> > This is not a "must have", neither I can name it a "feature". We have
> > internal partition state metadata. When there is a lot of caches, there
> is
> > a lot of metadata. It consumes local Java heap, causes high network
> traffic
> > on rebalance, and require Ignite to create a lot of files when
> persistence
> > is enabled, what slows down checkpoints. All these problems could be
> > resolved by better storage architecture and "joining" of partition maps
> of
> > caches with same affinity functions in runtime.
> >
> > But this is difficult, so we created "cache groups" as a kind of
> shortcut.
> > It saves heap, saves network, and reduces number of files. But it comes
> at
> > a cost - now single data page contain data from different caches. This
> > causes higher than usual miss rate (and as a result more OS calls) for
> > random cache operations and index lookups.
>
> Do you mean longer traverse of the b+tree under the "higher miss rate”?
> Has anybody measured the impact? Personally, for me log(n1) is not that
> different from log(n1 + n2 + n3) unless n is a big coefficient.
>
>
> > In future it will also cause
> > poor compression rates when compression is implemented, and it will cause
> > poor scan performance when efficient scans are implemented.
> >
>
> How do we scan grouped caches presently? Simply filtering out the entries
> not belonging to a cache of interest?
>
> > To summarize, we *SHOULD NOT* advise users to use this feature unless
> they
> > have problems with high heap usage due to partition maps, or poor
> > chekpointing performance due to excessive fsyncs.
> >
>
> Ivan R., Alex G., could you comment on the checkpointing performance? I
> don’t get why a number of opened files affects it. What should matter is
> the frequency of fsync, shouldn’t it? If we have fewer files then the
> frequency will soar since every cache writes into a single destination.
>
> Vladimir, what’s about long joining process and rebalancing kick-off on
> node failure? I heard an amount of partition maps influences on this and
> put this on paper.
>
> —
> Denis
>
> > On Tue, Oct 3, 2017 at 10:48 PM, Denis Magda  wrote:
> >
> >> Vladimir,
> >>
> >> Please share more details that I can put on the paper. Presently the
> >> feature is described as a must have and I struggled finding any negative
> >> impact related info.
> >>
> >> —
> >> Denis
> >>
> >>> On Oct 3, 2017, at 12:46 PM, Vladimir Ozerov 
> >> wrote:
> >>>
> >>> Denis,
> >>>
> >>> This feature should not be enabled by default as it negatively affects
> >> read
> >>> performance.
> >>>
> >>> On Tue, Oct 3, 2017 at 10:31 PM, Denis Magda 
> wrote:
> >>>
>  Sam,
> 
>  Is there any technical limitation that prevents us from assigning
> caches
>  with similar parameters to relevant groups on-the-fly?
> 
>  After finishing the doc, I’m convinced the feature should be enabled
> by
>  default unless there are some pitfalls not known by me.
> 
>  BTW, decided to avoid logical caches term usage falling back to vivid
>  cache groups notion:
>  https://apacheignite.readme.io/docs/cache-groups <
>  https://apacheignite.readme.io/docs/cache-groups>
> 
>  —
>  Denis
> 
> > On Oct 3, 2017, at 12:10 AM, Semyon Boikov 
> >> wrote:
> >
> > Hi,
> >
> > Regarding question about  default cache group: by default cache
> groups
>  are
> > not enabled, each cache is started in separate group. Cache group is
> > enabled only if groupName is set in CacheConfiguration.
> >
> > Thanks
> >
> > On Sat, Sep 30, 2017 at 11:55 PM,  wrote:
> >
> >> Why not? Obviously compression would have to be enabled per group,
> not
>  per
> >> cache.
> >>
> >> ⁣D.​
> >>
> >> On Sep 29, 2017, 10:50 PM, at 10:50 PM, Vladimir Ozerov <
> >> voze...@gridgain.com> wrote:
> >>> And it will continue hitting us in future. For example, when data
> >>> compression is implemented, for logical caches compression rate
> will
> >> be
> >>> poor, as it would be impossbile to build efficient dictionaries in
> >>> mixed
> >>> data pages.
> >>>
> >>> On Sat, Sep 30, 2017 at 8:48 AM, Vladimir Ozerov <
> >> voze...@gridgain.com
> >
> >>> wrote:
> >>>
>  Folks,
> 
>  Honesly, to

[GitHub] ignite pull request #2815: IGNITE-6568 cache descriptor SQL attribute store/...

2017-10-10 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2815


---