date:20170824

Re: Ignite Usability: Deadlocks and Starvation

2017-08-24 Thread Dmitriy Setrakyan

On Thu, Aug 24, 2017 at 10:09 AM, Yakov Zhdanov  wrote:

> > I think as a first step, the deadlock detection should kick off after a
> > certain timeout, even if the transaction timeout was not set.
>
> > What do you think?
>
> Dmitry, I thought that was what I suggested, no? =)
>

I am never sure with you :)

Re: [DISCUSSION] Urgent Ignite bug fix release

2017-08-24 Thread Dmitriy Setrakyan

+1 on the release, let's do it ASAP.

On Thu, Aug 24, 2017 at 12:32 PM, Denis Magda  wrote:

> Igniters,
>
> Presently every Apache Ignite node requests 80% of RAM from an operating
> system. We debated a lot about this default [1] and as a practice shows
> those who voted for the lower value (20%-25%) were right.
>
> Our users started reporting about bizarre issues ([2], [3], [4]) when a
> cluster hangs, freezes for a while or even crashes during standard
> preloading phase or minor workloads.
>
> It’s turned out to be that lack of RAM causes extensive swapping and
> checkpointing activity. Even more, the issue can be reproduced on your
> laptop if you launch a couple of nodes and start data preloading.
>
> All this can be avoided if you allocated less than 80% and tune the
> checkpointing if the persistence is used. Unfortunately, only few could get
> to this point, the rest would kick Ignite out from their deployments.
>
> Thus, we have to fix the defaults ([5] and [6]) on our own and release an
> emergency release with the fixes as soon as possible.
>
> Any objections or other thoughts?
>
> [1] http://apache-ignite-developers.2346864.n4.nabble.
> com/IGNITE-5717-improvements-of-MemoryPolicy-default-size-td20264.html
> [2] http://apache-ignite-users.70518.x6.nabble.com/
> Performance-of-persistent-store-too-low-when-bulb-
> loading-tp16247p16337.html
> [3] http://apache-ignite-users.70518.x6.nabble.com/Strange-
> problems-with-Ignite-native-Persistence-when-Data-exceeds-
> Memory-td16187.html#a16194
> [4] http://apache-ignite-users.70518.x6.nabble.com/
> Activation-slow-and-Ignite-node-crashed-in-the-middle-of-
> checkpoint-td16144.html
>
> [5] https://issues.apache.org/jira/browse/IGNITE-6003
> [6] https://issues.apache.org/jira/browse/IGNITE-6182
>
> —
> Denis

[DISCUSSION] Urgent Ignite bug fix release

2017-08-24 Thread Denis Magda

Igniters,

Presently every Apache Ignite node requests 80% of RAM from an operating 
system. We debated a lot about this default [1] and as a practice shows those 
who voted for the lower value (20%-25%) were right.

Our users started reporting about bizarre issues ([2], [3], [4]) when a cluster 
hangs, freezes for a while or even crashes during standard preloading phase or 
minor workloads.

It’s turned out to be that lack of RAM causes extensive swapping and 
checkpointing activity. Even more, the issue can be reproduced on your laptop 
if you launch a couple of nodes and start data preloading.

All this can be avoided if you allocated less than 80% and tune the 
checkpointing if the persistence is used. Unfortunately, only few could get to 
this point, the rest would kick Ignite out from their deployments. 

Thus, we have to fix the defaults ([5] and [6]) on our own and release an 
emergency release with the fixes as soon as possible. 

Any objections or other thoughts?

[1] 
http://apache-ignite-developers.2346864.n4.nabble.com/IGNITE-5717-improvements-of-MemoryPolicy-default-size-td20264.html
[2] 
http://apache-ignite-users.70518.x6.nabble.com/Performance-of-persistent-store-too-low-when-bulb-loading-tp16247p16337.html
[3] 
http://apache-ignite-users.70518.x6.nabble.com/Strange-problems-with-Ignite-native-Persistence-when-Data-exceeds-Memory-td16187.html#a16194
[4] 
http://apache-ignite-users.70518.x6.nabble.com/Activation-slow-and-Ignite-node-crashed-in-the-middle-of-checkpoint-td16144.html

[5] https://issues.apache.org/jira/browse/IGNITE-6003
[6] https://issues.apache.org/jira/browse/IGNITE-6182

—
Denis

Re: [DISCUSS] Ignite Update Checker

2017-08-24 Thread Denis Magda

I see nothing wrong with this approach.

Cos, Roman, Raul, as Apache veterans, what do you think? Is it good to go?

—
Denis

> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan  wrote:
> 
> Is everyone OK with this approach? Should I file a ticket on it?
> 
> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan 
> wrote:
> 
>> Igniters,
>> 
>> There has been lots of talk of proposals about various usage metrics for
>> Ignite and nothing came of it. I would like to resurrect the topic and
>> propose something very simple and non-intrusive.
>> 
>> 1. Update Checker
>> The main purpose of the update checker is not to collect metrics, but to
>> notify users about a new version of Ignite by accessing maven.org and
>> getting the version out of the metadata file:
>> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
>> maven-metadata.xml
>> 
>> This way we do not send any information anywhere and, at the same time,
>> urge our users to download and start using the latest version of Ignite.
>> 
>> 2. Startup Counter
>> This piece is optional, but we can also get an insight in how many times a
>> certain Ignite release gets started. This is just a cool metric for the
>> community to gauge the project popularity. You can think of it as of a page
>> visit counter shown on many websites. We can even decide to display this
>> counter on the Ignite website as well.
>> 
>> To do this, we can simply add a JAR in maven for every release, e.g.
>> ignite-start-counter.jar, which will contain only 1 byte. Every time an
>> Ignite node starts, it will download this JAR in the background. Then we
>> will be able to view the number of the total downloads for this JAR in
>> Maven Central, which is essentially the number of starts of Ignite nodes.
>> 
>> *Note that neither of the above suggestions require Ignite to send or
>> track any user information whatsoever.*
>> 
>> Please reply suggesting weather you are OK with this approach.
>> 
>> D.
>>

Re: Adding Apache Ignite to Kubernetes Example

2017-08-24 Thread Denis Magda

Dani,

I’m totally for this! This is an easy way to spread a word about Ignite.

Do you want to take a lead in this activity and add the example to Kube repo? 
You can rely on me if any technical assistance is needed.

—
Denis

> On Aug 23, 2017, at 7:32 PM, Dani Traphagen  wrote:
> 
> Hi Dmitriy,
> 
> Sure do but that is off of our docs site only. This is different in that it
> is proposing to add an example or two to the actual Kubernetes project so
> when someone downloads Kubernetes, the Ignite example comes along for the
> ride.
> 
> A few other technologies have done this as shown in the link so their
> samples are baked-in to the Kubernetes download.
> 
> Thanks,
> Dani
> 
> On Wed, Aug 23, 2017 at 7:27 PM Dmitriy Setrakyan 
> wrote:
> 
>> Hm... don't we have one already?
>> 
>> https://apacheignite.readme.io/docs/kubernetes-deployment
>> 
>> D.
>> 
>> On Wed, Aug 23, 2017 at 7:23 PM, Dani Traphagen  wrote:
>> 
>>> Hi Igniters,
>>> 
>>> What are your thoughts on providing an Apache Ignite example for the
>>> Kubernetes
>>> project  as shown for some other technologies
>>> here:
>>> https://github.com/kubernetes/kubernetes/tree/master/examples?
>>> 
>>> 
>>> As a specific one to emulate, Spark
>>> 
>>> has a nice
>>> Readme
>>> >> gluster/README.md>,
>>> example and guide complete with a video on how to do the setup. The
>> overall
>>> examples directory shown above comes included with every Kubernetes
>>> download.
>>> 
>>> The idea would be to enhance Apache Ignite's visibility and contributions
>>> by providing this sample with Kubernetes downloads, as we have a
>> compatible
>>> deployment integration. We could offer a simple scale-out example or any
>>> other ideas you think may be helpful.
>>> 
>>> Please provide your ideas/opinions and if positively received, I am happy
>>> to get started on this endeavor. Also let me know if you'd be interested
>> in
>>> contributing.
>>> 
>>> Happy coding,
>>> Dani
>>> --
>>> Dani Traphagen | d...@gridgain.com
>>> Solutions Architect
>>> *GridGain*
>>> 
>> 
> -- 
> Dani Traphagen | d...@gridgain.com
> Solutions Architect
> *GridGain*

Re: Expanding Apache Ignite Use Case

2017-08-24 Thread Denis Magda

Good catch, Dmitriy! Didn’t know the pages existed. Made them up-to-date.

—
Denis

> On Aug 23, 2017, at 7:14 PM, Dmitriy Setrakyan  wrote:
> 
> Denis, great progress.
> 
> I think you forgot to update these pages:
> https://ignite.apache.org/features.html
> https://ignite.apache.org/usecases.html
> 
> D.
> 
> On Wed, Aug 23, 2017 at 6:28 PM, Denis Magda  wrote:
> 
>> The task is completed from my side. Below you’ll find the brand new pages
>> added to the site. In addition, the items of “Docs” menu were rearranged
>> for clarity.
>> 
>> Features Menu:
>> Collocated processing: https://ignite.apache.org/collocatedprocessing.html
>> 
>> Use Cases:
>> Distributed database: https://ignite.apache.org/use-
>> cases/database/distributed-database.html
>> In-Memory database: https://ignite.apache.org/use-
>> cases/database/in-memory-database.html
>> SQL database: https://ignite.apache.org/use-cases/database/sql-database.
>> html
>> Key-value database: https://ignite.apache.org/use-
>> cases/database/key-value-database.html
>> Ignite for NoSQL users: https://ignite.apache.org/use-
>> cases/comparison/ignite-for-nosql.html
>> Ignite for RDBMS users: https://ignite.apache.org/use-
>> cases/comparison/ignite-for-rdbms.html
>> 
>> Please feel free to suggest any edits. Will wait for them for a couple of
>> days and send the pages to my fellow editor and SEO for *non-technical*
>> corrections and improvements.
>> 
>> —
>> Denis
>> 
>>> On Aug 21, 2017, at 9:30 PM, Dmitriy Setrakyan 
>> wrote:
>>> 
>>> Denis, great progress.
>>> 
>>> At a first glance, I would suggest to move the page "What is Collocated
>>> Processing" to Features and add some coding examples there.
>>> 
>>> Also, I am not sure if I agree with "Featured Solutions" page.
>>> 
>>> D.
>>> 
>>> P.S. don't forget to update sitemap.xml file.
>>> 
>>> On Mon, Aug 21, 2017 at 5:38 PM, Denis Magda  wrote:
>>> 
 Igniters,
 
 I’ve been working on this ticket that aims to add more use cases we
>> have:
 https://issues.apache.org/jira/browse/IGNITE-6036 <
 https://issues.apache.org/jira/browse/IGNITE-6036>
 
 Presently these pages are ready:
 SQL database page: http://localhost/use-cases/
>> database/sql-database.html <
 http://localhost/use-cases/database/sql-database.html>
 Key-value database page: http://localhost/use-cases/
 database/key-value-database.html 
 Collocated processing page: http://localhost/collocatedprocessing.html
>> <
 http://localhost/collocatedprocessing.html>
 
 You have to install the site dev environment locally to see them. INFRA
 should set up the staging for us soon. Will appreciate if anybody can
>> look
 at the progress even with this temporary inconvenience.
 
 —
 Denis
>> 
>>

[GitHub] ignite pull request #2513: Ignite 6180

2017-08-24 Thread sergey-chugunov-1985

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/2513

Ignite 6180



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6180

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2513.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2513


commit 1e08c3fb5c02ec8acafd71b50b6ad3b749259f1a
Author: Andrey V. Mashenkov 
Date:   2017-07-31T11:14:56Z

IGNITE-4800: Lucene query may fails with NPE. This closes #2315.

commit 3fdf453e89a7bd76dff6b6d0646e3821ea3921d5
Author: Andrey V. Mashenkov 
Date:   2017-07-31T14:32:12Z

IGNITE-4800: Lucene query may fails with NPE.
Test fixed.

commit e255a564985a12113984ec02f15a4443495b8ffc
Author: Nikolay Izhikov 
Date:   2017-08-02T08:52:44Z

ignite-5712 Context switching for optimistic transactions

commit 772d462b68c7de8517d1f61e2e05ec8eefb18eac
Author: Alexey Kuznetsov 
Date:   2017-08-03T04:55:15Z

Merge branch ignite-2.1.3 into ignite-2.1.4

commit 0f3b7ca25313083e4dc35e7842931a655abd
Author: tledkov-gridgain 
Date:   2017-08-04T08:46:14Z

IGNITE-5126: Batch support for this JDBC driver. This closes #2162.

commit d1a74a4be8744528e6ed23706174041ddb0f2618
Author: devozerov 
Date:   2017-08-04T09:04:38Z

Merge remote-tracking branch 'upstream/ignite-2.1.4' into ignite-2.1.4

commit 0b3a9a7176f5ae44a96ecf700c8147193dfbf064
Author: Igor Sapego 
Date:   2017-08-04T10:18:00Z

IGNITE-5923: ODBC: SQLGetTypeInfo now works with SQL_ALL_TYPES

(cherry picked from commit 48c914d)

commit 4e0385fbc0f50548f2da3407fdfdfe939b463c67
Author: Igor Sapego 
Date:   2017-08-04T15:34:27Z

IGNITE-5939: ODBC: SQLColAttributes now works with legacy attribute codes.

(cherry picked from commit 70ffa2c)

commit 4f02504475fd1e5cc3b9f4754856e44d20fdc1cb
Author: Alexey Kuznetsov 
Date:   2017-08-07T02:41:22Z

Merge branch ignite-2.1.3 into ignite-2.1.4.

commit b093afb8231135a4904f5fffd62f5b4c332f1d47
Author: tledkov-gridgain 
Date:   2017-08-08T09:05:36Z

IGNITE-5211: Added new constructor: QueryEntity(Class keyCls, Class 
valCls). This closes #2371. This closes #2388. This closes #2407.

commit 879f19106b22e66d5f6ea94424d961d049397410
Author: devozerov 
Date:   2017-08-08T12:16:58Z

IGNITE-5982: GridMapQueryExecutor was split into several pieces.

commit 7c77b869bc3efdf19e53cc2b064f4290fd73e2b2
Author: devozerov 
Date:   2017-08-09T08:47:58Z

IGNITE-5993: Removed unused SQL-related classes and methods (old tree 
index, snapshots, etc). This closes #2414.

commit ab18fdfcc4f6db1e54fb1f3b68ba7fbc31a7f6e7
Author: Andrey V. Mashenkov 
Date:   2017-07-14T17:14:47Z

IGNITE-5452: GridTimeoutProcessor can hang on stop. This closes #2279.

commit 580b6aa8e5a8a887397eab5c4c830ec28f45cd30
Author: Alexey Kuznetsov 
Date:   2017-08-09T10:22:54Z

IGNITE-5734 Web Console: Fixed npm dependencies.
(cherry picked from commit aeafbf1)

commit 841db65e56063605475710bc170de4aea672c31d
Author: Alexey Kuznetsov 
Date:   2017-08-09T11:55:04Z

IGNITE-5987 Added -nq (visor will not quit in batch mode) option for Visor 
Cmd.
(cherry picked from commit 8d6e842)

commit 5c2097856714a7803956d754735c68b21156019c
Author: Alexey Kuznetsov 
Date:   2017-08-11T03:25:36Z

IGNITE-5902 Implemented stop caches at once.
(cherry picked from commit ebb8765)

commit 8b2461942c18f228c0107020aa28c03711bdceda
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:07:26Z

IGNITE-6012 Refactored GridJettyRestHandler.processRequest(): replace 
mapper.writeValueAsString with writeValue(stream, v).
(cherry picked from commit 3a390c8)

commit 3a7d4f4a79e7c0a23244387e3a68535375a66a87
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:18:42Z

IGNITE-6013 Optimized processing response from cluster.
(cherry picked from commit b02c481)

commit 74d6ab9916b3a01c78cdf1ad86211c9fcbb2214d
Author: Alexey Goncharuk 
Date:   2017-08-14T08:08:28Z

Merge branch 'ignite-2.1.3' into ignite-2.1.4

commit fde550bac56fd0cc7c51c62a9c291dd4c3f3030c
Author: Ilya Lantukh 
Date:   2017-08-14T08:32:11Z

IGNITE-5941 - Fixed index name length restrictions. This closes #2408

commit 13f38d79b57b395e43d42a8f3c278cf48336d7c5
Author: Dmitriy Govorukhin 
Date:

[jira] [Created] (IGNITE-6182) Change default max memory size from 80% to 20%

2017-08-24 Thread Vladimir Ozerov (JIRA)

Vladimir Ozerov created IGNITE-6182:
---

 Summary: Change default max memory size from 80% to 20%
 Key: IGNITE-6182
 URL: https://issues.apache.org/jira/browse/IGNITE-6182
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 2.1
Reporter: Vladimir Ozerov
Assignee: Igor Seliverstov
Priority: Critical
 Fix For: 2.2


Currently we can allocate up to 80% of available RAM by default. It could lead 
to swap with potential risk of hang.

In order to improve user experience, we need to do the following:
1) Decrease default to 20% ({{MemoryConfiguration.DFLT_MEMORY_POLICY_FRACTION}})
2) When node starts it should sum max sizes of all memory regions plus Java's 
XMX. If result is greater than 50% of total RAM, we should issue a warning 
about potential swap.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite Usability: Deadlocks and Starvation

2017-08-24 Thread Yakov Zhdanov

> I think as a first step, the deadlock detection should kick off after a
> certain timeout, even if the transaction timeout was not set.

> What do you think?

Dmitry, I thought that was what I suggested, no? =)

--Yakov

[jira] [Created] (IGNITE-6181) Tx is not rolled back on timeout leading to potential whole grid hang

2017-08-24 Thread Alexei Scherbakov (JIRA)

Alexei Scherbakov created IGNITE-6181:
-

 Summary: Tx is not rolled back on timeout leading to potential 
whole grid hang
 Key: IGNITE-6181
 URL: https://issues.apache.org/jira/browse/IGNITE-6181
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.1
Reporter: Alexei Scherbakov
 Fix For: 2.3


Unit test reproducer:

{noformat}
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *  http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.ignite.cache;

import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteException;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.configuration.TransactionConfiguration;
import org.apache.ignite.internal.IgniteInternalFuture;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
import org.apache.ignite.transactions.Transaction;
import org.apache.ignite.transactions.TransactionConcurrency;
import org.apache.ignite.transactions.TransactionIsolation;

/**
 * Tests ability to rollback not properly closed transaction.
 */
public class IgniteTxTimeoutTest extends GridCommonAbstractTest {
/** */
private static final long TX_TIMEOUT = 3_000L;

/** */
private static final String CACHE_NAME = "test";

/** IP finder. */
private static final TcpDiscoveryVmIpFinder IP_FINDER = new 
TcpDiscoveryVmIpFinder(true);

/** */
private final CountDownLatch l = new CountDownLatch(1);

/** */
private final Object mux = new Object();

/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String 
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);

cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(IP_FINDER));

TransactionConfiguration txCfg = new TransactionConfiguration();
txCfg.setDefaultTxTimeout(TX_TIMEOUT);

cfg.setTransactionConfiguration(txCfg);

CacheConfiguration ccfg = new CacheConfiguration(CACHE_NAME);
ccfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);

cfg.setCacheConfiguration(ccfg);

return cfg;
}

/** */
public void testTxTimeoutHandling() throws Exception {
try {
final Ignite ignite = startGrid();

final AtomicBoolean released = new AtomicBoolean();

multithreadedAsync(new Runnable() {
@Override public void run() {
// Start tx with default settings.
try (Transaction tx = ignite.transactions().txStart()) {
ignite.cache(CACHE_NAME).put(1, 1);

l.countDown();

// Wait longer than default timeout.
synchronized (mux) {
while (!released.get()) {
try {
mux.wait();
}
catch (InterruptedException e) {
throw new IgniteException(e);
}
}
}

try {
tx.commit();

fail();
}
catch (IgniteException e) {
// Expect exception - tx is rolled back.
}
}
}
}, 1, "Locker");

IgniteInternalFuture fut2 = multithreadedAsync(new Runnable() {
@Override public void run() {
U.awaitQuiet(l);

// Try to acquire lock.
// Acquisition will be

[jira] [Created] (IGNITE-6180) Marshaller mappings are not restored from disk on node start

2017-08-24 Thread Sergey Chugunov (JIRA)

Sergey Chugunov created IGNITE-6180:
---

 Summary: Marshaller mappings are not restored from disk on node 
start
 Key: IGNITE-6180
 URL: https://issues.apache.org/jira/browse/IGNITE-6180
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.1
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov
 Fix For: 2.2


h2. Steps to reproduce
# Start node1 with persistence enabled.
# Put instance of custom class to cache so marshaller mapping for the class is 
created.
# Restart node1.
# Start node2 and ensure it joins the cluster with node1.
# Get instance from cache on node2 added on step #2.

h2. Expected behavior
Instance is retrieved and deserialized successfully.

h2. Actual behavior
Exception is thrown, no instance is retrieved from cache.
{noformat}
Caused by: java.lang.ClassNotFoundException: Unknown pair [platformId=0, 
typeId=-347776464]
at 
org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:392)
at 
org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:342)
at 
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:686)
... 15 more
{noformat}

JUnit test is attached to the ticket.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2512: IGNITE-5855 Type-safe parameters setting fixes cr...

2017-08-24 Thread alamar

GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/2512

IGNITE-5855 Type-safe parameters setting fixes crash when querying 
BigInteger key



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5855

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2512.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2512


commit 643f2e58415525a75c3630f8d2f174065db9a0e0
Author: Ilya Kasnacheev 
Date:   2017-08-24T16:17:04Z

IGNITE-5855 Type-safe parameters setting fixes crash when querying 
BigInteger key




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-6179) Test fail DynamicIndexReplicatedAtomicConcurrentSelfTest.testClientReconnectWithCacheRestart

2017-08-24 Thread Dmitriy Govorukhin (JIRA)

Dmitriy Govorukhin created IGNITE-6179:
--

 Summary: Test fail 
DynamicIndexReplicatedAtomicConcurrentSelfTest.testClientReconnectWithCacheRestart
 Key: IGNITE-6179
 URL: https://issues.apache.org/jira/browse/IGNITE-6179
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Dmitriy Govorukhin
Priority: Critical
 Fix For: 2.2


Test fail with assertion 
{code}
[2017-08-24 
18:34:06,207][ERROR][tcp-client-disco-msg-worker-#61%index.DynamicIndexReplicatedAtomicConcurrentSelfTest4%][IgniteClientReconnectAbstractTest$TestTcpDiscoverySpi]
 Failed to unmarshal discovery custom message.
java.lang.AssertionError
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.onSchemaFinishDiscovery(GridQueryProcessor.java:498)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.onDiscovery(GridQueryProcessor.java:894)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.onCustomEvent(GridCacheProcessor.java:2906)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery0(GridDiscoveryManager.java:660)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery(GridDiscoveryManager.java:560)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.notifyDiscovery(ClientImpl.java:2391)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.processCustomMessage(ClientImpl.java:2297)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.processDiscoveryMessage(ClientImpl.java:1874)
at 
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1758)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2511: IGNITE-6178 Make CheckpointWriteOrder.SEQUENTIAL ...

2017-08-24 Thread glukos

GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/2511

IGNITE-6178 Make CheckpointWriteOrder.SEQUENTIAL and checkpointingThreads=4 
default in persistent store confguration



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6178

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2511.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2511


commit b9246fcb261ce50604ab220e542d879c6c8d8b0d
Author: Ivan Rakov 
Date:   2017-08-24T15:18:31Z

IGNITE-6178 Make CheckpointWriteOrder.SEQUENTIAL and checkpointingThreads=4 
default in persistent store confguration




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-6178) Make CheckpointWriteOrder.SEQUENTIAL and checkpointingThreads=4 default in persistent store confguration

2017-08-24 Thread Ivan Rakov (JIRA)

Ivan Rakov created IGNITE-6178:
--

 Summary: Make CheckpointWriteOrder.SEQUENTIAL and 
checkpointingThreads=4 default in persistent store confguration
 Key: IGNITE-6178
 URL: https://issues.apache.org/jira/browse/IGNITE-6178
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.1
Reporter: Ivan Rakov
Assignee: Ivan Rakov
 Fix For: 2.2


Multithreaded and ordered checkpoints show better performance on most 
enviroments.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2510: IGNITE-6130 JDBC Thin: JdbcThinResultSet must sup...

2017-08-24 Thread tledkov-gridgain

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/2510

IGNITE-6130 JDBC Thin: JdbcThinResultSet must support types conversions



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6130

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2510.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2510


commit 99a02afab6248bb32e2bb5cda4f7d595ec04c62c
Author: tledkov-gridgain 
Date:   2017-08-24T12:57:17Z

IGNITE-6130 JDBC Thin: JdbcThinResultSet must support types conversions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-6177) Update docs for integration with Apache Zeppelin

2017-08-24 Thread Ilya Suntsov (JIRA)

Ilya Suntsov created IGNITE-6177:


 Summary: Update docs for integration with Apache Zeppelin
 Key: IGNITE-6177
 URL: https://issues.apache.org/jira/browse/IGNITE-6177
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.1
Reporter: Ilya Suntsov
 Fix For: 2.2


Since release AI 1.1 we haven't updated the following documentation section:
https://apacheignite.readme.io/v1.1/docs/data-analysis-with-apache-zeppelin



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6176) Request to BinaryMetadataTransport may cause deadlock on grid stop

2017-08-24 Thread Ivan Rakov (JIRA)

Ivan Rakov created IGNITE-6176:
--

 Summary: Request to BinaryMetadataTransport may cause deadlock on 
grid stop
 Key: IGNITE-6176
 URL: https://issues.apache.org/jira/browse/IGNITE-6176
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.1
Reporter: Ivan Rakov
 Fix For: 2.2


We may access BinaryMetadataTransport while holding striped GridCacheIOManager 
lock in some cases (for example, during partition exhange). 
When grid is stopping, BinaryMetadataTransport#requestMetadataUpdate may hang 
on future. This will result a deadlock: this future won't be cancelled. as we 
firstly stop GridCacheIOManager (which, in turn, requires releasin striped 
lock).

Steps to reproduce:
1) Remove partition exchange messages from classnames.properties
2) Run IgniteClusterActivateDeactivateTestWithPersistence multiple times

Stacktrace of deadlocked threads:
{noformat}
"sys-#9065%cache.IgniteClusterActivateDeactivateTestWithPersistence4%@18095" 
prio=5 tid=0x2b55 nid=NA waiting
  java.lang.Thread.State: WAITING
  at sun.misc.Unsafe.park(Unsafe.java:-1)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
  at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:176)
  at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:139)
  at 
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:432)
  at 
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:173)
  at 
org.apache.ignite.internal.binary.BinaryContext.updateMetadata(BinaryContext.java:1276)
  at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:783)
  at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
  at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
  at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
  at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
  at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteMap(BinaryWriterExImpl.java:786)
  at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:699)
  at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
  at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
  at 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
  at 
org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:251)
  at 
org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:82)
  at 
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
  at 
org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9849)
  at 
org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:9913)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsSingleMessage.prepareMarshal(GridDhtPartitionsSingleMessage.java:289)
  at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onSend(GridCacheIoManager.java:1120)
  at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1154)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processSinglePartitionRequest(GridDhtPartitionsExchangeFuture.java:2588)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$1500(GridDhtPartitionsExchangeFuture.java:114)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$6.apply(GridDhtPartitionsExchangeFuture.java:2492)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$6.apply(GridDhtPartitionsExchangeFuture.java:2490)
  at 
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:382)
  at 
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:352)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceivePartitionRequest(GridDhtPartitionsExchangeFuture.java:2490)
  at

[GitHub] ignite pull request #2509: For testing

2017-08-24 Thread ilantukh

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/2509

For testing



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.1.4-5682

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2509.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2509


commit 1e08c3fb5c02ec8acafd71b50b6ad3b749259f1a
Author: Andrey V. Mashenkov 
Date:   2017-07-31T11:14:56Z

IGNITE-4800: Lucene query may fails with NPE. This closes #2315.

commit 3fdf453e89a7bd76dff6b6d0646e3821ea3921d5
Author: Andrey V. Mashenkov 
Date:   2017-07-31T14:32:12Z

IGNITE-4800: Lucene query may fails with NPE.
Test fixed.

commit e255a564985a12113984ec02f15a4443495b8ffc
Author: Nikolay Izhikov 
Date:   2017-08-02T08:52:44Z

ignite-5712 Context switching for optimistic transactions

commit 772d462b68c7de8517d1f61e2e05ec8eefb18eac
Author: Alexey Kuznetsov 
Date:   2017-08-03T04:55:15Z

Merge branch ignite-2.1.3 into ignite-2.1.4

commit 0f3b7ca25313083e4dc35e7842931a655abd
Author: tledkov-gridgain 
Date:   2017-08-04T08:46:14Z

IGNITE-5126: Batch support for this JDBC driver. This closes #2162.

commit d1a74a4be8744528e6ed23706174041ddb0f2618
Author: devozerov 
Date:   2017-08-04T09:04:38Z

Merge remote-tracking branch 'upstream/ignite-2.1.4' into ignite-2.1.4

commit 0b3a9a7176f5ae44a96ecf700c8147193dfbf064
Author: Igor Sapego 
Date:   2017-08-04T10:18:00Z

IGNITE-5923: ODBC: SQLGetTypeInfo now works with SQL_ALL_TYPES

(cherry picked from commit 48c914d)

commit 4e0385fbc0f50548f2da3407fdfdfe939b463c67
Author: Igor Sapego 
Date:   2017-08-04T15:34:27Z

IGNITE-5939: ODBC: SQLColAttributes now works with legacy attribute codes.

(cherry picked from commit 70ffa2c)

commit 4f02504475fd1e5cc3b9f4754856e44d20fdc1cb
Author: Alexey Kuznetsov 
Date:   2017-08-07T02:41:22Z

Merge branch ignite-2.1.3 into ignite-2.1.4.

commit b093afb8231135a4904f5fffd62f5b4c332f1d47
Author: tledkov-gridgain 
Date:   2017-08-08T09:05:36Z

IGNITE-5211: Added new constructor: QueryEntity(Class keyCls, Class 
valCls). This closes #2371. This closes #2388. This closes #2407.

commit 879f19106b22e66d5f6ea94424d961d049397410
Author: devozerov 
Date:   2017-08-08T12:16:58Z

IGNITE-5982: GridMapQueryExecutor was split into several pieces.

commit 7c77b869bc3efdf19e53cc2b064f4290fd73e2b2
Author: devozerov 
Date:   2017-08-09T08:47:58Z

IGNITE-5993: Removed unused SQL-related classes and methods (old tree 
index, snapshots, etc). This closes #2414.

commit ab18fdfcc4f6db1e54fb1f3b68ba7fbc31a7f6e7
Author: Andrey V. Mashenkov 
Date:   2017-07-14T17:14:47Z

IGNITE-5452: GridTimeoutProcessor can hang on stop. This closes #2279.

commit 580b6aa8e5a8a887397eab5c4c830ec28f45cd30
Author: Alexey Kuznetsov 
Date:   2017-08-09T10:22:54Z

IGNITE-5734 Web Console: Fixed npm dependencies.
(cherry picked from commit aeafbf1)

commit 841db65e56063605475710bc170de4aea672c31d
Author: Alexey Kuznetsov 
Date:   2017-08-09T11:55:04Z

IGNITE-5987 Added -nq (visor will not quit in batch mode) option for Visor 
Cmd.
(cherry picked from commit 8d6e842)

commit 5c2097856714a7803956d754735c68b21156019c
Author: Alexey Kuznetsov 
Date:   2017-08-11T03:25:36Z

IGNITE-5902 Implemented stop caches at once.
(cherry picked from commit ebb8765)

commit 8b2461942c18f228c0107020aa28c03711bdceda
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:07:26Z

IGNITE-6012 Refactored GridJettyRestHandler.processRequest(): replace 
mapper.writeValueAsString with writeValue(stream, v).
(cherry picked from commit 3a390c8)

commit 3a7d4f4a79e7c0a23244387e3a68535375a66a87
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:18:42Z

IGNITE-6013 Optimized processing response from cluster.
(cherry picked from commit b02c481)

commit 74d6ab9916b3a01c78cdf1ad86211c9fcbb2214d
Author: Alexey Goncharuk 
Date:   2017-08-14T08:08:28Z

Merge branch 'ignite-2.1.3' into ignite-2.1.4

commit fde550bac56fd0cc7c51c62a9c291dd4c3f3030c
Author: Ilya Lantukh 
Date:   2017-08-14T08:32:11Z

IGNITE-5941 - Fixed index name length restrictions. This closes #2408

commit 13f38d79b57b395e43d42a8f3c278cf48336d7c5
Author: Dmitriy Govorukhin 
Date:

Re: [DISCUSS] Ignite Update Checker

2017-08-24 Thread Dmitriy Setrakyan

Is everyone OK with this approach? Should I file a ticket on it?

On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan 
wrote:

> Igniters,
>
> There has been lots of talk of proposals about various usage metrics for
> Ignite and nothing came of it. I would like to resurrect the topic and
> propose something very simple and non-intrusive.
>
> 1. Update Checker
> The main purpose of the update checker is not to collect metrics, but to
> notify users about a new version of Ignite by accessing maven.org and
> getting the version out of the metadata file:
> http://repo2.maven.org/maven2/org/apache/ignite/ignite-core/
> maven-metadata.xml
>
> This way we do not send any information anywhere and, at the same time,
> urge our users to download and start using the latest version of Ignite.
>
> 2. Startup Counter
> This piece is optional, but we can also get an insight in how many times a
> certain Ignite release gets started. This is just a cool metric for the
> community to gauge the project popularity. You can think of it as of a page
> visit counter shown on many websites. We can even decide to display this
> counter on the Ignite website as well.
>
> To do this, we can simply add a JAR in maven for every release, e.g.
> ignite-start-counter.jar, which will contain only 1 byte. Every time an
> Ignite node starts, it will download this JAR in the background. Then we
> will be able to view the number of the total downloads for this JAR in
> Maven Central, which is essentially the number of starts of Ignite nodes.
>
> *Note that neither of the above suggestions require Ignite to send or
> track any user information whatsoever.*
>
> Please reply suggesting weather you are OK with this approach.
>
> D.
>

Re: Ignite Usability: Deadlocks and Starvation

Re: [DISCUSSION] Urgent Ignite bug fix release

[DISCUSSION] Urgent Ignite bug fix release

Re: [DISCUSS] Ignite Update Checker

Re: Adding Apache Ignite to Kubernetes Example

Re: Expanding Apache Ignite Use Case

[GitHub] ignite pull request #2513: Ignite 6180

[jira] [Created] (IGNITE-6182) Change default max memory size from 80% to 20%

Re: Ignite Usability: Deadlocks and Starvation

[jira] [Created] (IGNITE-6181) Tx is not rolled back on timeout leading to potential whole grid hang

[jira] [Created] (IGNITE-6180) Marshaller mappings are not restored from disk on node start

[GitHub] ignite pull request #2512: IGNITE-5855 Type-safe parameters setting fixes cr...

[jira] [Created] (IGNITE-6179) Test fail DynamicIndexReplicatedAtomicConcurrentSelfTest.testClientReconnectWithCacheRestart

[GitHub] ignite pull request #2511: IGNITE-6178 Make CheckpointWriteOrder.SEQUENTIAL ...

[jira] [Created] (IGNITE-6178) Make CheckpointWriteOrder.SEQUENTIAL and checkpointingThreads=4 default in persistent store confguration

[GitHub] ignite pull request #2510: IGNITE-6130 JDBC Thin: JdbcThinResultSet must sup...

[jira] [Created] (IGNITE-6177) Update docs for integration with Apache Zeppelin

[jira] [Created] (IGNITE-6176) Request to BinaryMetadataTransport may cause deadlock on grid stop

[GitHub] ignite pull request #2509: For testing

Re: [DISCUSS] Ignite Update Checker

20 matches

Site Navigation

Mail list logo

Footer information