date:20170915

Ray,

This looks odd. Sure, data has to be available for you after the restart,
it's a basic capability. Then I tend to agree with the rest of the folks
that you might haven encountered the issue mentioned by Yakov. To to be
100% sure about that please do the following:

* Clean "work/db" directory completely to avoid side effects.
* Run the app that preloads data. Copy the generated logs somewhere.
* Restart the cluster and start the second app.
* Compare the first and second logs checking that the "page store" and WAL
directories are identical in both files.
* If the directories vary then you hit issue [1] and let us know if this
the case. We have to come up how to solve this automatically. As a
workaround try to send IgniteConfiguration.setConsistentId() to a
predefined value.

[1] https://issues.apache.org/jira/browse/IGNITE-6285

--
Denis

On Fri, Sep 15, 2017 at 6:52 PM, Ray  wrote:

> Yes, I tried to work with the data after restarting.
> As I stated in the original post, I can't see any cache after I restart the
> cluster.
> So I called
> https://apacheignite.readme.io/docs/rest-api#section-get-or-create-cache
> API
> with the exact cache name before restarting as parameter.
> Then I try to run a sql query with that cache name and I got an empty
> cache.
> If the data loading from disk to RAM is transparent, I should see the data
> before restarting, right?
>
> I agree with you Dmitriy, according to the debug log, the WAL folder is
> resolved.
> And I can confirm there're several WAL logs under that folder.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Dmitriy,

Check up Ray’s initial message. There is no single cache.get command or SQL 
query that can trigger data movement from disk to RAM. Plus, the output 
generated after the restart suggests that the node found all the persistence 
directories. Thus, not sure how this issue is related to consistent ID mess 
discovered by Yakov before.

Ray, please chime in and clarify if you tried to work with the data after the 
restart. 

—
Denis

> On Sep 15, 2017, at 4:02 PM, Dmitry Pavlov  wrote:
> 
> Hi Denis,
> 
> I agree that data presence should be double-checked. In case check will show 
> that 1) cache doesn't exist at all after restart and 2) we can find two 
> folders with binary_meta and wal that means this question is similar with 
> recent question 
> http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html
>  
> 
>  
> 
> It is completely transparent to user that data is in RAM or on disk.
> 
> 
> Hi Yakov,
> 
> To my mind ticket is reasonable. I can suggest to generate such warning only 
> in case consistent ID is not overriden by IgniteConfiguration. Parameter 
> override may indicate that several nodes running at the same machine.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> сб, 16 сент. 2017 г. в 1:47, Denis Magda  >:
> Alexey,
> 
> The data preloading from disk to RAM doesn’t occur automatically after the 
> restart. It will be moved to RAM on demand while your application executes 
> its logic.
> 
> Ray,
> 
> Have you executed any query (key-value, SQL) to confirm that the data is 
> missing after the shutdown?
> 
> —
> Denis
> 
> > On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin  > > wrote:
> >
> > Ray,
> >
> > Yes, enabling persistence must automatically bring all your data and 
> > metadata upon the cluster startup. Please give us more details why you 
> > think the cache was not created. BTW, do you activate the cluster after 
> > startup? Enabling persistence requires explicit cluster activation:
> >
> > $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> >
> >
>

Re: Ignite PDS WAL analysis with human readable records

Hi Denis,

Thank you for your feedback. It is very important to us to understand that
this tool may be useful not only for Ignite tests but for all developers.

I will create wiki page.

Sincerely,
Dmitriy Pavlov

сб, 16 сент. 2017 г. в 1:54, Denis Magda :

> Dmitriy, Eduard,
>
> Sounds like a useful thing. I’ll be happy to give a feedback but need a
> documentation that explains how to use the tool, what’s the output format,
> how do decipher the output, etc (basic things). The Wiki description should
> be enough.
>
> —
> Denis
>
> > On Sep 15, 2017, at 11:36 AM, Dmitry Pavlov 
> wrote:
> >
> > Hi Igniters,
> >
> > Eduard Shangareev created WAL converter utility which shows WAL
> > archive/work directory segments content in human readable form. This
> > utility outputs all WAL records to output stream (may be redirected to
> > file). How to use tips can be found in issue
> > https://issues.apache.org/jira/browse/IGNITE-6277
> >
> >
> > This utility does not require Ignite node to be up and running and may be
> > used for offline analysis of copied files. Thanks to Alexey G. for idea
> and
> > review. Code is now available in modules/ignite-dev-tools.
> >
> > 1. Could you please write if utility can be useful for you?
> >
> > 2. Please respond if you need short wiki description.
> >
> > 3. What is your opinion what should be next steps to develop this
> utility?
> >
> > 4. Does utility need property file to refer Ignite Work directory using
> > conf file.
> >
> > 5. What do you think if utility remains console non interactive, or some
> > UI/Interactive console shell may be preferable?
> >
> > Sincerely,
> >
> > Dmitriy Pavlov
>
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Hi Denis,

I agree that data presence should be double-checked. In case check will
show that 1) cache doesn't exist at all after restart and 2) we can find
two folders with binary_meta and wal that means this question is similar
with recent question
http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html

It is completely transparent to user that data is in RAM or on disk.

Hi Yakov,

To my mind ticket is reasonable. I can suggest to generate such warning
only in case consistent ID is not overriden by IgniteConfiguration.
Parameter override may indicate that several nodes running at the same
machine.

Sincerely,
Dmitriy Pavlov

сб, 16 сент. 2017 г. в 1:47, Denis Magda :

> Alexey,
>
> The data preloading from disk to RAM doesn’t occur automatically after the
> restart. It will be moved to RAM on demand while your application executes
> its logic.
>
> Ray,
>
> Have you executed any query (key-value, SQL) to confirm that the data is
> missing after the shutdown?
>
> —
> Denis
>
> > On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin 
> wrote:
> >
> > Ray,
> >
> > Yes, enabling persistence must automatically bring all your data and
> metadata upon the cluster startup. Please give us more details why you
> think the cache was not created. BTW, do you activate the cluster after
> startup? Enabling persistence requires explicit cluster activation:
> >
> > $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> >
> >
>
>

Re: Ignite PDS WAL analysis with human readable records

Dmitriy, Eduard,

Sounds like a useful thing. I’ll be happy to give a feedback but need a 
documentation that explains how to use the tool, what’s the output format, how 
do decipher the output, etc (basic things). The Wiki description should be 
enough.

—
Denis

> On Sep 15, 2017, at 11:36 AM, Dmitry Pavlov  wrote:
> 
> Hi Igniters,
> 
> Eduard Shangareev created WAL converter utility which shows WAL
> archive/work directory segments content in human readable form. This
> utility outputs all WAL records to output stream (may be redirected to
> file). How to use tips can be found in issue
> https://issues.apache.org/jira/browse/IGNITE-6277
> 
> 
> This utility does not require Ignite node to be up and running and may be
> used for offline analysis of copied files. Thanks to Alexey G. for idea and
> review. Code is now available in modules/ignite-dev-tools.
> 
> 1. Could you please write if utility can be useful for you?
> 
> 2. Please respond if you need short wiki description.
> 
> 3. What is your opinion what should be next steps to develop this utility?
> 
> 4. Does utility need property file to refer Ignite Work directory using
> conf file.
> 
> 5. What do you think if utility remains console non interactive, or some
> UI/Interactive console shell may be preferable?
> 
> Sincerely,
> 
> Dmitriy Pavlov

Re: HOLE query entry in CacheContinuousQueryPartitionRecovery

I like the name. "Black holes" pop up first in my head :) The hole can absorb 
events and confine and digest them for billions of year. Sorry for off topic.

—
Denis


> On Sep 15, 2017, at 10:14 AM, ALEKSEY KUZNETSOV  
> wrote:
> 
> Hi, Ignters!
> 
> 
> We have a strange field HOLE in CacheContinuousQueryPartitionRecovery
> 
> which compared to pending events in
> CacheContinuousQueryPartitionRecovery#collectEntries. And it is never
> equals any entry.
> 
> Do we need it ? Or it can be removed.
> 
> -- 
> 
> *Best Regards,*
> 
> *Kuznetsov Aleksey*

Re: Durable Memory and Ignite Persistence Tuning

Thanks Ivan, *no less* sounds best for me.

Prachi, please do final editing of the doc:
https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning

—
Denis

> On Sep 15, 2017, at 12:24 AM, Ivan Rakov  wrote:
> 
> Denis,
> 
> Yes, Ignite page size should be *greater or equal* than OS page cache size 
> and SSD page size. I mentioned it in advice list:
> 
>> page size of Ignite shouldn't be less than SSD page size
>> Page size of Ignite shouldn't be less than OS page size
> 
> Sorry for vague wording.
> We should fix this in documentation.
> 
> 
> Best Regards,
> Ivan Rakov
> 
> On 14.09.2017 3:46, Denis Magda wrote:
>> Ivan,
>> 
>> Documented:
>> https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning 
>> 
>> 
>> However, I’m a bit confused with you 6. point regarding the page size 
>> calculation. Should Ignite page size be always less than the SSD’s page size 
>> and OS page cache size? Or it can be equal? For instance if the OS page 
>> cache size is 4 KB what should be Ignite’s page size? That’s the section 
>> about this: 
>> https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning#section-page-size
>>  
>> 
>> 
>> —
>> Denis
>> 
>>> On Sep 13, 2017, at 2:29 AM, Ivan Rakov  wrote:
>>> 
>>> Folks,
>>> 
>>> We had some experience of benchmarking Ignite with persistent store on SSD. 
>>> I think we can share some helpful advice. None of them require changing 
>>> configuration of Ignite or persistent store.
>>> 
>>> *Tuning advice for users*
>>> 
>>> 1) Be prepared for LFS performance decrease after several hours of 
>>> intensive load. Unfortunately, that's how SSD drives work: 
>>> http://codecapsule.com/2014/02/12/coding-for-ssds-part-2-architecture-of-an-ssd-and-benchmarking/
>>> Consider buying fast production-level SSD drives.
>>> 2) Consider using separate drives for LFS files and WAL. Ignite actively 
>>> performs writes to both LFS and WAL under intensive load, and having two 
>>> devices will double your throughput limit.
>>> 3) Over-provision your SSD. Performance of random writes on 50% filled disk 
>>> is much better than on 90% filled. SSD Over-Provisioning And Its Benefits: 
>>> http://www.seagate.com/ru/ru/tech-insights/ssd-over-provisioning-benefits-master-ti/
>>> 4) Leave free space in RAM to let OS use page cache and optimize writes. 
>>> Total size of all memory policies shouldn't exceed 70% of your RAM.
>>> 5) Make sure that OS doesn't utilize swap. If you use Unix, best option is 
>>> set vm.swappiness to 0.
>>> 6) Try to find out page size of your SSD. Ideally, page size of Ignite 
>>> shouldn't be less than SSD page size. Possible approaches:
>>> Find it in device specification (some manufacturers don't reveal it)
>>> Try running SSD benchmarks
>>> If you are not sure, just set page size to 4K. As various benchmarks use 4K 
>>> pages, manufacturers have to adapt drives for 4K random write workload. 
>>> Whitepaper from Intel showing that 4K pages are enough: 
>>> https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ssd-server-storage-applications-paper.pdf
>>> Check your OS page cache size. Page size of Ignite shouldn't be less than 
>>> OS page size. How to check OS cache page size in Unix: 
>>> https://unix.stackexchange.com/questions/128213/how-is-page-size-determined-in-virtual-address-space
>>> 
>>> 
>>> Best Regards,
>>> Ivan Rakov
>>> 
>>> On 01.09.2017 21:08, Denis Magda wrote:
 Igniters,
 
 I see a lot of complains regarding the performance of the subj on the user 
 list. At the same time, I do believe that in most scenarios it’s a lack of 
 knowledge that we keep in secret.
 
 It's time to document Durable Memory and its Native Persistence tuning 
 parameters. Let's start doing this for Linux based deployments first. Here 
 is what we have for now (which is almost nothing): 
 https://apacheignite.readme.io/docs/durable-memory-tuning 
 
 
 Ideally, at some point we have to come up with doc like this: 
 https://access.redhat.com/sites/default/files/attachments/deploying-oracle-12c-on-rhel6_1.2_1.pdf
 
 Please share your expertise in a form of settings that have to be put on 
 the paper. We put them in JIRA and document afterwords:
 https://issues.apache.org/jira/browse/IGNITE-6246 
 
 
 —
 Denis
>> 
>

Re: Monitoring of active transactions

Hi Ilya,

I can help with including this utility into build/release, I've recenty
finished same steps for PDS WAL analysing tool for converting records to
human readable format.
Please feel free to contact me.

Sincerely,
Dmitriy Pavlov

пт, 15 сент. 2017 г. в 6:37, Dmitriy Setrakyan :

> It seems that the community (including me) really would like to see this
> feature in Ignite.
>
> Ilya, can you create a ticket and submit it for review?
>
> D.
>
> On Fri, Sep 8, 2017 at 7:15 AM, Anton Vinogradov  wrote:
>
> > Ilya,
> >
> > We extremely need this!
> >
> > Txs and Locks info should be collected on each cluster hang.
> > We already have an issue related to this problem -
> > https://issues.apache.org/jira/browse/IGNITE-4937
> >
> > Nikolay,
> >
> > Good point,
> > but, seems you should start separate thread to discuss this.
> >
> > On Fri, Sep 8, 2017 at 4:28 PM, Dmitry Pavlov 
> > wrote:
> >
> > > Hi Ilya,
> > >
> > > I'm definitely +1 for including the utility in the product. Perfect
> > > contribution.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 8 сент. 2017 г. в 14:28, Ilya Lantukh :
> > >
> > > > Igniters,
> > > >
> > > > According to our current design and implementation, unclosed
> > transaction
> > > or
> > > > unreleased lock can hang ignite cluster forever. This is logical, and
> > > with
> > > > correct usage of those mechanics such issue should never happen, in
> > real
> > > > world developers can make mistakes and leave transaction open. We
> have
> > a
> > > > feature "transaction timeout", but turns out it doesn't work in all
> > cases
> > > > (see https://issues.apache.org/jira/browse/IGNITE-6181). Even if all
> > > known
> > > > issues are fixed, there is still a lot of room for mistake and
> > incorrect
> > > > usage.
> > > >
> > > > To make it possible for Ignite users to discover such problem and
> trace
> > > it
> > > > to a particular part of code, I've created a very simple utility that
> > > > collects and prints information about long running transactions for
> the
> > > > whole cluster. It is available here:
> > > > https://github.com/ilantukh/IgniteTxViewer.
> > > >
> > > > One might expect such monitoring utilities to be included in Ignite
> > > > codebase. Personally, I think that such information should be
> available
> > > > from public API, without using of additional applications or diving
> > into
> > > > Ignite internals.
> > > >
> > > > What do you think?
> > > >
> > > > --
> > > > Best regards,
> > > > Ilya
> > > >
> > >
> >
>

Ignite Full Description on Docker Hub

Nick,

I want to fill out full description section on Docker Hub. Presently, it’s poor:
https://hub.docker.com/r/apacheignite/ignite/

Is it enough to update ignite/modules/docker/README.txt and push the changes to 
master? Guess, there are extra steps.

—
Denis

Re: Ignite Benchmarking Rules

2017-09-15 Thread Dmitriy Setrakyan

Cos,

I think Apache BigTop is using servers provided by Amazon. Can you make a
suggestion on how can Ignite community get a few servers from Amazon for
benchmarking as well?

D.

On Fri, Sep 15, 2017 at 5:57 AM, Anton Vinogradov  wrote:

> Guys,
>
> I fully agree that configured servers at Amazon is the best choice.
>
> But when you need to check that your changes has no performance drop you're
> able to use your own PC or PCs to checks that.
> All you need is to benchmark already released version vs version with your
> fix at same environment.
>
> So, seems we should have couple of configuration recommendations
> - reasonable for standalone PC
> - reasonable for cluster
>
> On Fri, Sep 15, 2017 at 12:20 PM, Nikolay Izhikov 
> wrote:
>
> > Hello, Dmitriy.
> >
> > I think experienced members of community have specific number for
> > benchmarking.
> >
> > Can we start from reference hardware configuration: Num of CPU, RAM and
> > HDD(SDD?) configuration, network configs, etc.
> >
> > Can someone share that kind of knowledge - Which hardware is best for
> > Ignite benchmarking?
> >
> > I found some numbers here - [1]. Is it well suited for Apache Ignite?
> >
> > [1] https://www.gridgain.com/resources/benchmarks/gridgain-vs-
> > hazelcast-benchmarks
> >
> > 14.09.2017 23:27, Dmitriy Setrakyan пишет:
> >
> > Alexey, I completely agree. However, for the benchmarks to be useful,
> then
> >> need to be run on the same hardware all the time. Apache Ignite does not
> >> have servers sitting around, available to run the benchmarks.
> >>
> >> Would be nice to see how other projects address it. Can Amazon donate
> >> servers for the Apache projects?
> >>
> >> D.
> >>
> >> On Thu, Sep 14, 2017 at 6:25 AM, Aleksei Zaitsev <
> ign...@alexzaitzev.pro>
> >> wrote:
> >>
> >> Hi, Igniters.
> >>>
> >>> Recently I’ve done some research in benchmarks for Ignite, and noticed
> >>> that we don’t have any rules for running benchmarks and collecting
> result
> >>> from them. Although sometimes we have tasks, which results need to be
> >>> measured. I propose to formalize such things as:
> >>>   * set of benchmarks,
> >>>   * parameters of launching them,
> >>>   * way of result collection and interpretation,
> >>>   * Ignite cluster configuration.
> >>>
> >>> I don’t think that we need to run benchmarks before every merge into
> >>> master, but in some cases it should be mandatory to compare new results
> >>> with reference values to be sure that changes do not lead to the
> >>> performance degradation.
> >>>
> >>> What do you think?
> >>>
> >>>
> >>
>

Ignite PDS WAL analysis with human readable records

Hi Igniters,

Eduard Shangareev created WAL converter utility which shows WAL
archive/work directory segments content in human readable form. This
utility outputs all WAL records to output stream (may be redirected to
file). How to use tips can be found in issue
https://issues.apache.org/jira/browse/IGNITE-6277


This utility does not require Ignite node to be up and running and may be
used for offline analysis of copied files. Thanks to Alexey G. for idea and
review. Code is now available in modules/ignite-dev-tools.

1. Could you please write if utility can be useful for you?

2. Please respond if you need short wiki description.

3. What is your opinion what should be next steps to develop this utility?

4. Does utility need property file to refer Ignite Work directory using
conf file.

5. What do you think if utility remains console non interactive, or some
UI/Interactive console shell may be preferable?

Sincerely,

Dmitriy Pavlov

HOLE query entry in CacheContinuousQueryPartitionRecovery

2017-09-15 Thread ALEKSEY KUZNETSOV

Hi, Ignters!


We have a strange field HOLE in CacheContinuousQueryPartitionRecovery

which compared to pending events in
CacheContinuousQueryPartitionRecovery#collectEntries. And it is never
equals any entry.

Do we need it ? Or it can be removed.

-- 

*Best Regards,*

*Kuznetsov Aleksey*

[GitHub] ignite pull request #2091: IGNITE-1553 Optimize transaction prepare step whe...

Github user voipp closed the pull request at:

https://github.com/apache/ignite/pull/2091


---

[GitHub] ignite pull request #2332: Ignite 1094

Github user voipp closed the pull request at:

https://github.com/apache/ignite/pull/2332


---

[GitHub] ignite pull request #1557: IGNITE-2845 Get operation might ignore entry upda...

Github user voipp closed the pull request at:

https://github.com/apache/ignite/pull/1557


---

[GitHub] ignite pull request #2126: IGNITE-1094 recovery resolved

Github user voipp closed the pull request at:

https://github.com/apache/ignite/pull/2126


---

Re: Unintuitive error message when invalid marshaller files found

2017-09-15 Thread Valentin Kulichenko

Mike,

Can you show the exception that is thrown?

-Val

On Fri, Sep 15, 2017 at 7:12 AM, Michael Griggs 
wrote:

> This afternoon I came across an unusual case where there were files in
> my work/marshaller folder with invalid filenames.  It seems that the
> valid format is -[0-9]+.classname[0-9].  However, I had files that
> were in the format -[0-9]+.classname - i.e., no trailing zero.  Where
> these files came from I'm not sure, perhaps a significantly older
> version of Ignite?
>
> The error message could be improved, and unless there is an
> outstanding JIRA I will open one to
>
> 1. Print the full file path, not just the filename - this will help in
> determining where the work/marshaller folder is located
> 2. Suggesting to clear out the contents of the work/marshaller folder
> and restart
>
> Alternatively, can we just ignore files that do not end in [0-9] ?
>
> Regards
> Mike
>
>
>

[GitHub] ignite pull request #2677: ignite-5960 Sending backupQueue to node.

GitHub user voipp opened a pull request:

https://github.com/apache/ignite/pull/2677

ignite-5960  Sending backupQueue to node.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/voipp/ignite ignite-5960-3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2677.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2677


commit c42adb5c3e8c46fdec38ac0e6a3cbe3ad2db5520
Author: voipp 
Date:   2017-09-15T16:54:08Z

ignite-5960  Sending backupQueue to node.




---

[jira] [Created] (IGNITE-6405) Deadlock is not detected if timed out on client.

2017-09-15 Thread Alexei Scherbakov (JIRA)

Alexei Scherbakov created IGNITE-6405:
-

 Summary: Deadlock is not detected if timed out on client.
 Key: IGNITE-6405
 URL: https://issues.apache.org/jira/browse/IGNITE-6405
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.1
Reporter: Alexei Scherbakov
Priority: Minor
 Fix For: 2.3


Timeout exception is thrown instead.

Reproducer:

{noformat}
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *  http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.ignite.internal.processors.cache.transactions;

import java.util.Collections;
import java.util.concurrent.CountDownLatch;
import javax.cache.CacheException;
import org.apache.ignite.Ignite;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.configuration.TransactionConfiguration;
import org.apache.ignite.internal.IgniteInternalFuture;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
import org.apache.ignite.transactions.Transaction;
import org.apache.ignite.transactions.TransactionDeadlockException;

import static org.apache.ignite.cache.CacheAtomicityMode.TRANSACTIONAL;
import static org.apache.ignite.transactions.TransactionConcurrency.PESSIMISTIC;
import static 
org.apache.ignite.transactions.TransactionIsolation.REPEATABLE_READ;

/**
 * Tests an ability to eagerly rollback timed out transactions.
 */
public class TxPessimisticDeadlockDetectionClient extends 
GridCommonAbstractTest {
/** */
private static final long TX_MIN_TIMEOUT = 1;

/** */
private static final long TX_TIMEOUT = 300;

/** */
private static final long TX_DEFAULT_TIMEOUT = 3_000;

/** */
private static final String CACHE_NAME = "test";

/** IP finder. */
private static final TcpDiscoveryVmIpFinder IP_FINDER = new 
TcpDiscoveryVmIpFinder(true);

/** */
private static final int GRID_CNT = 3;

/** */
private final CountDownLatch blocked = new CountDownLatch(1);

/** */
private final CountDownLatch unblocked = new CountDownLatch(1);

/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String 
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);

cfg.setClientMode("client".equals(igniteInstanceName));

((TcpDiscoverySpi)cfg.getDiscoverySpi()).setIpFinder(IP_FINDER);

TransactionConfiguration txCfg = new TransactionConfiguration();
txCfg.setDefaultTxTimeout(TX_DEFAULT_TIMEOUT);

cfg.setTransactionConfiguration(txCfg);

CacheConfiguration ccfg = new CacheConfiguration(CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
ccfg.setBackups(2);

cfg.setCacheConfiguration(ccfg);

return cfg;
}

/** {@inheritDoc} */
@Override protected void beforeTest() throws Exception {
super.beforeTest();

startGridsMultiThreaded(GRID_CNT);
}

/** {@inheritDoc} */
@Override protected void afterTest() throws Exception {
super.afterTest();

stopAllGrids();
}

/** */
protected void validateException(Exception e) {
assertEquals("Deadlock report is expected",
TransactionDeadlockException.class, 
e.getCause().getCause().getClass());
}

/**
 * Tests if deadlock is resolved on timeout with correct message.
 *
 * @throws Exception If failed.
 */
public void testDeadlockUnblockedOnTimeout() throws Exception {
Ignite client = startGrid("client");

testDeadlockUnblockedOnTimeout0(client, ignite(0));
}

/**
 * Tests if deadlock is resolved on timeout with correct message.
 * @throws Exception
 */
private void testDeadlockUnblockedOnTimeout0(final Ignite node1, final 
Ignite node2) throws Exception {
final CountDownLatch l = new CountDownLatch(2);

[GitHub] ignite pull request #2676: Ignite-gg-12717

2017-09-15 Thread AMashenkov

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/2676

Ignite-gg-12717

fix test

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-12717

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2676.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2676


commit 1093819ac0f3e7a0faacde59919117b8977e6d5b
Author: Igor Sapego 
Date:   2016-11-09T15:19:01Z

IGNITE-4201: Fixed version fix maven step.

commit bac0cba7fddd412dfbff98163afbc15d81d5e0d4
Author: Dmitriy Govorukhin 
Date:   2016-11-10T06:02:41Z

ignite-4044  always authenticate local node

commit 26daa57ca82254d68ac04a7b33223c6eb5ade0e4
Author: sboikov 
Date:   2016-11-10T08:17:29Z

Fixed javadoc.

commit 8b59f4e76138e08e80aa219c1a9cf0c3df6fdb4b
Author: Andrey V. Mashenkov 
Date:   2016-11-10T11:43:00Z

Backport commit of the following:

commit 612eb3daffe608995aac28eed019b3e6ef9d66d3
Author: Aleksei Scherbakov 
Date:   Fri Aug 19 13:28:39 2016 +0300

ignite-2795 Support 'copyOnRead' for SQL queries

commit b7499828c928e02e8e554f960f3754e4d08bfbe0
Author: Anton Vinogradov 
Date:   2016-11-10T13:10:21Z

IGNITE-500 CacheLoadingConcurrentGridStartSelfTest fails (DataStreamer data 
loss at unstable topology in !allowOverwrite mode fixed)

commit baa752660c6eddf27d15a812252b01b5872385de
Author: iveselovskiy 
Date:   2016-11-10T15:47:09Z

IGNITE-4208: Hadoop: Fixed a bug preventing normal secondary file system 
start. This closes #1228.

commit ef9d6cf9e334c35b03dfa42e4ce0680c85a693a4
Author: iveselovskiy 
Date:   2016-11-10T15:47:09Z

IGNITE-4208: Hadoop: Fixed a bug preventing normal secondary file system 
start. This closes #1228.

commit 884b281218d88c028daab25d35c14ee2b41be36e
Author: Pavel Tupitsyn 
Date:   2016-11-10T17:35:15Z

IGNITE-4186 .NET: Fix "Invalid session release request" exception in 
IgniteSessionStateStoreProvider.SetAndReleaseItemExclusive

This closes #1227

commit eac5f0c05b85b620f979bd73d5043ae521b6f2f4
Author: Igor Sapego 
Date:   2016-11-10T18:13:13Z

IGNITE-4100: Improved README and DEVNOTES for CPP.

commit e159fdee0152a6df1bbf4dafc3d18a284e86a57e
Author: Alexey Kuznetsov 
Date:   2016-11-11T02:39:32Z

Fixed classnames.properties generation for ignite-hadoop module.

commit 5a4ebd5de8751dcf32a26c96bf4f39e43bcbb341
Author: Alexey Kuznetsov 
Date:   2016-11-11T02:49:41Z

Fixed classnames.properties generation for ignite-hadoop module.

commit 73a8fa8b635cce3b9d8dcad364a32d29f12d4398
Author: Alexey Kuznetsov 
Date:   2016-11-11T03:20:32Z

Fixed classnames.properties generation for ignite-hadoop module.

commit 53876d3f1b3e561702fc0cd8c6ddd078d6c7dce6
Author: Alexey Kuznetsov 
Date:   2016-11-11T03:26:09Z

Fixed classnames.properties generation for ignite-hadoop module.

commit f8aa957327312d76f90231b9bfe6d386d1d4ec37
Author: Alexey Kuznetsov 
Date:   2016-11-11T08:56:42Z

Reverted wrong commit.

commit c6921a311f437504a45a4667ddde85b14269ba57
Author: Alexey Kuznetsov 
Date:   2016-11-11T09:01:33Z

Fixed classnames.properties generation for ignite-hadoop module.

commit d69e26dd8d4fd9383a149c93c251911a8dd67528
Author: Pavel Tupitsyn 
Date:   2016-11-11T09:22:55Z

IGNITE-4126 .NET: Add IgniteConfiguration.SwapSpaceSpi

commit a70f0bac3ac2487b8ab58598ad921daa952b485f
Author: Andrey V. Mashenkov 
Date:   2016-11-11T10:03:40Z

IGNITE-4145: Fixes "No query result found for request" exception when 
running multiple queries concurrently. This closes #1218.

commit 8bb8bdda2e846dcc92a2fd449e64d7594b2700ed
Author: tledkov-gridgain 
Date:   2016-11-11T12:01:14Z

IGNITE-4053: Moved task error output from console to logger. This closes 
#1160.

commit 7128a395085b60e86436f807b4bdbca83627d41a
Author: sboikov 
Date:   2016-11-11T12:29:38Z

ignite-4154 Optimize amount of data stored in discovery history
Discovery history optimizations:
- remove discarded message for discovery pending messages
- remove duplicated data from TcpDiscoveryNodeAddedMessage.oldNodesDiscoData
- do not store unnecessary data in discovery EnsuredMessageHistory
- use special property for EnsuredMessageHistory size instead of 
IGNITE_DISCOVERY_HISTORY_SIZE
Affinity history

[jira] [Created] (IGNITE-6404) Failures in .NET test suite

Vladimir Ozerov created IGNITE-6404:
---

 Summary: Failures in .NET test suite
 Key: IGNITE-6404
 URL: https://issues.apache.org/jira/browse/IGNITE-6404
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Reporter: Vladimir Ozerov
Assignee: Pavel Tupitsyn
 Fix For: 2.3


A kind of NPE. Affected tests:
{code}
ConsoleRedirectTest.TestStartupJavaError
ExceptionsTest.TestExceptions   
ExceptionsTest.TestInvalidSpringUrl 
LifecycleTest.TestError 
MarshallerTest.TestInvalidMarshaller 
CustomLoggerTest.TestStartupDotNetError 
CustomLoggerTest.TestStartupJavaError 
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6403) IgniteSqlNotNullConstraintTest.testTransactionalAddColumnNotNullCheckDmlInsertValues fails periodically

Vladimir Ozerov created IGNITE-6403:
---

 Summary: 
IgniteSqlNotNullConstraintTest.testTransactionalAddColumnNotNullCheckDmlInsertValues
 fails periodically
 Key: IGNITE-6403
 URL: https://issues.apache.org/jira/browse/IGNITE-6403
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Vladimir Ozerov
Assignee: Alexander Paschenko
 Fix For: 2.3


Looks like we have a bug in {{ALTER TABLE}} - column is not visible after 
command is completed:
{code}
Caused by: org.h2.jdbc.JdbcSQLException: Column "NAME" not found; SQL statement:
INSERT INTO test(id, name, age) VALUES (1, 'ok', 1), (2, NULLIF('a', 'a'), 2), 
(3, 'ok', 3) [42122-195]
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6402) Make yardstick delete config-db from and not copy logs to remotes

2017-09-15 Thread Ksenia Rybakova (JIRA)

Ksenia Rybakova created IGNITE-6402:
---

 Summary: Make yardstick delete config-db from and not copy logs to 
remotes
 Key: IGNITE-6402
 URL: https://issues.apache.org/jira/browse/IGNITE-6402
 Project: Ignite
  Issue Type: Improvement
  Components: yardstick
Affects Versions: 2.1
Reporter: Ksenia Rybakova


1) Currently if we run yardstick from folder that contains output/logs-, 
this logs- will be also copied to remotes during yardstick deployment. This 
is not needed.

2) During clean up process folder config-db is not removed from remotes along 
with others (config, bin, etc.)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6401) Change size to maxSize for memory config in ignite-base-load-config.xml

2017-09-15 Thread Ksenia Rybakova (JIRA)

Ksenia Rybakova created IGNITE-6401:
---

 Summary: Change size to maxSize for memory config in 
ignite-base-load-config.xml 
 Key: IGNITE-6401
 URL: https://issues.apache.org/jira/browse/IGNITE-6401
 Project: Ignite
  Issue Type: Bug
  Components: yardstick
Affects Versions: 2.1
Reporter: Ksenia Rybakova


Typo in ignite-base-load-config.xml: 'size' need to be changed to 'maxSize' for 
memory config since there is no 'size' property.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2603: IGNITE-6256: DiscoCache should always contains lo...

2017-09-15 Thread AMashenkov

Github user AMashenkov closed the pull request at:

https://github.com/apache/ignite/pull/2603


---

[GitHub] ignite pull request #2675: Ignite-gg-12692

2017-09-15 Thread AMashenkov

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/2675

Ignite-gg-12692

for tests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-12692

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2675.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2675


commit cf75471a8daa69aa31c4029a728d23cca3cb7789
Author: dkarachentsev 
Date:   2017-04-18T09:31:10Z

Merge branch 'ignite-1.8.5-p1' into ignite-1.9.2

# Conflicts:
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/sql/GridSqlOperationType.java
#   
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/IgniteSqlSplitterSelfTest.java

commit 320def0b0cc357cfae4423352623ead7aa1d5984
Author: Alexander Belyak 
Date:   2017-04-18T11:56:50Z

IGNITE-4927: Write behind - add an option to skip write coalescing

commit c48322b79cafb561211501813c805aebd4b85540
Author: Valentin Kulichenko 
Date:   2017-04-13T08:29:30Z

IGNITE-4954 - Configurable expiration timeout for Cassandra session. This 
closes #1785.

commit 1c1a2f3e86c091bbbd2668db1cc7d72ff13d91ec
Author: dkarachentsev 
Date:   2017-04-19T10:27:49Z

Fix merge ignite-1.8.5-p1

commit ea3e625fe1269383d46d0be55adc85662a216792
Author: Alexander Belyak 
Date:   2017-04-19T10:29:01Z

IGNITE-4927 Use write behind coalescing from CacheConfiguration in 
GridCacheStoreManagerAdapter to configure actual cache store.

Signed-off-by: nikolay_tikhonov 

commit c8155fd8c2b12ca69fd5877fd21327ec746e1225
Author: Dmitriy Govorukhin 
Date:   2017-04-21T13:23:00Z

ignite-1.9.2 backport of ignite-4949

commit 8cae6dfd7ed000d06b37ad53f5d04aa3fd534d66
Author: Alexander Belyak 
Date:   2017-04-21T13:53:05Z

IGNITE-4927 Use write behind coalescing from CacheConfiguration in 
GridCacheStoreManagerAdapter to configure actual cache store.

Signed-off-by: nikolay_tikhonov 

commit 0d4d3cc36b17b9dcda8be02f9931ba01213f007f
Author: Andrey V. Mashenkov 
Date:   2017-04-21T16:40:08Z

Merge branch 'ignite-1.7.10' into ignite-1.9.2

# Conflicts:
#   
modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/datasource/DataSource.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/expiry/IgniteCacheExpiryPolicyAbstractTest.java

commit da3a1bd399832a8c012a5bd5ce1f07fe178cb8a6
Author: Andrey V. Mashenkov 
Date:   2017-04-21T16:41:16Z

Merge remote-tracking branch 'origin/ignite-1.9.2' into ignite-1.9.2

commit fde6d4427131f4b74fcdf88f1caae4f57d225a87
Author: Pavel Tupitsyn 
Date:   2017-04-20T16:39:45Z

IGNITE-5043 .NET: CacheConfiguration.WriteBehindCoalescing

(cherry picked from commit 217c6be)

commit 474add7c79eb5c57b2f88ca71ceb58ed3e5cb20b
Author: dkarachentsev 
Date:   2017-04-24T11:52:07Z

IGNITE-5043 .NET: CacheConfiguration.WriteBehindCoalescing

Fix merge.

commit a4c70f12305c15cbf15a6e5eb7c500f883a8b317
Author: agura 
Date:   2017-04-20T17:45:58Z

ignite-5041 NPE in deadlock detection fixed

commit a1fec4b0b3ddcc208154114e0bf8f202eb597fc9
Author: dkarachentsev 
Date:   2017-04-25T06:14:07Z

Fix index initialization.

commit d94172e7bb7ff4ebab422997e5c70f8cd03c0ae4
Author: agura 
Date:   2017-04-26T12:16:46Z

Redundant test removed

commit c64dc2426a70896116dbdffbf90faea5e52b991c
Author: Andrey V. Mashenkov 
Date:   2017-04-27T13:35:25Z

Revert IGNITE-4863 partially. Warn user if default log level was changed. 
As default log level value (DEBUG) can cause performance issues.

commit f9ecacc625b458539775e6550bd9b7613ed38f21
Author: dkarachentsev 
Date:   2017-04-28T08:46:23Z

IGNITE-5077 - Support service security permissions

backport from master
(cherry picked from commit 6236b5f)

commit 91c899b909383c78b78b9bf0c8f233b8c75ef29e
Author: Valentin Kulichenko 
Date:   2017-04-28T12:48:57Z

IGNITE-5081 - Removed redundant duplication of permissions in 
SecurityPermissionSetBuilder

commit b48a26b9b1e97fb8eb52c2a2f36005770922ac3d
Author: Valentin Kulichenko 
Date:   2017-04-28T12:53:33Z

IGNITE-5080 - Fixes in SecurityBasicPermissionSet

commit f66c23cbb9a6f2c923ebf75c58f00afaf1c0b5f3

[GitHub] ignite pull request #2674: Ignite 5136 GridLogThrottle memory leak

2017-09-15 Thread vadopolski

GitHub user vadopolski opened a pull request:

https://github.com/apache/ignite/pull/2674

Ignite 5136 GridLogThrottle memory leak



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vadopolski/ignite ignite-5136-li

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2674.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2674


commit 63bfe6a8681ea7a9f40de9d17f44df0c66c678ec
Author: vadopolski 
Date:   2017-09-05T17:29:58Z

Added test and cleanUpOldEntries

commit 30b132e7e2a62ae1d0605db1cdd107eb9ca1ec37
Author: vadopolski 
Date:   2017-09-05T18:19:27Z

Added check of timeStamp in map.

commit 09d0fed6fdc8ca910a5e1b67bbe4d3c1911a37d3
Author: vadopolski 
Date:   2017-09-05T19:28:59Z

Added testOnChangingTimeout

commit a13a6e78e9ecf1507cf2e82a0859fa10bf02babc
Author: vadopolski 
Date:   2017-09-06T14:12:04Z

Added lazy initialization.

commit eb75fe7d7219b3d1f9a2ba64d4a6fd532aee742d
Author: vadopolski 
Date:   2017-09-06T14:19:34Z

Added throttleTimeout.

commit 795593570e78623fea8c0d16f7c19423d6b83654
Author: vadopolski 
Date:   2017-09-06T14:23:42Z

Changed sleep timeout.

commit 18b8c75df9673d7c1223d66f74da68d436a45eee
Author: vadopolski 
Date:   2017-09-06T15:07:41Z

Added removing from errors with iterator.

commit 184f092b3bc0fd604480ae2cbd16f0fa8a7fed6e
Author: vadopolski 
Date:   2017-09-06T15:41:39Z

Code style.n

commit 881b78fd69ad1939bb580c49880623eb3ed74cca
Author: vadopolski 
Date:   2017-09-07T13:40:07Z

Improved iterator.

commit 90e9665f6371abbc3180c55ab75c714a38c1ed22
Author: vadopolski 
Date:   2017-09-07T16:24:34Z

Changed sleep timeout.

commit 6d2e0158bb5c0d8fc0c602cd0a8c874f0ed685e7
Author: vadopolski 
Date:   2017-09-07T16:37:13Z

Refactoring.

commit 9bfa9c3b60b53b1f444a6721f01dd84dc1f028f4
Author: vadopolski 
Date:   2017-09-07T16:47:31Z

Refactoring.

commit 0769ce86fdf5d73658548f31af21f9782d210f70
Author: vadopolski 
Date:   2017-09-07T16:50:36Z

Deleted loggedTs.

commit df05944a66448b4fcd3f635f8e348827096b5f15
Author: vadopolski 
Date:   2017-09-11T16:24:33Z

Review fixes

commit ec037cab659107d3f9959fe1b2540028f95d9345
Author: vadopolski 
Date:   2017-09-13T13:24:31Z

Added synchronized.

commit ec7d65527199039f84fcd4f0aa30cb1758c0e2e9
Author: vadopolski 
Date:   2017-09-13T17:46:39Z

Added cancel to synchronized area.

commit 2f8ba6b1864e479e934033fa7c49b44e1e57156e
Author: vadopolski 
Date:   2017-09-15T10:28:14Z

Added cancel to synchronized.

commit fabbc8bc22237f63e6a93851bdbbec7ee8bf5ec2
Author: vadopolski 
Date:   2017-09-15T13:31:46Z

Added cleanUpCancelEnabled to mapCleaningPeriodSetup.

commit ac850a5f6bf0cf44ea3afcd895a9ec95398834ca
Author: vadopolski 
Date:   2017-09-15T13:36:51Z

Refactoring.

commit 45c0eb6159bd51b23ed35ae7f49b6502c8c96ba8
Author: vadopolski 
Date:   2017-09-15T13:46:12Z

Overwritten mapCleaningPeriodSetup.

commit b2ec2a25de6fe484ac3dcfcd2b90256f2cf416bd
Author: vadopolski 
Date:   2017-09-15T14:11:31Z

Refactoring.




---

[GitHub] ignite pull request #2673: IGNITE-6395: remove old test

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2673


---

[GitHub] ignite pull request #2671: IGNITE-6394: Can't create off-heap matrix with si...

2017-09-15 Thread ybabak

Github user ybabak closed the pull request at:

https://github.com/apache/ignite/pull/2671


---

Unintuitive error message when invalid marshaller files found

2017-09-15 Thread Michael Griggs

This afternoon I came across an unusual case where there were files in
my work/marshaller folder with invalid filenames.  It seems that the
valid format is -[0-9]+.classname[0-9].  However, I had files that
were in the format -[0-9]+.classname - i.e., no trailing zero.  Where
these files came from I'm not sure, perhaps a significantly older
version of Ignite?

The error message could be improved, and unless there is an
outstanding JIRA I will open one to 

1. Print the full file path, not just the filename - this will help in
determining where the work/marshaller folder is located
2. Suggesting to clear out the contents of the work/marshaller folder
and restart

Alternatively, can we just ignore files that do not end in [0-9] ?

Regards
Mike

[GitHub] ignite pull request #2673: IGNITE-6395: remove old test

2017-09-15 Thread tledkov-gridgain

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/2673

IGNITE-6395: remove old test



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6395

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2673.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2673


commit 0d466f4ff8c5af18b6bb6fde9c0b3b148523bf5d
Author: tledkov-gridgain 
Date:   2017-09-15T14:03:50Z

IGNITE-6395: remove old test




---

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

2017-09-15 Thread Alexey Kukushkin

Ray,

Yakov just pointed me to this thread

where
user had a problem with IP address changing between the cluster restarts.
The persistent data directories inside work/db have names built as a
concatenation of local IP addresses and ports. Thus, if your node receives
a different IP address between the restarts, you will not have your
persistent data since the node will initialise from a different persistent
data file. You would need to bind your discovery SPI to specific address to
work around that. There is an issue opened to properly fix it.

Please also consider that as a potential cause.

[jira] [Created] (IGNITE-6399) .NET: support ClientConnectorConfiguration

Vladimir Ozerov created IGNITE-6399:
---

 Summary: .NET: support ClientConnectorConfiguration
 Key: IGNITE-6399
 URL: https://issues.apache.org/jira/browse/IGNITE-6399
 Project: Ignite
  Issue Type: Task
  Components: clients, platforms
Reporter: Vladimir Ozerov
Assignee: Pavel Tupitsyn
 Fix For: 2.3


See {{IgniteConfiguration.clientConnectorConfiguration}}. We will use it 
instead of {{IgniteConfiguration.sqlConnectorConfiguration}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6400) Document ClientConnectorConfiguration

Vladimir Ozerov created IGNITE-6400:
---

 Summary: Document ClientConnectorConfiguration
 Key: IGNITE-6400
 URL: https://issues.apache.org/jira/browse/IGNITE-6400
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: 2.3






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6398) WebConsole: support ClientConnectorConfiguration

Vladimir Ozerov created IGNITE-6398:
---

 Summary: WebConsole: support ClientConnectorConfiguration
 Key: IGNITE-6398
 URL: https://issues.apache.org/jira/browse/IGNITE-6398
 Project: Ignite
  Issue Type: Task
  Components: clients, wizards
Reporter: Vladimir Ozerov
Assignee: Alexey Kuznetsov
 Fix For: 2.3


See {{IgniteConfiguration.clientConnectorConfiguration}}. We will use it 
instead of {{IgniteConfiguration.sqlConnectorConfiguration}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2672: IGNITE-6327

2017-09-15 Thread devozerov

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/2672

IGNITE-6327



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6327

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2672.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2672


commit 7b2d715a7083327db0d1decc6c83393ae7cbfa7f
Author: devozerov 
Date:   2017-09-15T12:45:47Z

Copy-pasted class.

commit 310cf50a65d29d9d2f4d8164058b1ec2fe0aee21
Author: devozerov 
Date:   2017-09-15T13:06:53Z

IgniteConfiguration.

commit b67793b9e233b4815fcdbb9c807ffcbacf19ec95
Author: devozerov 
Date:   2017-09-15T13:08:22Z

Better deprecations.

commit 348b582cb0ae856e72c82d239c453e8e5b1502bf
Author: devozerov 
Date:   2017-09-15T13:10:12Z

WIP.

commit 6277ab7662df9f940e15e7893a2dde81fa4c68fe
Author: devozerov 
Date:   2017-09-15T13:32:48Z

Naming and conversion.

commit eaad0658199677087cdcd6237be9aff0de3d9ec9
Author: devozerov 
Date:   2017-09-15T13:46:06Z

WIP.

commit b389998a603b59d91911a26429af64f32ec28603
Author: devozerov 
Date:   2017-09-15T13:50:25Z

Fixing usages.




---

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

2017-09-15 Thread Yakov Zhdanov

Guys, it seems that we have the same issue that was described in thread -
Specifying location of persistent storage location

I filed a ticket some time ago -
https://issues.apache.org/jira/browse/IGNITE-6285, Alex G, can you take a
look if this ticket makes sense?

--Yakov

Re: Ignite Benchmarking Rules

2017-09-15 Thread Anton Vinogradov

Guys,

I fully agree that configured servers at Amazon is the best choice.

But when you need to check that your changes has no performance drop you're
able to use your own PC or PCs to checks that.
All you need is to benchmark already released version vs version with your
fix at same environment.

So, seems we should have couple of configuration recommendations
- reasonable for standalone PC
- reasonable for cluster

On Fri, Sep 15, 2017 at 12:20 PM, Nikolay Izhikov 
wrote:

> Hello, Dmitriy.
>
> I think experienced members of community have specific number for
> benchmarking.
>
> Can we start from reference hardware configuration: Num of CPU, RAM and
> HDD(SDD?) configuration, network configs, etc.
>
> Can someone share that kind of knowledge - Which hardware is best for
> Ignite benchmarking?
>
> I found some numbers here - [1]. Is it well suited for Apache Ignite?
>
> [1] https://www.gridgain.com/resources/benchmarks/gridgain-vs-
> hazelcast-benchmarks
>
> 14.09.2017 23:27, Dmitriy Setrakyan пишет:
>
> Alexey, I completely agree. However, for the benchmarks to be useful, then
>> need to be run on the same hardware all the time. Apache Ignite does not
>> have servers sitting around, available to run the benchmarks.
>>
>> Would be nice to see how other projects address it. Can Amazon donate
>> servers for the Apache projects?
>>
>> D.
>>
>> On Thu, Sep 14, 2017 at 6:25 AM, Aleksei Zaitsev 
>> wrote:
>>
>> Hi, Igniters.
>>>
>>> Recently I’ve done some research in benchmarks for Ignite, and noticed
>>> that we don’t have any rules for running benchmarks and collecting result
>>> from them. Although sometimes we have tasks, which results need to be
>>> measured. I propose to formalize such things as:
>>>   * set of benchmarks,
>>>   * parameters of launching them,
>>>   * way of result collection and interpretation,
>>>   * Ignite cluster configuration.
>>>
>>> I don’t think that we need to run benchmarks before every merge into
>>> master, but in some cases it should be mandatory to compare new results
>>> with reference values to be sure that changes do not lead to the
>>> performance degradation.
>>>
>>> What do you think?
>>>
>>>
>>

[GitHub] ignite pull request #2542: IGNITE-5905 .NET: Thin client: cache.Get for prim...

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2542


---

[GitHub] ignite pull request #2625: IGNITE-6244 .NET: Thin client: ScanQuery

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2625


---

[GitHub] ignite pull request #2580: IGNITE-6236 .NET: Thin client: cache.Get and Put ...

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2580


---

[jira] [Created] (IGNITE-6397) .NET thin client: implement base cache operations

Vladimir Ozerov created IGNITE-6397:
---

 Summary: .NET thin client: implement base cache operations
 Key: IGNITE-6397
 URL: https://issues.apache.org/jira/browse/IGNITE-6397
 Project: Ignite
  Issue Type: Bug
  Components: clients, platforms
Reporter: Vladimir Ozerov
Assignee: Pavel Tupitsyn
 Fix For: 2.3


We need to implement base cache operations, such as "remove", "replace", 
"putIfAbsent". 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2562: IGNITE-6225: Improve tests of WAL iterator with c...

2017-09-15 Thread dspavlov

Github user dspavlov closed the pull request at:

https://github.com/apache/ignite/pull/2562


---

[GitHub] ignite pull request #2671: IGNITE-6394: Can't create off-heap matrix with si...

2017-09-15 Thread ybabak

GitHub user ybabak opened a pull request:

https://github.com/apache/ignite/pull/2671

IGNITE-6394: Can't create off-heap matrix with size over 2 Gb

Fixed int overflow.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6394

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2671.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2671


commit c0b5f9f7a83f110cf2036a3e654690929797d199
Author: Yury Babak 
Date:   2017-09-15T11:40:54Z

IGNITE-6394: Can't create off-heap matrix with size over 2 Gb (int overflow 
in DenseOffHeapMatrixStorage.allocateMemory)
Fixed.




---

[GitHub] ignite pull request #2620: Ignite 6235: Add ability to handle CacheObject fr...

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2620


---

[jira] [Created] (IGNITE-6396) SQL: Support SQLSTATE error code for "NOT NULL" constraint

Vladimir Ozerov created IGNITE-6396:
---

 Summary: SQL: Support SQLSTATE error code for "NOT NULL" constraint
 Key: IGNITE-6396
 URL: https://issues.apache.org/jira/browse/IGNITE-6396
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.3
Reporter: Vladimir Ozerov
Assignee: Sergey Kalashnikov
 Fix For: 2.3


Respective SQLSTATE: 20004.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6395) Uncomment JdbcThinStatementSelfTest.testCloseOnCompletion

Vladimir Ozerov created IGNITE-6395:
---

 Summary: Uncomment JdbcThinStatementSelfTest.testCloseOnCompletion
 Key: IGNITE-6395
 URL: https://issues.apache.org/jira/browse/IGNITE-6395
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.3
Reporter: Vladimir Ozerov
Assignee: Taras Ledkov
 Fix For: 2.3






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6394) Can't create off-heap matrix with size over 2 Gb (int overflow in DenseOffHeapMatrixStorage.allocateMemory)

2017-09-15 Thread Oleg Ignatenko (JIRA)

Oleg Ignatenko created IGNITE-6394:
--

 Summary: Can't create off-heap matrix with size over 2 Gb (int 
overflow in DenseOffHeapMatrixStorage.allocateMemory)
 Key: IGNITE-6394
 URL: https://issues.apache.org/jira/browse/IGNITE-6394
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Oleg Ignatenko
Assignee: Yury Babak


In order to reproduce, try to create off-heap matrix with size larger than 2Gb, 
eg like this:

{code}new DenseLocalOffHeapMatrix(17_000, 17_000){code}

Above fails:
{noformat}
java.lang.IllegalArgumentException: null
at sun.misc.Unsafe.allocateMemory(Native Method)
at 
org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1068)
at 
org.apache.ignite.ml.math.impls.storage.matrix.DenseOffHeapMatrixStorage.allocateMemory(DenseOffHeapMatrixStorage.java:219)
...
{noformat}

(you may need to allow Java get more than 2Gb off-heap memory for that with VM 
argument like {{-XX:MaxDirectMemorySize=1000g}})

The reason for failure is that int arguments aren't converted to long prior to 
multiplying in {{DenseOffHeapMatrixStorage.allocateMemory}}:

{code}ptr = GridUnsafe.allocateMemory((long)rows * cols * Double.BYTES);{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6393) ODBC: need to cleanup server cursors on client disconnect

Vladimir Ozerov created IGNITE-6393:
---

 Summary: ODBC: need to cleanup server cursors on client disconnect
 Key: IGNITE-6393
 URL: https://issues.apache.org/jira/browse/IGNITE-6393
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
Reporter: Vladimir Ozerov
Assignee: Igor Sapego
 Fix For: 2.3






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6392) JDBC thin driver: need to cleanup server cursors on client disconnect

Vladimir Ozerov created IGNITE-6392:
---

 Summary: JDBC thin driver: need to cleanup server cursors on 
client disconnect
 Key: IGNITE-6392
 URL: https://issues.apache.org/jira/browse/IGNITE-6392
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.3
Reporter: Vladimir Ozerov
Assignee: Taras Ledkov
 Fix For: 2.3






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2670: IGNITE-6368 .NET: Separate interfaces for Ignite ...

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2670


---

[GitHub] ignite pull request #2496: IGNITE-6137: CAS logic fix in async commit.

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2496


---

[GitHub] ignite pull request #2605: Ignite 6277

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2605


---

[jira] [Created] (IGNITE-6390) Implement usable component cluster-selector on dependence pages

2017-09-15 Thread Dmitriy Shabalin (JIRA)

Dmitriy Shabalin created IGNITE-6390:


 Summary: Implement usable component cluster-selector on dependence 
pages
 Key: IGNITE-6390
 URL: https://issues.apache.org/jira/browse/IGNITE-6390
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Dmitriy Shabalin
Assignee: Dmitriy Shabalin
 Fix For: 2.3


1. add cluster-selector to pages.
2. add update information of cluster data in page after change cluster.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2670: IGNITE-6368 .NET: Separate interfaces for Ignite ...

2017-09-15 Thread ptupitsyn

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/2670

IGNITE-6368 .NET: Separate interfaces for Ignite and Cache facades for thin 
client



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6368

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2670.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2670


commit 5e1d5b1157a880f7d53fc94c7a706556aad170d9
Author: Pavel Tupitsyn 
Date:   2017-09-15T09:30:35Z

IGNITE-6368 .NET: Separate interfaces for Ignite and Cache facades for thin 
client




---

Re: Expiry policy for Cache.invoke

2017-09-15 Thread Denis Mekhanikov

I filed a ticket for this issue:
https://issues.apache.org/jira/browse/IGNITE-6388

Also I stumbled upon another bug, trying to reproduce the previous one.
Here is a ticket: https://issues.apache.org/jira/browse/IGNITE-6389

Denis

пт, 15 сент. 2017 г. в 3:07, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Denis,
>
> This looks like a bug. In my understanding, getExpiryForAccess should be
> called for any update, plus one of getExpiryForCreation/getExpiryForUpdate
> depending on whether it's a new entry or not. And this definitely should
> not depend on cache mode.
>
> -Val
>
> On Thu, Sep 14, 2017 at 3:02 AM, Denis Mekhanikov 
> wrote:
>
> > Dmitriy,
> >
> > You are right about transactions, but the spec describes invoke, so, if
> it
> > specifies some behavior in general, then it should be followed in both
> > cases.
> >
> > Here is the most relevant part I could find in the spec:
> > https://static.javadoc.io/javax.cache/cache-api/1.0.0/
> > javax/cache/processor/EntryProcessor.html
> > I think, that if the value is loaded from CacheStore, then
> > getExpiryForCreation() should be used. Other methods should be called
> > depending on operations performed.
> >
> > Denis
> >
> > чт, 14 сент. 2017 г. в 12:02, Denis Mekhanikov :
> >
> > > Val,
> > >
> > > Sorry, I may be didn't formulate the issue clearly. Other than
> predefined
> > > expiry policies (like CreatedExpiryPolicy, AccessedExpiryPolicy, etc)
> you
> > > can provide a custom expiry policy by calling
> > > setExpiryPolicyFactory(Factory)
> > >  > apache/ignite/configuration/CacheConfiguration.html#
> > setExpiryPolicyFactory(javax.cache.configuration.Factory)>.
> > > So, cache will consult the configured ExpiryPolicy by calling
> > > getExpiryForCreation(), getExpiryForAccess() or getExpiryForUpdate(),
> > > depending on the performed operation.
> > >
> > > So, the methods of ExpiryPolicy that are called when invoke(...)
> > >  > apache/ignite/IgniteCache.html#invoke(K,%20org.apache.ignite.cache.
> > CacheEntryProcessor,%20java.lang.Object...)> is
> > > used, somehow depend on the configured atomicity. Of course, the
> > configured
> > > ExpiryPolicy is used, but in some cases the wrong method is called.
> > >
> > > Denis
> > >
> > > чт, 14 сент. 2017 г. в 1:54, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com>:
> > >
> > >> Denis,
> > >>
> > >> I'm confused by the issue. Do you mean that we can use expiry policy
> > other
> > >> than the one provided in configuration? How is this possible? Can you
> > >> point
> > >> to the code that implements this logic?
> > >>
> > >> -Val
> > >>
> > >> On Wed, Sep 13, 2017 at 11:29 AM, Dmitriy Setrakyan <
> > >> dsetrak...@apache.org>
> > >> wrote:
> > >>
> > >> > Denis,
> > >> >
> > >> > I agree that the behavior should be consistent, but you will not
> find
> > >> > anything about transactions in JCache. To my knowledge, JCache does
> > not
> > >> > have transactions.
> > >> >
> > >> > I would file a ticket about the issue you found, so the community
> > could
> > >> > address it. If you are interested, perhaps you can contribute a fix
> > >> > yourself.
> > >> >
> > >> > Thanks,
> > >> > D.
> > >> >
> > >> > On Wed, Sep 13, 2017 at 5:47 AM, Denis Mekhanikov <
> > >> dmekhani...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Igniters!
> > >> > >
> > >> > > I noticed a weird behavior regarding expiry policy in Ignite. You
> > can
> > >> > find
> > >> > > an example in the attachment.
> > >> > > When you call invoke on a cache with configured CacheStore and
> > >> > > ExpiryPolicy, then chosen expiry depends on cache's atomicity
> mode.
> > >> > > If cache is atomic, then "creation" expiry timeout is chosen, but
> if
> > >> it
> > >> > is
> > >> > > transactional - then "access".
> > >> > >
> > >> > > I think, this behavior should at least be identical in both cases,
> > but
> > >> > > what is more important, it should conform to JCache specification.
> > >> > > But I wasn't able to find a clear statement regarding this
> question
> > in
> > >> > the
> > >> > > specification. Can somebody point out a place in the specification
> > >> that
> > >> > > defines a behavior in such case?
> > >> > >
> > >> > > Cheers,
> > >> > > Denis
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: Ignite Benchmarking Rules

2017-09-15 Thread Nikolay Izhikov


Hello, Dmitriy.

I think experienced members of community have specific number for 
benchmarking.


Can we start from reference hardware configuration: Num of CPU, RAM and 
HDD(SDD?) configuration, network configs, etc.


Can someone share that kind of knowledge - Which hardware is best for 
Ignite benchmarking?


I found some numbers here - [1]. Is it well suited for Apache Ignite?

[1] 
https://www.gridgain.com/resources/benchmarks/gridgain-vs-hazelcast-benchmarks


14.09.2017 23:27, Dmitriy Setrakyan пишет:

Alexey, I completely agree. However, for the benchmarks to be useful, then
need to be run on the same hardware all the time. Apache Ignite does not
have servers sitting around, available to run the benchmarks.

Would be nice to see how other projects address it. Can Amazon donate
servers for the Apache projects?

D.

On Thu, Sep 14, 2017 at 6:25 AM, Aleksei Zaitsev 
wrote:


Hi, Igniters.

Recently I’ve done some research in benchmarks for Ignite, and noticed
that we don’t have any rules for running benchmarks and collecting result
from them. Although sometimes we have tasks, which results need to be
measured. I propose to formalize such things as:
  * set of benchmarks,
  * parameters of launching them,
  * way of result collection and interpretation,
  * Ignite cluster configuration.

I don’t think that we need to run benchmarks before every merge into
master, but in some cases it should be mandatory to compare new results
with reference values to be sure that changes do not lead to the
performance degradation.

What do you think?

[jira] [Created] (IGNITE-6389) Cache.invoke fails on atomic cache with configured AccessedExpiryPolicy

2017-09-15 Thread Denis Mekhanikov (JIRA)

Denis Mekhanikov created IGNITE-6389:


 Summary: Cache.invoke fails on atomic cache with configured 
AccessedExpiryPolicy
 Key: IGNITE-6389
 URL: https://issues.apache.org/jira/browse/IGNITE-6389
 Project: Ignite
  Issue Type: Bug
Reporter: Denis Mekhanikov


When calling {{invoke(...)}} on atomic cache with {{AccessedExpiryPolicy}}, and 
the target key already exists in cache, then the following exception is thrown:
{noformat}
[2017-09-15 12:10:18,435][ERROR][main][GridDhtAtomicCache]  Unexpected 
exception during cache update
class org.apache.ignite.IgniteException: Runtime failure on search row: 
org.apache.ignite.internal.processors.cache.tree.SearchRow@7334aada
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1632)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1201)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:343)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:1693)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2481)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:1944)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1797)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1689)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:483)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:443)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1170)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.invoke0(GridDhtAtomicCache.java:879)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.invoke(GridDhtAtomicCache.java:837)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.invoke(IgniteCacheProxyImpl.java:1338)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.invoke(IgniteCacheProxyImpl.java:1384)
at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.invoke(GatewayProtectedCacheProxy.java:1348)
at org.apache.ignite.examples.ExpiryExample.main(ExpiryExample.java:67)
Caused by: java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.createRow(IgniteCacheOffheapManagerImpl.java:1253)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.initResultOnCancelUpdate(GridCacheMapEntry.java:4270)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:4157)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:3921)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:2988)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$6200(BPlusTree.java:2882)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1719)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1602)
... 18 more
Exception in thread "main" org.apache.ignite.cache.CachePartialUpdateException: 
Failed to update keys (retry update if possible).: [1]
at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1269)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1648)
at

[jira] [Created] (IGNITE-6388) ExpiryPolicy is used incorrectly during invoke

2017-09-15 Thread Denis Mekhanikov (JIRA)

Denis Mekhanikov created IGNITE-6388:


 Summary: ExpiryPolicy is used incorrectly during invoke
 Key: IGNITE-6388
 URL: https://issues.apache.org/jira/browse/IGNITE-6388
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.1, 2.0, 1.9, 1.8, 1.7
Reporter: Denis Mekhanikov


Behavior of {{invoke(...)}} regarding {{ExpiryPolicy}} doesn't conform to the 
spec.

Based on {{EntryProcessor}} doc, the expected behavior is:

{code:java}
cache.invoke(key, (e, a) -> {
e.getValue(); // getExpiryForAccess()
e.setValue(2);
e.getValue();
e.setValue(3); // getExpiryForUpdate()
return e.getValue();
});
{code}

{code:java}
cache.invoke(key, (e, a) -> {
e.getValue(); // getExpiryForAccess()
e.remove();
e.getValue();
e.setValue(2); // getExpiryForUpdate()
return e.getValue();
});
{code}

{code:java}
cache.invoke(key, (e, a) -> {
e.getValue(); // getExpiryForAccess()
e.setValue(2);
e.getValue();
e.setValue(3);
e.remove();
return e.getValue();
});
{code}

Additionally, if {{CacheStore}} with read-through is configured, and the 
accessed value wasn't loaded before, then {{getExpiryForCreation()}} should be 
called.

Currently behavior depends on configured atomicity mode and none of observed 
options conform to the spec.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6387) SQL: NOT NULL fields validation with read-through cache store

2017-09-15 Thread Sergey Kalashnikov (JIRA)

Sergey Kalashnikov created IGNITE-6387:
--

 Summary: SQL: NOT NULL fields validation with read-through cache 
store
 Key: IGNITE-6387
 URL: https://issues.apache.org/jira/browse/IGNITE-6387
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Sergey Kalashnikov
Assignee: Sergey Kalashnikov


There is a case left unsolved during implementation of SQL NOT NULL constraints 
feature.
It may happen so that cache update operation fails and the value loaded from 
store is put into cache.
This value must also be validated with regards to configured NOT NULL 
constraints.
See {{CacheConfiguration.setCacheStoreFactory()}}, 
{{CacheConfiguration.setReadThrough}}, {{QueryEntity.setNotNullFields}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2669: Ignite 1.9.6 b4

2017-09-15 Thread dkarachentsev

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/2669

Ignite 1.9.6 b4



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.9.6-b4

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2669.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2669


commit 6d12bd4b9fbded5e5862b5c3d89b05a5ddd11755
Author: Igor Sapego 
Date:   2017-03-21T14:54:51Z

IGNITE-4200: Added copying of the C++ binaries.

(cherry picked from commit 8b3860f)

commit 0f7ef74216fab64f5d1d2b6d432b552b7fe40d2f
Author: Andrey V. Mashenkov 
Date:   2017-04-12T10:01:25Z

IGNITE-4907: Fixed excessive service instances can be started with dynamic 
deployment. This closes #1766.

commit 465084da5b00dcfc056d338f5d0a24875ca2af08
Author: Andrey V. Mashenkov 
Date:   2017-04-12T10:01:25Z

IGNITE-4907: Fixed excessive service instances can be started with dynamic 
deployment. This closes #1766.

(cherry picked from commit 0f7ef74)

commit a20c307df1dd000309a273ef93231fdc41a2a81c
Author: dkarachentsev 
Date:   2017-04-13T06:31:17Z

IGNITE-4891 - Fix. Key is deserialized during transactional get() even if 
withKeepBinary is set

(Backport from master)

commit 630558dfeb373f237057e394e8f2f63230d59dab
Author: vladisav 
Date:   2017-04-13T10:24:42Z

ignite-4173 IgniteSemaphore with failoverSafe enabled doesn't release 
permits in case permits owner node left topology

Backport from master.

(cherry picked from commit 76485fc)

commit 870b752c095ed3776e91a65b99763142b9f2ebc0
Author: Vladisav Jelisavcic 
Date:   2017-04-11T11:09:12Z

ignite-1977 - fixed IgniteSemaphore fault tolerance.

Backport from master.

(cherry picked from commit 902bf42)

commit cd0b92950c6691c6fc1a26cb4f7e55f5ee459298
Author: Yakov Zhdanov 
Date:   2017-04-13T12:52:20Z

ignite-4946 GridCacheP2PUndeploySelfTest became failed

(cherry picked from commit d298e75)

commit 241e9291dbe43138be1c027a3d8d1b7e3ed9b4dd
Author: dkarachentsev 
Date:   2017-04-14T06:36:10Z

Merge branch 'ignite-1.9.1-p1' into ignite-1.9.2

commit 405ce563fb7c35627c6e1bb0b68f423ba089c6f2
Author: Dmitriy Shabalin 
Date:   2017-04-14T10:55:38Z

IGNITE-4068 Added common primitive for buttons group. Refactored existing 
button groups.
(cherry picked from commit e5200c2)

commit 60cf48dc175fa288cd74d1189f0e992c9a16dc99
Author: Vasiliy Sisko 
Date:   2017-04-14T11:00:47Z

IGNITE-4886 Catch all errors.
(cherry picked from commit 7e8d9e8)

commit 81c3ed4c0511841f7056677db6063b4eb8d2def0
Author: Alexey Kuznetsov 
Date:   2017-04-14T11:18:23Z

IGNITE-4896 Rewored GridClientNodeBean serialization.
(cherry picked from commit a025268)

commit 4a1415ad01ff9fde30d5c7c02e6d938f1515178d
Author: Andrey V. Mashenkov 
Date:   2017-04-12T10:01:25Z

IGNITE-4907: Fixed excessive service instances can be started with dynamic 
deployment. This closes #1766.

(cherry picked from commit 0f7ef74)

commit e206b9f1fd3dbf927f940d558144a4796479ed5d
Author: vsisko 
Date:   2017-04-14T11:32:30Z

IGNITE-4871 Added Kubernetes IP finder to Cluster configuration screen.
(cherry picked from commit f978ff2)

commit 4518f51f4c1c29517d518f5e44f5d70d17c51170
Author: sboikov 
Date:   2017-04-14T13:38:46Z

Added IgniteGetFromComputeBenchmark.

commit b22738080101536a8af1ed60e70d693897e9bc7c
Author: dkarachentsev 
Date:   2017-04-14T14:54:02Z

ignite-4173 Fix test. Permits must be released on node fail.

(cherry picked from commit 1f867c6)

commit 41c5288606710b9c42983780ac7464a746d09eb0
Author: dkarachentsev 
Date:   2017-04-14T14:56:25Z

Merge remote-tracking branch 'origin/ignite-1.8.6' into ignite-1.8.6

commit 1985496ea98f4d7112a0b99727297427f343e9b2
Author: dkarachentsev 
Date:   2017-04-17T07:20:41Z

Add missed license headers.

commit ae794ab45b22abe1e570d5d29661396ea7c0f951
Author: dkarachentsev 
Date:   2017-04-17T10:40:28Z

IGNITE-4159 Fix version.

commit bf1049741f7a64728bd433f78262ba273f969848
Author: Andrey V. Mashenkov 
Date:   2017-04-17T16:00:30Z

IGNITE-4954 - Configurable expiration timeout for Cassandra session. This 
closes #1785.

commit 86c49514c64225059a31912fdccadc6dc68a8397
Author: sboikov

[GitHub] ignite pull request #2609: ignite-6289 Remove "allOrNone" flag from IgniteSe...

2017-09-15 Thread dmekhanikov

Github user dmekhanikov closed the pull request at:

https://github.com/apache/ignite/pull/2609


---

[GitHub] ignite pull request #2590: IGNITE-5817 Changed checksum calculation methods

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2590


---

[GitHub] ignite pull request #2535: IGNITE-6182

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2535


---

[GitHub] ignite pull request #2586: IGNITE-6237 created script for signing and upload...

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2586


---

Re: Durable Memory and Ignite Persistence Tuning

2017-09-15 Thread Ivan Rakov

Denis,

Yes, Ignite page size should be *greater or equal* than OS page cache
size and SSD page size. I mentioned it in advice list:

page size of Ignite shouldn't be less than SSD page size
Page size of Ignite shouldn't be less than OS page size

Sorry for vague wording.
We should fix this in documentation.

Best Regards,
Ivan Rakov

On 14.09.2017 3:46, Denis Magda wrote:

Ivan,

Documented:
https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning

However, I’m a bit confused with you 6. point regarding the page size calculation.
Should Ignite page size be always less than the SSD’s page size and OS page cache
size? Or it can be equal? For instance if the OS page cache size is 4 KB what should
be Ignite’s page size? That’s the section about this:
https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning#section-page-size

—
Denis

On Sep 13, 2017, at 2:29 AM, Ivan Rakov wrote:

Folks,

We had some experience of benchmarking Ignite with persistent store on SSD. I
think we can share some helpful advice. None of them require changing
configuration of Ignite or persistent store.

*Tuning advice for users*

1) Be prepared for LFS performance decrease after several hours of intensive
load. Unfortunately, that's how SSD drives work:
http://codecapsule.com/2014/02/12/coding-for-ssds-part-2-architecture-of-an-ssd-and-benchmarking/
Consider buying fast production-level SSD drives.
2) Consider using separate drives for LFS files and WAL. Ignite actively
performs writes to both LFS and WAL under intensive load, and having two
devices will double your throughput limit.
3) Over-provision your SSD. Performance of random writes on 50% filled disk is
much better than on 90% filled. SSD Over-Provisioning And Its Benefits:
http://www.seagate.com/ru/ru/tech-insights/ssd-over-provisioning-benefits-master-ti/
4) Leave free space in RAM to let OS use page cache and optimize writes. Total
size of all memory policies shouldn't exceed 70% of your RAM.
5) Make sure that OS doesn't utilize swap. If you use Unix, best option is set
vm.swappiness to 0.
6) Try to find out page size of your SSD. Ideally, page size of Ignite
shouldn't be less than SSD page size. Possible approaches:
Find it in device specification (some manufacturers don't reveal it)
Try running SSD benchmarks
If you are not sure, just set page size to 4K. As various benchmarks use 4K
pages, manufacturers have to adapt drives for 4K random write workload.
Whitepaper from Intel showing that 4K pages are enough:
https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ssd-server-storage-applications-paper.pdf
Check your OS page cache size. Page size of Ignite shouldn't be less than OS
page size. How to check OS cache page size in Unix:
https://unix.stackexchange.com/questions/128213/how-is-page-size-determined-in-virtual-address-space

Best Regards,
Ivan Rakov

On 01.09.2017 21:08, Denis Magda wrote:

Igniters,

I see a lot of complains regarding the performance of the subj on the user
list. At the same time, I do believe that in most scenarios it’s a lack of
knowledge that we keep in secret.

It's time to document Durable Memory and its Native Persistence tuning parameters.
Let's start doing this for Linux based deployments first. Here is what we have for
now (which is almost nothing):
https://apacheignite.readme.io/docs/durable-memory-tuning

Ideally, at some point we have to come up with doc like this:
https://access.redhat.com/sites/default/files/attachments/deploying-oracle-12c-on-rhel6_1.2_1.pdf

Please share your expertise in a form of settings that have to be put on the
paper. We put them in JIRA and document afterwords:
https://issues.apache.org/jira/browse/IGNITE-6246

—
Denis

[GitHub] ignite pull request #2668: ignite-5960 race fixed