Re: MVCC configuration

[Dmitriy Setrakyan]

Can caches within the same group have different MVCC configuration?

On Tue, Sep 19, 2017 at 2:34 AM, Vladimir Ozerov 
wrote:

> What I mean is that it might be not applicable for DML by design. E.g. may
> be we will have to fallback to per-memory-policy approach, or to
> per-cache-group. As we do not know it at the moment and there is no clear
> demand from users, I would simply put it aside to avoid in mess in public
> API in future.
>
> Moreover, per-cache flag raises additional questions which can be put out
> of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
> single transaction? If yes, what is the contract? Without fine-grained
> per-cache approach in the first iteration we can avoid answering it.
>
> On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov 
> wrote:
>
> > If it is not valid for DML then we can easily detect this situation and
> > throw exception, but if I do not use DML why non make it configurable
> > per-cache?
> >
> > On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov 
> > wrote:
> >
> > > I would say that per-cache configuration should be out of scope as well
> > for
> > > the first iteration. Because we do not know whether it will be valid
> for
> > > DML.
> > >
> > > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov 
> > > wrote:
> > >
> > > > Folks, thank you for feedback, I want to summarize some decisions:
> > > >
> > > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > > per-cache flag - CacheConfiguration.isMvccEnabled, default value for
> > all
> > > > caches - IgniteConfiguration.isMvccEnabled.
> > > > 2. For initial implementation mvcc for ATOMIC cache is out of scope,
> it
> > > can
> > > > be enabled only for TRANSACTIONAL caches.
> > > > 3. Mvcc coordinator can be any server node (oldest server node is
> > > selected
> > > > automatically). Also I believe we need possibility to have
> *dedicated*
> > > mvcc
> > > > coordinator nodes which will process only internal mvcc messages.
> Node
> > > can
> > > > be marked as dedicated coordinator via new flag
> > > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > > MvccConfiguration bean. But let's skip this decision for now before
> we
> > > have
> > > > benchmarks numbers.
> > > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > > >
> > > >
> > > > Thanks
> > > >
> > > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> > voze...@gridgain.com>
> > > > wrote:
> > > >
> > > > > This could be something like "preferredMvccCoordinator".
> > > > >
> > > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > > alexey.goncha...@gmail.com> wrote:
> > > > >
> > > > > > >
> > > > > > > I agree that we need coordinator nodes, but I do not understand
> > why
> > > > > can't
> > > > > > > we reuse some cache nodes for it? Why do we need to ask user to
> > > start
> > > > > up
> > > > > > > yet another type of node?
> > > > > > >
> > > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > My understanding is that Semyon does not deny a cache node to be
> > used
> > > > as
> > > > > a
> > > > > > coordinator. This property will allow to optionally have a
> > > *dedicated*
> > > > > node
> > > > > > serving as a coordinator to improve cluster throughput under
> heavy
> > > > load.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Ignite Enhancement Proposal process

[Dmitriy Setrakyan]

Great idea. Either name is fine by me, but we badly need to add these Wiki
pages ASAP. Here are some candidates:

- Usability
- SQL
- MVCC
- Persistence
- Performance

D.

On Tue, Sep 19, 2017 at 6:00 AM, Vladimir Ozerov 
wrote:

> Both "improvement" and "enhancement" are OK as both are widely used in OSS
> projects. Ideally, the abbreviation itself should sound cool. Consider
> "Python Ehnacement Proposals" - PEPS! :-)
>
> On Tue, Sep 19, 2017 at 3:10 PM, Andrey Kuznetsov 
> wrote:
>
> > Really cool idea!
> >
> > It's not convenient to look for subtle details in devlist discussion
> > thread.
> >
> > But I'd prefer the word "improvement" rather than "enhancement": the
> stuff
> > being described is not always a sharp-cut functionality.
> >
> > 2017-09-19 14:56 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > great suggestion
> > >
> > >
> >
> > --
> > Best regards,
> >   Andrey Kuznetsov.
> >
>

[jira] [Created] (IGNITE-6446) Stuck on "Loading" screen

[Ilya Borisov (JIRA)]

Ilya Borisov created IGNITE-6446:


 Summary: Stuck on "Loading" screen
 Key: IGNITE-6446
 URL: https://issues.apache.org/jira/browse/IGNITE-6446
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Affects Versions: 2.1
Reporter: Ilya Borisov
Assignee: Ilya Borisov
 Fix For: 2.3


*What happens:*
Web console gets stuck in loading screen when user opens any URL without prior 
app navigation and any permission check or HTTP request fails due to 
missing/expired authorization cookie.

*How to reproduce:*
1. Log out of web console or open private browser tab.
2. Go to http://locahost:9000/configuration/basic

*What should happen:*
User should be redirected to either 403 or signin screen.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Annoying extra steps for enabling metrics

[Denis Magda]

Let’s simplifying the metrics as a part of this ticket:
https://issues.apache.org/jira/browse/IGNITE-5796 


Expanded its scope.

—
Denis

> On Sep 9, 2017, at 2:44 PM, Valentin Kulichenko 
>  wrote:
> 
> statisticsEnabled property comes from JCache, BTW.
> 
> -Val
> 
> On Sat, Sep 9, 2017 at 11:09 AM, Dmitriy Setrakyan 
> wrote:
> 
>> On Sat, Sep 9, 2017 at 8:56 AM, Denis Magda  wrote:
>> 
>>> Surprise!
>>> 
>>> If you want to see cache events then you have to enable one more flag!
>>> 
>>> 
>> 
>> 
>> What is the overhead of this statistics collection?
>> 
>> 
>>> Three flags/beans have to be in the config in total, three! Just to see
>>> cache events. The API is a mess. Let’s contemplate how to fix it.
>> 
>> 
>> Agree, this is horrible. We need to fix it in 2.3. Is there a ticket?
>> 
>> 
>>> 
>>> —
>>> Denis
>>> 
 On Sep 7, 2017, at 7:33 PM, Dmitriy Setrakyan 
>>> wrote:
 
 On Thu, Sep 7, 2017 at 7:30 PM, Denis Magda  wrote:
 
> My point is different. Before I had to do this only assuming that
>>> “Ignite
> will spend 99%” sending events:
> 
> 
>   
>   
>   
>   
>   
>   
>   
>   
>   
> 
> Now the platform forces me to do that (probably thinking that I’m
>> crazy
>>> if
> I want to waste resources for metrics and giving one more change to
> contemplate the decision):
> 
>   
>  
>  
> 
> Does the issue make sense to you know?
> 
 
 I understand now. Why did we change this behavior? Can someone comment?
>>> 
>>> 
>> 

JCache time-based metrics

[Вячеслав Коптилин]

I'd want to make a new attempt to discuss JCache metrics, especially
time-based metrics.
As discussed earlier (
http://apache-ignite-developers.2346864.n4.nabble.com/Cache-Metrics-tt19699.html
),
there are two approaches (and at first glance, the second one is
preferable):
#1 Node that starts some operation is responsible for updating the
cache metrics.
#2 Primary node (node that actually executes a request)

I have one question/concern about time-based metrics for transactional
caches.
The JCache specification does not have definition like a transaction,
partitioned cache etc,
and, therefore, cannot provide convenience and clear understanding about
the average time for that case.

Let's assume, for example, we have the following code:

try (Transaction tx =
transactions.txStart(TransactionConcurrency.OPTIMISTIC,
TransactionIsolation.READ_COMMITTED)) {
value = cache.get(key);
// calculate new value for the given key
...
cache.put(key, new_value); // (1)

// some additional operations

// another update
value = cache.get(key);// (2)
...
cache.put(key, new_value);

tx.commit();   // (3)
}

What is the average time for write operation? Is it a time needed for
enlisting an entry in the transaction scope, acquiring locks, committing
changes?
For that particular case, current implementation accumulates both timings
(1)&(2) on the local node during performing put operation, but 'write'
counter is updated only once on data node during commit phase.
I think this behavior is not obvious at least.

Thanks,
Slava.

Re: IGNITE-2894 - Binary object inside of Externalizable still serialized with OptimizedMarshaller

[Valentin Kulichenko]

Nikita,

It sounds like the test should be changed, no? In case I'm missing
something, can you please give more details about the scenario which
requires deserialization? Generally, this sounds weird - in cases when we
can get advantage of binary format and avoid deserialization, we definitely
should not deserialize.

-Val

On Tue, Sep 19, 2017 at 6:17 AM, Nikita Amelchev 
wrote:

> I have some problem when we don't deserialize Externalizable. Some messages
> require deserializing in GridCacheIoManager.message0(). For example, tests
> like testResponseMessageOnUnmarshallingFailed where readExternal throws an
> exception. A message containing Externalizable is deserialized and
> processed as a failed message. If we do not deserialize here, we won't
> process this message as failed. What way to resolve it? I see we can try to
> deserialize after a check on Externalizable in a finishUnmarshall method,
> but it looks bad. What are your thoughts?
>
> 2017-09-07 12:57 GMT+03:00 Nikita Amelchev :
>
> > I also agree that we should try to use Externalizable without
> > deserialization on servers. I will do it in a way suggested in the
> review.
> > The Externalizable will marshal through type GridBinaryMarshaller.OBJ, in
> > addition, I’ll add a flag in BinaryConfiguration which will be used to
> turn
> > on the old way with OptimizedMarshaller for compatibility with the
> current
> > data format.
> >
> > 2017-09-06 4:21 GMT+03:00 Dmitriy Setrakyan :
> >
> >> Vova, I agree. Let's stay loyal to our binary serialization protocol.
> >>
> >> Do you know if we will be loosing any functionality available in
> >> Externalizable, but not present in our binary protocol?
> >>
> >> D.
> >>
> >> On Mon, Sep 4, 2017 at 11:38 PM, Vladimir Ozerov 
> >> wrote:
> >>
> >> > Folks,
> >> >
> >> > Let's discuss how this should be handled properly. I proposed to use
> the
> >> > same format as regular binary object, with all data being written in
> >> "raw"
> >> > form. This would give us one critical advantage - we will be able to
> >> work
> >> > with such objects without deserialization on the server. Hence, no
> >> classes
> >> > will be needed on the server side. Current implementation (see PR in
> the
> >> > ticket) defines separate format which require deserialization, I am
> not
> >> OK
> >> > with it.
> >> >
> >> > Thoughts?
> >> >
> >> > On Wed, Aug 23, 2017 at 6:11 PM, Nikita Amelchev <
> nsamelc...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hello, Igniters!
> >> > >
> >> > > I've developed Externalizable interface support using
> BinaryMarshaller
> >> > [1].
> >> > >
> >> > > I have a misunderstanding with defining BinaryWriteMode in
> >> > > BinaryUtils.mode(cls): there is AffinityKey class which implements
> >> > > Externalizable and registered with ReflectiveSerialize,
> >> BinaryMarshaller
> >> > > defines it as BinaryWriteMode.OBJ and uses reflection according to
> >> > current
> >> > > logic. I want to say that AffinityKey must be defined as
> >> > > BinaryWriteMode.OBJ although the class implements the Externalizable
> >> > > interface.
> >> > > I have to add a new one more parameter in BinaryUtils.mode(cls) to
> >> define
> >> > > BinaryWriteMode in a proper way.
> >> > > Could you please review and comment my solution [2]?
> >> > >
> >> > > BTW, I have benchmarked my solution by GridMarshallerPerformanceTest
> >> and
> >> > it
> >> > > becomes faster up to 2 times (GridMarshaller).My JMH tests show that
> >> > > marshal faster up to 50% and unmarshal faster up to 100% on an
> >> > > Externalizable object.
> >> > >
> >> > > Also, I've filed an issue for Serializable interface support using
> >> > > BinaryMarshaller [3] as it has been described earlier.
> >> > >
> >> > > [1] https://issues.apache.org/jira/browse/IGNITE-2894
> >> > > [2] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-278
> >> > > [3] https://issues.apache.org/jira/browse/IGNITE-6172
> >> > >
> >> > > 2017-08-21 20:43 GMT+03:00 Valentin Kulichenko <
> >> > > valentin.kuliche...@gmail.com>:
> >> > >
> >> > > > Nikita,
> >> > > >
> >> > > > I think anything binary related should have higher priority than
> >> > > > Externalizable. I.e. if user explicitly implemented Binarylizable
> or
> >> > > > provided a BinarySerializer, then BinaryMarshaller should of
> course
> >> use
> >> > > > that and ignore Externalizable.
> >> > > >
> >> > > > -Val
> >> > > >
> >> > > > On Mon, Aug 21, 2017 at 9:29 AM, Nikita Amelchev <
> >> nsamelc...@gmail.com
> >> > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hello, Igniters.
> >> > > > >
> >> > > > > I am developing Externalizable interface support by
> >> BinaryMarshaller
> >> > > > > through new type constant. BinaryMarshaller allows using
> >> > > BinarySerializer
> >> > > > > to manage serialization. I need to define BinaryWriteMode in the
> >> > > > > BinaryClassDescriptor constructor. In case of the Binarylizable
> >> > > > 

Re: Binary compatibility of persistent storage

[Valentin Kulichenko]

In my view, there are two different scenarios.

First - user just upgrades the version (to get some bug fix, for example),
but does not intend to change anything in their project and/or use any new
features. In this case it should work transparently and cluster must be
able to work with older format. Ideally, we should detect this
automatically, but if it's not possible we can introduce some kind of
'compatibility mode' enabled by a system property.

Second - user upgrades to get new features that require data format change.
In this case, I think it's OK to suggest using a conversion tool. Or
probably we can apply it automatically on node startup?

-Val

On Tue, Sep 19, 2017 at 6:38 AM, Yakov Zhdanov  wrote:

> >Any major change in data/index page format. E.g. this could happen once
> transactional SQL is ready.
>
> I would suggest we automatically disable this feature for databases created
> with older versions.
>
> --Yakov
>

[GitHub] ignite pull request #2700: probable fix

[voipp]

GitHub user voipp opened a pull request:

https://github.com/apache/ignite/pull/2700

probable fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/voipp/ignite ignite-5960-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2700.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2700


commit 7fba1c7cf34d4828af334f084f530f9f7c011460
Author: voipp 
Date:   2017-09-19T16:33:26Z

probable fix




---

[GitHub] ignite pull request #2658: ignite-5960 cont counter null if nobody listens

[voipp]

Github user voipp closed the pull request at:

https://github.com/apache/ignite/pull/2658


---

[jira] [Created] (IGNITE-6445) IgniteTxManager.txLocksInfo method misses locks

[Vitaliy Biryukov (JIRA)]

Vitaliy Biryukov created IGNITE-6445:


 Summary: IgniteTxManager.txLocksInfo method misses locks
 Key: IGNITE-6445
 URL: https://issues.apache.org/jira/browse/IGNITE-6445
 Project: Ignite
  Issue Type: Bug
Reporter: Vitaliy Biryukov


In some cases "IgniteTxManager.txLocksInfo" method (searches for locks) misses 
locks.
For example:
In case of a configuration with near cache, entities are created for the near 
cache and for the ordinal cache. For each entity, their own MVCC candidates are 
created.
For non-custom objects of type (Integer, etc.), the entity stored in 
"GridNearTxLocal" is not associated with MVCC candidates with which the same 
entity is associated in another format stored in "GridDhtTxLocal"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6444) Validate that copyOnRead flag is configured with on-heap cache enabled

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-6444:


 Summary: Validate that copyOnRead flag is configured with on-heap 
cache enabled
 Key: IGNITE-6444
 URL: https://issues.apache.org/jira/browse/IGNITE-6444
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.0
Reporter: Alexey Goncharuk
 Fix For: 2.3


Link to the user-list discussion:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-1-0-CopyOnRead-Problem-td17009.html

It makes sense to validate the flag and print out a warning if on-heap cache is 
disabled. I do not think that we should prevent node from startup because this 
may break existing deployments.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [RESULT] [VOTE] Apache Ignite 2.2.0 Release (RC2)

[Pavel Tupitsyn]

NuGet packages pushed: https://www.nuget.org/packages?q=Apache.Ignite

On Tue, Sep 19, 2017 at 4:46 PM, Anton Vinogradov  wrote:

> - Maven artifacts released
> - Sources released
> - Site updated
> - Git tag added
>
> On Mon, Sep 18, 2017 at 6:19 PM, Anton Vinogradov  wrote:
>
> > Igniters,
> >
> > Apache Ignite 2.2.0 release (RC2) has been accepted.
> >
> > 4 "+1" binding votes received:
> >
> > - Vladimir Ozerov
> > - Alexey Kuznetsov
> > - Andrey Novikov
> > - Pavel Tupitsyn
> >
> > Vote thread:
> >
> > http://apache-ignite-developers.2346864.n4.nabble.
> > com/VOTE-Apache-Ignite-2-2-0-RC2-td22261.html
> >
> > Ignite 2.2.0 will be released soon.
> >
> > Yabba Dabba Doo!
> >
>

[jira] [Created] (IGNITE-6443) Test IgniteSourceConnectorTest has flaky fails

[Andrey Gura (JIRA)]

Andrey Gura created IGNITE-6443:
---

 Summary: Test IgniteSourceConnectorTest has flaky fails
 Key: IGNITE-6443
 URL: https://issues.apache.org/jira/browse/IGNITE-6443
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Gura
 Fix For: 2.3


{{ IgniteSourceConnectorTest.testEventsInjectedIntoKafka}} and {{ 
IgniteSourceConnectorTest.testEventsInjectedIntoKafkaWithoutFilter}} fail 
periodically.

{noformat}
junit.framework.AssertionFailedError: expected:<100> but was:<117>
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.failNotEquals(Assert.java:329)
at junit.framework.Assert.assertEquals(Assert.java:78)
at junit.framework.Assert.assertEquals(Assert.java:234)
at junit.framework.Assert.assertEquals(Assert.java:241)
at junit.framework.TestCase.assertEquals(TestCase.java:409)
at 
org.apache.ignite.stream.kafka.connect.IgniteSourceConnectorTest.checkDataDelivered(IgniteSourceConnectorTest.java:304)
at 
org.apache.ignite.stream.kafka.connect.IgniteSourceConnectorTest.doTest(IgniteSourceConnectorTest.java:213)
at 
org.apache.ignite.stream.kafka.connect.IgniteSourceConnectorTest.testEventsInjectedIntoKafka(IgniteSourceConnectorTest.java:155)
{noformat}

Tests shold be unmuted on TC after fix.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2699: IGNITE-6442

[BiryukovVA]

GitHub user BiryukovVA opened a pull request:

https://github.com/apache/ignite/pull/2699

IGNITE-6442



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/BiryukovVA/ignite IGNITE-6442

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2699.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2699


commit b2ee52f335c02211b3c26aab3eae708b6fe09cd9
Author: Vitaliy Biryukov 
Date:   2017-09-19T15:39:40Z

IGNITE-6442: Bug fixed.




---

[jira] [Created] (IGNITE-6442) Deadlock detection doesn't execute.

[Vitaliy Biryukov (JIRA)]

Vitaliy Biryukov created IGNITE-6442:


 Summary: Deadlock detection doesn't execute.
 Key: IGNITE-6442
 URL: https://issues.apache.org/jira/browse/IGNITE-6442
 Project: Ignite
  Issue Type: Bug
Reporter: Vitaliy Biryukov


In case of a configuration with near cache and if all entities of one of the 
transactions involved in the deadlock are on the node being the initiator of 
this transaction, then immediately after the timeout, the transaction rolls 
back (without calling DD).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2698: IGNITE-6250 .NET: Thin client: Basic exception ha...

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/2698

IGNITE-6250 .NET: Thin client: Basic exception handling



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6250

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2698.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2698


commit dd94981c250070d0b36b6da398b032c2b745b73d
Author: Pavel Tupitsyn 
Date:   2017-09-19T15:13:49Z

IGNITE-6250 .NET: Thin client: Basic exception handling




---

[jira] [Created] (IGNITE-6441) Thin client: configurable message size limit

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6441:
--

 Summary: Thin client: configurable message size limit
 Key: IGNITE-6441
 URL: https://issues.apache.org/jira/browse/IGNITE-6441
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Reporter: Pavel Tupitsyn


* Limit thin client message size (to avoid OOMEs)
* Make it configurable (see {{ClientConnectorConfiguration}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6440) Flaky failures in DynamicColumnsAbstractConcurrentSelfTest

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-6440:
---

 Summary: Flaky failures in DynamicColumnsAbstractConcurrentSelfTest
 Key: IGNITE-6440
 URL: https://issues.apache.org/jira/browse/IGNITE-6440
 Project: Ignite
  Issue Type: Task
  Components: sql
Affects Versions: 2.3
Reporter: Vladimir Ozerov
Assignee: Alexander Paschenko
 Fix For: 2.3


Multiple failures are observed in concrete implementations of 
{{DynamicColumnsAbstractConcurrentSelfTest}}. Specifically:
{code}
testQueryConsistencyMultithreaded
testClientReconnect
testConcurrentRebalance 
testCoordinatorChange
testConcurrentPutRemove
{code}

Apparently there are some bugs in the test itself, as the following root causes 
could be observed in logs:
{code}
junit.framework.AssertionFailedError: Found nodes from different clusters, 
probable some test does not stop nodes 
[allNodes=[index.DynamicColumnsConcurrentTransactionalReplicatedSelfTest3, 
index.DynamicColumnsConcurrentTransactionalReplicatedSelfTest2, 
index.DynamicColumnsConcurrentTransactionalReplicatedSelfTest1]]
{code}

{code}
Caused by: java.lang.NullPointerException: null
at 
org.apache.ignite.internal.processors.cache.index.DynamicColumnsAbstractConcurrentSelfTest$3.call(DynamicColumnsAbstractConcurrentSelfTest.java:565)
at 
org.apache.ignite.internal.processors.cache.index.DynamicColumnsAbstractConcurrentSelfTest$3.call(DynamicColumnsAbstractConcurrentSelfTest.java:560)
at 
org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:86)
{code}

Please see TeamCity, history of "Queries 2" suite in master branch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2677: ignite-5960 Sending backupQueue to node.

[voipp]

Github user voipp closed the pull request at:

https://github.com/apache/ignite/pull/2677


---

Re: [RESULT] [VOTE] Apache Ignite 2.2.0 Release (RC2)

[Anton Vinogradov]

- Maven artifacts released
- Sources released
- Site updated
- Git tag added

On Mon, Sep 18, 2017 at 6:19 PM, Anton Vinogradov  wrote:

> Igniters,
>
> Apache Ignite 2.2.0 release (RC2) has been accepted.
>
> 4 "+1" binding votes received:
>
> - Vladimir Ozerov
> - Alexey Kuznetsov
> - Andrey Novikov
> - Pavel Tupitsyn
>
> Vote thread:
>
> http://apache-ignite-developers.2346864.n4.nabble.
> com/VOTE-Apache-Ignite-2-2-0-RC2-td22261.html
>
> Ignite 2.2.0 will be released soon.
>
> Yabba Dabba Doo!
>

Re: Binary compatibility of persistent storage

[Yakov Zhdanov]

>Any major change in data/index page format. E.g. this could happen once
transactional SQL is ready.

I would suggest we automatically disable this feature for databases created
with older versions.

--Yakov

[GitHub] ignite pull request #2696: IGNITE-6399 .NET: ClientConnectorConfiguration

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2696


---

[GitHub] ignite pull request #2697: ignite-6339 Segmented ring buffer implemented ins...

[agura]

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/2697

ignite-6339 Segmented ring buffer implemented instead of WAL records chain

…chain

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agura/incubator-ignite ignite-6339

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2697.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2697


commit 501e0a1b96ce67a07cac573692841e2bf362644d
Author: Andrey Gura 
Date:   2017-09-13T12:36:26Z

ignite-6339 Segmented ring buffer implemented instead of WAL records chain




---

[GitHub] ignite pull request #2689: IGNITE-6099: Fully implemented SQLGetInfo in ODBC...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2689


---

Re: IGNITE-2894 - Binary object inside of Externalizable still serialized with OptimizedMarshaller

[Nikita Amelchev]

I have some problem when we don't deserialize Externalizable. Some messages
require deserializing in GridCacheIoManager.message0(). For example, tests
like testResponseMessageOnUnmarshallingFailed where readExternal throws an
exception. A message containing Externalizable is deserialized and
processed as a failed message. If we do not deserialize here, we won't
process this message as failed. What way to resolve it? I see we can try to
deserialize after a check on Externalizable in a finishUnmarshall method,
but it looks bad. What are your thoughts?

2017-09-07 12:57 GMT+03:00 Nikita Amelchev :

> I also agree that we should try to use Externalizable without
> deserialization on servers. I will do it in a way suggested in the review.
> The Externalizable will marshal through type GridBinaryMarshaller.OBJ, in
> addition, I’ll add a flag in BinaryConfiguration which will be used to turn
> on the old way with OptimizedMarshaller for compatibility with the current
> data format.
>
> 2017-09-06 4:21 GMT+03:00 Dmitriy Setrakyan :
>
>> Vova, I agree. Let's stay loyal to our binary serialization protocol.
>>
>> Do you know if we will be loosing any functionality available in
>> Externalizable, but not present in our binary protocol?
>>
>> D.
>>
>> On Mon, Sep 4, 2017 at 11:38 PM, Vladimir Ozerov 
>> wrote:
>>
>> > Folks,
>> >
>> > Let's discuss how this should be handled properly. I proposed to use the
>> > same format as regular binary object, with all data being written in
>> "raw"
>> > form. This would give us one critical advantage - we will be able to
>> work
>> > with such objects without deserialization on the server. Hence, no
>> classes
>> > will be needed on the server side. Current implementation (see PR in the
>> > ticket) defines separate format which require deserialization, I am not
>> OK
>> > with it.
>> >
>> > Thoughts?
>> >
>> > On Wed, Aug 23, 2017 at 6:11 PM, Nikita Amelchev 
>> > wrote:
>> >
>> > > Hello, Igniters!
>> > >
>> > > I've developed Externalizable interface support using BinaryMarshaller
>> > [1].
>> > >
>> > > I have a misunderstanding with defining BinaryWriteMode in
>> > > BinaryUtils.mode(cls): there is AffinityKey class which implements
>> > > Externalizable and registered with ReflectiveSerialize,
>> BinaryMarshaller
>> > > defines it as BinaryWriteMode.OBJ and uses reflection according to
>> > current
>> > > logic. I want to say that AffinityKey must be defined as
>> > > BinaryWriteMode.OBJ although the class implements the Externalizable
>> > > interface.
>> > > I have to add a new one more parameter in BinaryUtils.mode(cls) to
>> define
>> > > BinaryWriteMode in a proper way.
>> > > Could you please review and comment my solution [2]?
>> > >
>> > > BTW, I have benchmarked my solution by GridMarshallerPerformanceTest
>> and
>> > it
>> > > becomes faster up to 2 times (GridMarshaller).My JMH tests show that
>> > > marshal faster up to 50% and unmarshal faster up to 100% on an
>> > > Externalizable object.
>> > >
>> > > Also, I've filed an issue for Serializable interface support using
>> > > BinaryMarshaller [3] as it has been described earlier.
>> > >
>> > > [1] https://issues.apache.org/jira/browse/IGNITE-2894
>> > > [2] https://reviews.ignite.apache.org/ignite/review/IGNT-CR-278
>> > > [3] https://issues.apache.org/jira/browse/IGNITE-6172
>> > >
>> > > 2017-08-21 20:43 GMT+03:00 Valentin Kulichenko <
>> > > valentin.kuliche...@gmail.com>:
>> > >
>> > > > Nikita,
>> > > >
>> > > > I think anything binary related should have higher priority than
>> > > > Externalizable. I.e. if user explicitly implemented Binarylizable or
>> > > > provided a BinarySerializer, then BinaryMarshaller should of course
>> use
>> > > > that and ignore Externalizable.
>> > > >
>> > > > -Val
>> > > >
>> > > > On Mon, Aug 21, 2017 at 9:29 AM, Nikita Amelchev <
>> nsamelc...@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > Hello, Igniters.
>> > > > >
>> > > > > I am developing Externalizable interface support by
>> BinaryMarshaller
>> > > > > through new type constant. BinaryMarshaller allows using
>> > > BinarySerializer
>> > > > > to manage serialization. I need to define BinaryWriteMode in the
>> > > > > BinaryClassDescriptor constructor. In case of the Binarylizable
>> > > > interface -
>> > > > > serializer is ignored and BinaryWriteMode is BINARY. Can I do the
>> > same
>> > > > with
>> > > > > the Externalizable interface?
>> > > > >
>> > > > > In this case, I have issues with AffinityKey: some tests have
>> failed
>> > > > > because of they except serialization logic of defined the
>> serializer
>> > > > > instead of Externalizable logic. What is the priority between
>> > > predefined
>> > > > > BinarySerializer for class and implementation of Externalizable
>> > > > interface?
>> > > > >
>> > > > > 2017-08-01 13:09 GMT+03:00 Vladimir Ozerov > >:
>> > > > >
>> > > > > > Valya,

Re: Binary compatibility of persistent storage

[Aleksei Zaitsev]

I vote for the 4th variant, because it is the most common approach to the 
versioning.

For example, SemVer says that you can make incompatible API changes only in 
major versions.

19.09.2017, 14:52, "Vladimir Ozerov" :
> Yakov,
>
> Any major change in data/index page format. E.g. this could happen once
> transactional SQL is ready.
>
> On Tue, Sep 19, 2017 at 2:51 PM, Yakov Zhdanov  wrote:
>
>>  Vladimir,
>>
>>  Can you please describe the situation when 2 is possible, but 4 is not?
>>
>>  --Yakov

Re: Ignite Enhancement Proposal process

[Vladimir Ozerov]

Both "improvement" and "enhancement" are OK as both are widely used in OSS
projects. Ideally, the abbreviation itself should sound cool. Consider
"Python Ehnacement Proposals" - PEPS! :-)

On Tue, Sep 19, 2017 at 3:10 PM, Andrey Kuznetsov  wrote:

> Really cool idea!
>
> It's not convenient to look for subtle details in devlist discussion
> thread.
>
> But I'd prefer the word "improvement" rather than "enhancement": the stuff
> being described is not always a sharp-cut functionality.
>
> 2017-09-19 14:56 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > great suggestion
> >
> >
>
> --
> Best regards,
>   Andrey Kuznetsov.
>

Re: Binary compatibility of persistent storage

[Vladimir Ozerov]

Yakov,

Any major change in data/index page format. E.g. this could happen once
transactional SQL is ready.

On Tue, Sep 19, 2017 at 2:51 PM, Yakov Zhdanov  wrote:

> Vladimir,
>
> Can you please describe the situation when 2 is possible, but 4 is not?
>
> --Yakov
>

Re: Binary compatibility of persistent storage

[Dmitry Pavlov]

I have same idea about testing of WAL compatibilty between releases. It
will be actual since 2.3 release (as 2.2 was emergency release without
impact changes).

I look forward to IGNITE-5732 completion and merge. And then I able to
start writting first test.

I think it is quite unexpected for user if Ignite is not able to start from
existing database after updating to next release. That is why I vote for
backward compatibilty with automatic transparent migration (if needed).

вт, 19 сент. 2017 г. в 15:16, Vyacheslav Daradur :

> I vote for: “4) Maintain compatibility between all versions within major
> release”.
> I think this is a trade-off between the complexity of implementing new
> features and UX.
>
> We will be able to get rid of all legacy tools every major release.
>
> I’m working on a testing framework, which helps us testing compatibility
> features between different Ignite version.
> Testing in the early stages will help us to consider the impact of changes
> in other releases and to be closer to the end user.
>
> Here is a dummy unit-test example:
>
> void testNodeStartByOldVersionPersistenceData() throws Exception {
> try {
> startGrid(1, "2.1.0", new PostConfigurationClosure(), new
> PostActionClosure());
>
> stopAllGrids(); // Stopping 2.1.0
>
> IgniteEx ignite = startGrid(0); // Starting current version
>
> ignite.active(true);
>
> IgniteCache cache =
> ignite.getOrCreateCache(TEST_CACHE_NAME);
>
> for (int i = 0; i < 10; i++)
> assertEquals("data" + i, cache.get(i));
> }
> finally {
> stopAllGrids();
> }
> }
>
> class PostActionClosure implements IgniteInClosure {
> @Override public void apply(Ignite ignite) {
> ignite.active(true);
>
> CacheConfiguration cacheCfg = new
> CacheConfiguration<>();
>
> IgniteCache cache = ignite.createCache(cacheCfg);
>
> for (int i = 0; i < 10; i++)
> cache.put(i, "data" + i);
> }
> }
>
> class PostConfigurationClosure implements
> IgniteInClosure {
> @Override public void apply(IgniteConfiguration cfg) {
> // Post configuration actions
> cfg.setPersistentStoreConfiguration(new
> PersistentStoreConfiguration());
> }
> }
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5732 - Provide API to
> test
> compatibility with old releases
>
> On Tue, Sep 19, 2017 at 2:51 PM, Yakov Zhdanov 
> wrote:
>
> > Vladimir,
> >
> > Can you please describe the situation when 2 is possible, but 4 is not?
> >
> > --Yakov
> >
>
>
>
> --
> Best Regards, Vyacheslav D.
>

[jira] [Created] (IGNITE-6439) IgnitePersistentStoreSchemaLoadTest is broken

[Dmitriy Govorukhin (JIRA)]

Dmitriy Govorukhin created IGNITE-6439:
--

 Summary: IgnitePersistentStoreSchemaLoadTest is broken
 Key: IGNITE-6439
 URL: https://issues.apache.org/jira/browse/IGNITE-6439
 Project: Ignite
  Issue Type: Bug
Reporter: Dmitriy Govorukhin
Assignee: Dmitriy Govorukhin


After start nodes, cluster must be activated explicit.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2690: IGNITE 6428: IgniteOOME in PDS Indexing test:

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2690


---

Re: IGNITE-1025

[Иван Федотов]

Anton,
Thanks, I'll start it.

2017-09-19 14:54 GMT+03:00 Anton Vinogradov :

> Ivan,
>
> Issue is still actual.
>
> On Tue, Sep 19, 2017 at 2:10 PM, Иван Федотов  wrote:
>
> > Hi, Igniters!
> > I've picked up task https://issues.apache.org/jira/browse/IGNITE-1025
> > "Need
> > to print out warning if IP finder has a lot of addresses on Windows", but
> > this task was created in June 2015. Is this task still relevant?
> > Thanks in advance.
> >
> > --
> > Ivan Fedotov.
> >
> > ivanan...@gmail.com
> >
>



-- 
Ivan Fedotov.

ivanan...@gmail.com

Re: Binary compatibility of persistent storage

[Vyacheslav Daradur]

I vote for: “4) Maintain compatibility between all versions within major
release”.
I think this is a trade-off between the complexity of implementing new
features and UX.

We will be able to get rid of all legacy tools every major release.

I’m working on a testing framework, which helps us testing compatibility
features between different Ignite version.
Testing in the early stages will help us to consider the impact of changes
in other releases and to be closer to the end user.

Here is a dummy unit-test example:

void testNodeStartByOldVersionPersistenceData() throws Exception {
try {
startGrid(1, "2.1.0", new PostConfigurationClosure(), new
PostActionClosure());

stopAllGrids(); // Stopping 2.1.0

IgniteEx ignite = startGrid(0); // Starting current version

ignite.active(true);

IgniteCache cache =
ignite.getOrCreateCache(TEST_CACHE_NAME);

for (int i = 0; i < 10; i++)
assertEquals("data" + i, cache.get(i));
}
finally {
stopAllGrids();
}
}

class PostActionClosure implements IgniteInClosure {
@Override public void apply(Ignite ignite) {
ignite.active(true);

CacheConfiguration cacheCfg = new
CacheConfiguration<>();

IgniteCache cache = ignite.createCache(cacheCfg);

for (int i = 0; i < 10; i++)
cache.put(i, "data" + i);
}
}

class PostConfigurationClosure implements
IgniteInClosure {
@Override public void apply(IgniteConfiguration cfg) {
// Post configuration actions
cfg.setPersistentStoreConfiguration(new
PersistentStoreConfiguration());
}
}

[1] https://issues.apache.org/jira/browse/IGNITE-5732 - Provide API to test
compatibility with old releases

On Tue, Sep 19, 2017 at 2:51 PM, Yakov Zhdanov  wrote:

> Vladimir,
>
> Can you please describe the situation when 2 is possible, but 4 is not?
>
> --Yakov
>



-- 
Best Regards, Vyacheslav D.

Re: Ignite Enhancement Proposal process

[Andrey Kuznetsov]

Really cool idea!

It's not convenient to look for subtle details in devlist discussion thread.

But I'd prefer the word "improvement" rather than "enhancement": the stuff
being described is not always a sharp-cut functionality.

2017-09-19 14:56 GMT+03:00 ALEKSEY KUZNETSOV :

> great suggestion
>
>

-- 
Best regards,
  Andrey Kuznetsov.

Re: Ignite Enhancement Proposal process

[ALEKSEY KUZNETSOV]

great suggestion

вт, 19 сент. 2017 г. в 14:31, Yakov Zhdanov :

> Vladimir, I like the suggestion. We should definitely try it.
>
> --Yakov
>
-- 

*Best Regards,*

*Kuznetsov Aleksey*

Re: IGNITE-1025

[Anton Vinogradov]

Ivan,

Issue is still actual.

On Tue, Sep 19, 2017 at 2:10 PM, Иван Федотов  wrote:

> Hi, Igniters!
> I've picked up task https://issues.apache.org/jira/browse/IGNITE-1025
> "Need
> to print out warning if IP finder has a lot of addresses on Windows", but
> this task was created in June 2015. Is this task still relevant?
> Thanks in advance.
>
> --
> Ivan Fedotov.
>
> ivanan...@gmail.com
>

Re: Exception handling in thin client: should we pass stack traces to the client?

[Yakov Zhdanov]

Fully agree. As Alex mentioned web servers support dev and prod modes. We
need to do the same and default mode should be dev with warning on startup
and ability to change in runtime.

--Yakov

Re: Binary compatibility of persistent storage

[Yakov Zhdanov]

Vladimir,

Can you please describe the situation when 2 is possible, but 4 is not?

--Yakov

[jira] [Created] (IGNITE-6438) .NET: Thin client: Advanced exception handling

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6438:
--

 Summary: .NET: Thin client: Advanced exception handling
 Key: IGNITE-6438
 URL: https://issues.apache.org/jira/browse/IGNITE-6438
 Project: Ignite
  Issue Type: Improvement
  Components: platforms, thin client
Reporter: Pavel Tupitsyn
 Fix For: 2.3


IGNITE-6250 introduced basic exception propagation: success code and a message.

We should add full message details to the protocol:
* Exception message
* Class
* Stack trace
* Native exception binary object (if any): for example, when exception occured 
in C++ or .NET callback

To avoid exposing sensitive information in production environmanets, introduce 
a configuration property to exclude exception details (like many web servers 
have, e.g. {{}} in ASP.NET)

Dev list discussion:
http://apache-ignite-developers.2346864.n4.nabble.com/Exception-handling-in-thin-client-should-we-pass-stack-traces-to-the-client-td22392.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Binary compatibility of persistent storage

[Nikolay Izhikov]

Hello, Vladimir.

I prefer 2 option.

> 2) No compatibility, but provide migration instruments

I think we have to provide some tool for converting WAL files from any 
older format to current.


19.09.2017 14:16, Vladimir Ozerov пишет:

igniters,

Ignite doesn't have compatibility for binary protocols between different
versions, as this would make development harder and slower. On the other
hand we maintain API compatibility what helps us move users to new versions
faster.

As native persistence is implemented, new challenge appeared - whether to
maintain binary compatibility of stored data. Many approaches exist:

1) No compatibility at all - easy for us, nightmare for users (IMO)
2) No compatibility, but provide migration instruments
3) Maintain compatibility between N latest minor versions
4) Maintain compatibility between all versions within major release

The more guarantees we offer, the harder them to maintain, the better UX.

Let's think on what compatibility mode we can offer to our users if any.
Any ideas?

Vladimir.

Re: Issues if -Djava.net.preferIPv4Stack=true is not set

[Yakov Zhdanov]

Val, can you please provide links to threads you meant? I will take a look.

--Yakov

Re: Binary compatibility of persistent storage

[Anton Vinogradov]

Vote for case #4.

As far as I know, Vyacheslav Daradur already works on framework allows to
check compatibility between different version of Ignite.

Vyacheslav,
Could you provide us more details?

On Tue, Sep 19, 2017 at 2:16 PM, Vladimir Ozerov 
wrote:

> igniters,
>
> Ignite doesn't have compatibility for binary protocols between different
> versions, as this would make development harder and slower. On the other
> hand we maintain API compatibility what helps us move users to new versions
> faster.
>
> As native persistence is implemented, new challenge appeared - whether to
> maintain binary compatibility of stored data. Many approaches exist:
>
> 1) No compatibility at all - easy for us, nightmare for users (IMO)
> 2) No compatibility, but provide migration instruments
> 3) Maintain compatibility between N latest minor versions
> 4) Maintain compatibility between all versions within major release
>
> The more guarantees we offer, the harder them to maintain, the better UX.
>
> Let's think on what compatibility mode we can offer to our users if any.
> Any ideas?
>
> Vladimir.
>

Re: Ignite Enhancement Proposal process

[Yakov Zhdanov]

Vladimir, I like the suggestion. We should definitely try it.

--Yakov

Re: Static code analysis for Java

[Yakov Zhdanov]

Bas, thanks for joining!

Can you please point me  to the page listing all types of issue LGTM can
find (like this one -
https://scan.coverity.com/faq#what-types-of-issues-tool-find)?

LGTM really helped to find some bugs like incorrect key type when working
with hash map instance, unnecessary boxing, unused collections, possible
resource leaks and some more.

Do you users integrate with CI servers somehow? esp TeamCity? It would be
cool to have project state (at least from code standpoint) in one place -
i.e. at CI.

--Yakov

Binary compatibility of persistent storage

[Vladimir Ozerov]

igniters,

Ignite doesn't have compatibility for binary protocols between different
versions, as this would make development harder and slower. On the other
hand we maintain API compatibility what helps us move users to new versions
faster.

As native persistence is implemented, new challenge appeared - whether to
maintain binary compatibility of stored data. Many approaches exist:

1) No compatibility at all - easy for us, nightmare for users (IMO)
2) No compatibility, but provide migration instruments
3) Maintain compatibility between N latest minor versions
4) Maintain compatibility between all versions within major release

The more guarantees we offer, the harder them to maintain, the better UX.

Let's think on what compatibility mode we can offer to our users if any.
Any ideas?

Vladimir.

IGNITE-1025

[Иван Федотов]

Hi, Igniters!
I've picked up task https://issues.apache.org/jira/browse/IGNITE-1025 "Need
to print out warning if IP finder has a lot of addresses on Windows", but
this task was created in June 2015. Is this task still relevant?
Thanks in advance.

-- 
Ivan Fedotov.

ivanan...@gmail.com

[jira] [Created] (IGNITE-6437) DataStructure can not be obtained on client if it is created on server node.

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-6437:
-

 Summary: DataStructure can not be obtained on client if it is 
created on server node.
 Key: IGNITE-6437
 URL: https://issues.apache.org/jira/browse/IGNITE-6437
 Project: Ignite
  Issue Type: Bug
  Components: data structures
Affects Versions: 2.1
Reporter: Mikhail Cherkasov
Priority: Critical
 Fix For: 2.3


DataStructure can not be obtained on client if it is created on server node.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: HOLE query entry in CacheContinuousQueryPartitionRecovery

[Anton Vinogradov]

Nikolay,

Could you please check?

On Tue, Sep 19, 2017 at 1:50 PM, ALEKSEY KUZNETSOV  wrote:

> HOLE was introduced in CacheContinuousQueryHandler.PartitionRecovery.
> Ticket *IGNITE-426 Implemented failover for Continuous query.*
> Then it was refactored in *Continuous queries fixes.*
> After refactoring the variable is never compares to true. Probably, its a
> bug.
>
> сб, 16 сент. 2017 г. в 1:52, Denis Magda :
>
> > I like the name. "Black holes" pop up first in my head :) The hole can
> > absorb events and confine and digest them for billions of year. Sorry for
> > off topic.
> >
> > —
> > Denis
> >
> >
> > > On Sep 15, 2017, at 10:14 AM, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com> wrote:
> > >
> > > Hi, Ignters!
> > >
> > >
> > > We have a strange field HOLE in CacheContinuousQueryPartitionRecovery
> > >
> > > which compared to pending events in
> > > CacheContinuousQueryPartitionRecovery#collectEntries. And it is never
> > > equals any entry.
> > >
> > > Do we need it ? Or it can be removed.
> > >
> > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> >
> > --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>

Re: HOLE query entry in CacheContinuousQueryPartitionRecovery

[ALEKSEY KUZNETSOV]

HOLE was introduced in CacheContinuousQueryHandler.PartitionRecovery.
Ticket *IGNITE-426 Implemented failover for Continuous query.*
Then it was refactored in *Continuous queries fixes.*
After refactoring the variable is never compares to true. Probably, its a
bug.

сб, 16 сент. 2017 г. в 1:52, Denis Magda :

> I like the name. "Black holes" pop up first in my head :) The hole can
> absorb events and confine and digest them for billions of year. Sorry for
> off topic.
>
> —
> Denis
>
>
> > On Sep 15, 2017, at 10:14 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com> wrote:
> >
> > Hi, Ignters!
> >
> >
> > We have a strange field HOLE in CacheContinuousQueryPartitionRecovery
> >
> > which compared to pending events in
> > CacheContinuousQueryPartitionRecovery#collectEntries. And it is never
> > equals any entry.
> >
> > Do we need it ? Or it can be removed.
> >
> > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
>
> --

*Best Regards,*

*Kuznetsov Aleksey*

[GitHub] ignite pull request #2696: IGNITE-6399 .NET: ClientConnectorConfiguration

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/2696

IGNITE-6399 .NET: ClientConnectorConfiguration



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-6399

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2696.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2696


commit f9062b1050e4f794806de29df865743073710497
Author: Pavel Tupitsyn 
Date:   2017-09-19T08:21:25Z

IGNITE-6399 .NET: support ClientConnectorConfiguration

commit 231494a8ba3393a09722f5b0d08b7ab1e4d301e5
Author: Pavel Tupitsyn 
Date:   2017-09-19T08:23:52Z

add test

commit 67e85fdbd41db17695c20509f633c0f44500dd23
Author: Pavel Tupitsyn 
Date:   2017-09-19T08:26:44Z

Propagate new flag

commit a287d3ac2b5022863c3625edcef39e9f2c83eab7
Author: Pavel Tupitsyn 
Date:   2017-09-19T08:39:32Z

Fix tests

commit e4ac4938893a48b201de2c0442ed41cb89790a18
Author: Pavel Tupitsyn 
Date:   2017-09-19T08:59:45Z

Add ClientConnectorConfiguration

commit c3d08aea0ec3158b7ad07cc6ef2ea066a5461422
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:13:19Z

Merge branch 'master' into ignite-6399

commit e3435664579c553b7584a8cc420275cb7a7371dc
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:24:06Z

wip

commit 39d61ae3dbdc9b2f6e3af70ae20035f801d0010e
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:33:05Z

wip

commit ddbdfacc1d0345eb839cdcbc681212ecc97113ba
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:35:02Z

Fix default config

commit 0fd31949f2f540080ce6325b0e699b834b6e7507
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:35:58Z

wip

commit 52fc74f89f1570436e8d4734df4190806eef0ad1
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:39:53Z

Deprecate SqlConnectorConfiguration

commit 2f39d7fe2146f448296b2e919ba4189e170febe7
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:49:03Z

Adding tests

commit 4fa3d779410cf641fe69425438c386cea37051ed
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:50:11Z

wip

commit a8a66bd7c4c9594861d5d8736e7b3c49ee4f81bf
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:56:02Z

Fixing tests

commit 28333ff2c8cba88cc57ab33e559dae5d36feb9df
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:56:53Z

wip

commit ee8ed3954c7ae24e7a4b1f1e4ba8db03cdc8eac2
Author: Pavel Tupitsyn 
Date:   2017-09-19T09:59:10Z

wip

commit 90bcd95a79611f678688528de74245cdb3ddcfe2
Author: Pavel Tupitsyn 
Date:   2017-09-19T10:40:19Z

Merge branch 'master' into ignite-6399




---

[jira] [Created] (IGNITE-6436) ransactions slow down

[Ksenia Rybakova (JIRA)]

Ksenia Rybakova created IGNITE-6436:
---

 Summary: ransactions slow down
 Key: IGNITE-6436
 URL: https://issues.apache.org/jira/browse/IGNITE-6436
 Project: Ignite
  Issue Type: Bug
Reporter: Ksenia Rybakova






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: MVCC configuration

[Yakov Zhdanov]

Guys, I we should additionally provide ability to manually switch mvcc
coordinator via MBean passing order or ID of a new one. We already have all
the machinery for this.

--Yakov

Ignite Enhancement Proposal process

[Vladimir Ozerov]

Igniters,

I'd like to discuss an idea of adding "Enhancement Proposal" concept to our
process. Many other OSS vendors use it with great success ([1], [2], [3],
[4]), so I think we can also benefit from it.

**Motivation**
Ignite project lacks transparency. We have a lot of thoughts and plans in
our heads. Some of them are materialized to tickets and discussions, some
don't. And yet there is no single place where one can understand major
features and challenges of the product for the nearest perspective. We do
not understand our own roadmap.

Another problem is that our WIKI is full of trash - lots and lots of
outdated design documents and discussions.

With Ignite Enhancement Proposal (IEP) process we can move all major
changes to a single place, thus increasing our understanding of the product
and community involvement.

**Proposal**
1) Create separate page on WIKI [5] where process flow will be defined
2) Create sections for active and inactive/rejected proposals
3) Every proposal should have separate page with the following fields:
- ID
- Summary
- Author
- Sponsor/shepherd/etc - committer or PMC member who will help author drive
the process
- Status (DRAFT, ACTIVE, COMPLETED, REJECTED)
- "Motivation" section
- "Description" section where actual design will reside
- "Risks and Assumptions" section
- Links to external resources, dev-list discussions and JIRA tickets
4) Sponsor is responsible for keeping the page up to date
5) Discussions should happen outside of WIKI - on the dev-list or inside
JIRA tickets
6) Relevant JIRA tickets will be tracked with special labels, e.g. "iep-N"
[6]

I created sample page for binary format improvements (still raw enough) [7].

Please share your thoughts.

Vladimir.

[1] https://www.python.org/dev/peps/
[2]
https://hazelcast.atlassian.net/wiki/spaces/COM/pages/27558010/Hazelcast+Enhancement+Proposals
[3] https://github.com/Kotlin/KEEP
[4] https://spark.apache.org/improvement-proposals.html
[5]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=73638545
[6] https://issues.apache.org/jira/browse/IGNITE-6057
[7]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-1%3A+Bulk+data+loading+performance+improvements

Re: MVCC configuration

[Vladimir Ozerov]

What I mean is that it might be not applicable for DML by design. E.g. may
be we will have to fallback to per-memory-policy approach, or to
per-cache-group. As we do not know it at the moment and there is no clear
demand from users, I would simply put it aside to avoid in mess in public
API in future.

Moreover, per-cache flag raises additional questions which can be put out
of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
single transaction? If yes, what is the contract? Without fine-grained
per-cache approach in the first iteration we can avoid answering it.

On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov 
wrote:

> If it is not valid for DML then we can easily detect this situation and
> throw exception, but if I do not use DML why non make it configurable
> per-cache?
>
> On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov 
> wrote:
>
> > I would say that per-cache configuration should be out of scope as well
> for
> > the first iteration. Because we do not know whether it will be valid for
> > DML.
> >
> > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov 
> > wrote:
> >
> > > Folks, thank you for feedback, I want to summarize some decisions:
> > >
> > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > per-cache flag - CacheConfiguration.isMvccEnabled, default value for
> all
> > > caches - IgniteConfiguration.isMvccEnabled.
> > > 2. For initial implementation mvcc for ATOMIC cache is out of scope, it
> > can
> > > be enabled only for TRANSACTIONAL caches.
> > > 3. Mvcc coordinator can be any server node (oldest server node is
> > selected
> > > automatically). Also I believe we need possibility to have *dedicated*
> > mvcc
> > > coordinator nodes which will process only internal mvcc messages. Node
> > can
> > > be marked as dedicated coordinator via new flag
> > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > MvccConfiguration bean. But let's skip this decision for now before we
> > have
> > > benchmarks numbers.
> > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > >
> > >
> > > Thanks
> > >
> > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > wrote:
> > >
> > > > This could be something like "preferredMvccCoordinator".
> > > >
> > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com> wrote:
> > > >
> > > > > >
> > > > > > I agree that we need coordinator nodes, but I do not understand
> why
> > > > can't
> > > > > > we reuse some cache nodes for it? Why do we need to ask user to
> > start
> > > > up
> > > > > > yet another type of node?
> > > > > >
> > > > >
> > > > > Dmitriy,
> > > > >
> > > > > My understanding is that Semyon does not deny a cache node to be
> used
> > > as
> > > > a
> > > > > coordinator. This property will allow to optionally have a
> > *dedicated*
> > > > node
> > > > > serving as a coordinator to improve cluster throughput under heavy
> > > load.
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-6435) Web Console: Add release version to footer

[Dmitriy Shabalin (JIRA)]

Dmitriy Shabalin created IGNITE-6435:


 Summary: Web Console: Add release version to footer
 Key: IGNITE-6435
 URL: https://issues.apache.org/jira/browse/IGNITE-6435
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Dmitriy Shabalin
Assignee: Dmitriy Shabalin






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: MVCC configuration

[Semyon Boikov]

If it is not valid for DML then we can easily detect this situation and
throw exception, but if I do not use DML why non make it configurable
per-cache?

On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov 
wrote:

> I would say that per-cache configuration should be out of scope as well for
> the first iteration. Because we do not know whether it will be valid for
> DML.
>
> On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov 
> wrote:
>
> > Folks, thank you for feedback, I want to summarize some decisions:
> >
> > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
> > caches - IgniteConfiguration.isMvccEnabled.
> > 2. For initial implementation mvcc for ATOMIC cache is out of scope, it
> can
> > be enabled only for TRANSACTIONAL caches.
> > 3. Mvcc coordinator can be any server node (oldest server node is
> selected
> > automatically). Also I believe we need possibility to have *dedicated*
> mvcc
> > coordinator nodes which will process only internal mvcc messages. Node
> can
> > be marked as dedicated coordinator via new flag
> > IgniteConfiguration.isMvccCoordinator or we can add separate
> > MvccConfiguration bean. But let's skip this decision for now before we
> have
> > benchmarks numbers.
> > 4. Need add some metrics to monitor mvcc coordinator performance.
> >
> >
> > Thanks
> >
> > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov 
> > wrote:
> >
> > > This could be something like "preferredMvccCoordinator".
> > >
> > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com> wrote:
> > >
> > > > >
> > > > > I agree that we need coordinator nodes, but I do not understand why
> > > can't
> > > > > we reuse some cache nodes for it? Why do we need to ask user to
> start
> > > up
> > > > > yet another type of node?
> > > > >
> > > >
> > > > Dmitriy,
> > > >
> > > > My understanding is that Semyon does not deny a cache node to be used
> > as
> > > a
> > > > coordinator. This property will allow to optionally have a
> *dedicated*
> > > node
> > > > serving as a coordinator to improve cluster throughput under heavy
> > load.
> > > >
> > >
> >
>

Re: MVCC configuration

[Vladimir Ozerov]

I would say that per-cache configuration should be out of scope as well for
the first iteration. Because we do not know whether it will be valid for
DML.

On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov 
wrote:

> Folks, thank you for feedback, I want to summarize some decisions:
>
> 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
> caches - IgniteConfiguration.isMvccEnabled.
> 2. For initial implementation mvcc for ATOMIC cache is out of scope, it can
> be enabled only for TRANSACTIONAL caches.
> 3. Mvcc coordinator can be any server node (oldest server node is selected
> automatically). Also I believe we need possibility to have *dedicated* mvcc
> coordinator nodes which will process only internal mvcc messages. Node can
> be marked as dedicated coordinator via new flag
> IgniteConfiguration.isMvccCoordinator or we can add separate
> MvccConfiguration bean. But let's skip this decision for now before we have
> benchmarks numbers.
> 4. Need add some metrics to monitor mvcc coordinator performance.
>
>
> Thanks
>
> On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov 
> wrote:
>
> > This could be something like "preferredMvccCoordinator".
> >
> > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > alexey.goncha...@gmail.com> wrote:
> >
> > > >
> > > > I agree that we need coordinator nodes, but I do not understand why
> > can't
> > > > we reuse some cache nodes for it? Why do we need to ask user to start
> > up
> > > > yet another type of node?
> > > >
> > >
> > > Dmitriy,
> > >
> > > My understanding is that Semyon does not deny a cache node to be used
> as
> > a
> > > coordinator. This property will allow to optionally have a *dedicated*
> > node
> > > serving as a coordinator to improve cluster throughput under heavy
> load.
> > >
> >
>

[jira] [Created] (IGNITE-6434) Assertion error in checkpointer during topology change

[Eduard Shangareev (JIRA)]

Eduard Shangareev created IGNITE-6434:
-

 Summary: Assertion error in checkpointer during topology change
 Key: IGNITE-6434
 URL: https://issues.apache.org/jira/browse/IGNITE-6434
 Project: Ignite
  Issue Type: Bug
Reporter: Eduard Shangareev
Assignee: Eduard Shangareev
Priority: Critical


{code}
2017-09-11 21:35:22.195 
[ERROR][db-checkpoint-thread-#101%node1%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
 Runtime error caught during grid runnable execution: GridWorker 
[name=db-checkpoint-thread, igniteInstanceName=cache1, finished=false, 
hashCode=1326137503, interrupted=false, runner=db-checkpoint-thread-#101%node1%]
java.lang.IllegalStateException: Failed to add new partition to the partitions 
state (no enough space reserved) [partId=459, reserved=459]
at 
org.apache.ignite.internal.pagemem.wal.record.CacheState.addPartitionState(CacheState.java:50)
 ~[ignite-core-2.1.4.jar:2.1.4]
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:2189)
 ~[ignite-core-2.1.4.jar:2.1.4]
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:1954)
 ~[ignite-core-2.1.4.jar:2.1.4]
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:1879)
 ~[ignite-core-2.1.4.jar:2.1.4]
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) 
~[ignite-core-2.1.4.jar:2.1.4]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_121]
{code}

After checkpoint thread died.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: MVCC configuration

[Semyon Boikov]

Folks, thank you for feedback, I want to summarize some decisions:

1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
caches - IgniteConfiguration.isMvccEnabled.
2. For initial implementation mvcc for ATOMIC cache is out of scope, it can
be enabled only for TRANSACTIONAL caches.
3. Mvcc coordinator can be any server node (oldest server node is selected
automatically). Also I believe we need possibility to have *dedicated* mvcc
coordinator nodes which will process only internal mvcc messages. Node can
be marked as dedicated coordinator via new flag
IgniteConfiguration.isMvccCoordinator or we can add separate
MvccConfiguration bean. But let's skip this decision for now before we have
benchmarks numbers.
4. Need add some metrics to monitor mvcc coordinator performance.


Thanks

On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov 
wrote:

> This could be something like "preferredMvccCoordinator".
>
> On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
> > >
> > > I agree that we need coordinator nodes, but I do not understand why
> can't
> > > we reuse some cache nodes for it? Why do we need to ask user to start
> up
> > > yet another type of node?
> > >
> >
> > Dmitriy,
> >
> > My understanding is that Semyon does not deny a cache node to be used as
> a
> > coordinator. This property will allow to optionally have a *dedicated*
> node
> > serving as a coordinator to improve cluster throughput under heavy load.
> >
>

[GitHub] ignite pull request #2693: IGNITE-6244 .NET: Thin client: ScanQuery

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2693


---

[GitHub] ignite pull request #2648: ignite-4931 tests reworked

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2648


---

[GitHub] ignite pull request #2695: Ignite 2.1.5-IGN-8173 remove locDepOwner

[DmitriyGovorukhin]

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/2695

Ignite 2.1.5-IGN-8173 remove locDepOwner 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.1.5-IGN-8173

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2695.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2695


commit d1a74a4be8744528e6ed23706174041ddb0f2618
Author: devozerov 
Date:   2017-08-04T09:04:38Z

Merge remote-tracking branch 'upstream/ignite-2.1.4' into ignite-2.1.4

commit 0b3a9a7176f5ae44a96ecf700c8147193dfbf064
Author: Igor Sapego 
Date:   2017-08-04T10:18:00Z

IGNITE-5923: ODBC: SQLGetTypeInfo now works with SQL_ALL_TYPES

(cherry picked from commit 48c914d)

commit 4e0385fbc0f50548f2da3407fdfdfe939b463c67
Author: Igor Sapego 
Date:   2017-08-04T15:34:27Z

IGNITE-5939: ODBC: SQLColAttributes now works with legacy attribute codes.

(cherry picked from commit 70ffa2c)

commit 4f02504475fd1e5cc3b9f4754856e44d20fdc1cb
Author: Alexey Kuznetsov 
Date:   2017-08-07T02:41:22Z

Merge branch ignite-2.1.3 into ignite-2.1.4.

commit b093afb8231135a4904f5fffd62f5b4c332f1d47
Author: tledkov-gridgain 
Date:   2017-08-08T09:05:36Z

IGNITE-5211: Added new constructor: QueryEntity(Class keyCls, Class 
valCls). This closes #2371. This closes #2388. This closes #2407.

commit 879f19106b22e66d5f6ea94424d961d049397410
Author: devozerov 
Date:   2017-08-08T12:16:58Z

IGNITE-5982: GridMapQueryExecutor was split into several pieces.

commit 7c77b869bc3efdf19e53cc2b064f4290fd73e2b2
Author: devozerov 
Date:   2017-08-09T08:47:58Z

IGNITE-5993: Removed unused SQL-related classes and methods (old tree 
index, snapshots, etc). This closes #2414.

commit ab18fdfcc4f6db1e54fb1f3b68ba7fbc31a7f6e7
Author: Andrey V. Mashenkov 
Date:   2017-07-14T17:14:47Z

IGNITE-5452: GridTimeoutProcessor can hang on stop. This closes #2279.

commit 580b6aa8e5a8a887397eab5c4c830ec28f45cd30
Author: Alexey Kuznetsov 
Date:   2017-08-09T10:22:54Z

IGNITE-5734 Web Console: Fixed npm dependencies.
(cherry picked from commit aeafbf1)

commit 841db65e56063605475710bc170de4aea672c31d
Author: Alexey Kuznetsov 
Date:   2017-08-09T11:55:04Z

IGNITE-5987 Added -nq (visor will not quit in batch mode) option for Visor 
Cmd.
(cherry picked from commit 8d6e842)

commit 5c2097856714a7803956d754735c68b21156019c
Author: Alexey Kuznetsov 
Date:   2017-08-11T03:25:36Z

IGNITE-5902 Implemented stop caches at once.
(cherry picked from commit ebb8765)

commit 8b2461942c18f228c0107020aa28c03711bdceda
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:07:26Z

IGNITE-6012 Refactored GridJettyRestHandler.processRequest(): replace 
mapper.writeValueAsString with writeValue(stream, v).
(cherry picked from commit 3a390c8)

commit 3a7d4f4a79e7c0a23244387e3a68535375a66a87
Author: Alexey Kuznetsov 
Date:   2017-08-11T04:18:42Z

IGNITE-6013 Optimized processing response from cluster.
(cherry picked from commit b02c481)

commit 74d6ab9916b3a01c78cdf1ad86211c9fcbb2214d
Author: Alexey Goncharuk 
Date:   2017-08-14T08:08:28Z

Merge branch 'ignite-2.1.3' into ignite-2.1.4

commit fde550bac56fd0cc7c51c62a9c291dd4c3f3030c
Author: Ilya Lantukh 
Date:   2017-08-14T08:32:11Z

IGNITE-5941 - Fixed index name length restrictions. This closes #2408

commit 13f38d79b57b395e43d42a8f3c278cf48336d7c5
Author: Dmitriy Govorukhin 
Date:   2017-08-14T09:12:46Z

IGNITE-5890 Added estimated time to rebalance completion and time to 
rebalance start to MXBean - Fixes #2386.

commit c23a2dcfb1395e87cb4e14457a053c6b4727b318
Author: Dmitriy Govorukhin 
Date:   2017-08-14T13:33:12Z

IGNITE-5741 - Replaced HeapByteBuffer with DirectByteBuffer in WAL records 
iterator - Fixes #2329.

Signed-off-by: Alexey Goncharuk 

commit 2f38065cd10fd61d51771d14188380dc7cc74ed7
Author: Ivan Rakov 
Date:   2017-08-14T13:44:50Z

GG-12629 Backport IGNITE-5961 to 8.1.4

commit cdac5a87cb1432ffd0ec32a2888505805e7348da
Author: EdShangGG 
Date:   2017-08-14T13:56:11Z

IGNITE-5843 Persist cache configuration received on node join - Fixes #2347.

commit 305c0f4ffb745bdc04cd0a6f3b45dbd9ff5da302
Author: Dmitriy Govorukhin 
Date:   

[jira] [Created] (IGNITE-6433) We need to cancel eviction instead of waiting it when we should own a partition because we had lost it

[Eduard Shangareev (JIRA)]

Eduard Shangareev created IGNITE-6433:
-

 Summary: We need to cancel eviction instead of waiting it when we 
should own a partition because we had lost it
 Key: IGNITE-6433
 URL: https://issues.apache.org/jira/browse/IGNITE-6433
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.1
Reporter: Eduard Shangareev
 Fix For: 2.3


If PartitionLossPolicy.IGNORE is used and we have lost some partition which 
would belong to us because of affinity assignment and its state was RENTING 
then we would wait for its eviction completing what would hang cluster (the 
time of exchange would significantly increase).

Instead of waiting we should cancel eviction and it's all.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2572: IGNITE-6238 Fix GridClosureProcessorRemoteTest, a...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2572


---

[GitHub] ignite pull request #2694: Ignite 6342 2

[EdShangGG]

GitHub user EdShangGG opened a pull request:

https://github.com/apache/ignite/pull/2694

Ignite 6342 2



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6342-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2694.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2694


commit 08d09903f67e173b1743169729c29cf379079e8c
Author: Eduard Shangareev 
Date:   2017-09-19T07:17:16Z

IGNITE-6342 Exchange is hanging on eviction
-reproduction test
-small fixes

commit f4a09c7e2e2be936d9fa20d27d28a2ba2177e817
Author: Eduard Shangareev 
Date:   2017-09-19T08:02:38Z

IGNITE-6342 Exchange is hanging on eviction
-fixing issue with renting partition size after restart
-fixing issue with owning partition after it was lost




---

[GitHub] ignite pull request #2693: IGNITE-6244 .NET: Thin client: ScanQuery

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/2693

IGNITE-6244 .NET: Thin client: ScanQuery



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6244

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2693.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2693


commit bb6f3fa8d6a634f2112d5764f4548a3860ee244e
Author: Pavel Tupitsyn 
Date:   2017-08-29T15:57:07Z

IGNITE-5899 Thin client: cache.Get for primitives

This closes #2376

commit 085cc80e5337ff01f7173342b0fa913392aa90a6
Author: Pavel Tupitsyn 
Date:   2017-08-30T15:58:57Z

IGNITE-5905 .NET: Thin client: cache.Get for primitives

This closes #2542

commit 4241f5d2cd5a724c3f781de86d472480a2ce1e85
Author: Pavel Tupitsyn 
Date:   2017-08-31T15:13:08Z

Merge branch 'master' into ignite-5896

# Conflicts:
#   modules/platforms/dotnet/Apache.Ignite.Core/Apache.Ignite.Core.csproj

commit 50f6bdcb264654d38570ac547948b52f9bde92b6
Author: Pavel Tupitsyn 
Date:   2017-09-04T14:38:38Z

IGNITE-6236 .NET: Thin client: cache.Get and Put for user types

This closes #2580

commit 4f8f08522af1595582552da8348a1f31cf5b5192
Author: Pavel Tupitsyn 
Date:   2017-09-05T08:52:31Z

Merge branch 'master' into ignite-5896

# Conflicts:
#   modules/platforms/dotnet/Apache.Ignite.Core/Apache.Ignite.Core.csproj

commit 29bc29ec561a11501a21a3cb88135b9442ed2e0a
Author: Pavel Tupitsyn 
Date:   2017-09-05T09:56:52Z

IGNITE-5896 Organize operations

commit 6533e1d12409ccec2da494dde92cffcfaf549bc6
Author: Pavel Tupitsyn 
Date:   2017-09-06T14:16:13Z

Merge branch 'master' into ignite-5896

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/odbc/SqlListenerNioListener.java
#   modules/platforms/dotnet/Apache.Ignite.Core/Apache.Ignite.Core.csproj

commit de7d421c1705457406863041c096f8bcee74da9d
Author: Pavel Tupitsyn 
Date:   2017-09-06T14:27:27Z

Merge branch 'master' into ignite-5896

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/odbc/SqlListenerNioListener.java
#   modules/platforms/dotnet/Apache.Ignite.Core/Apache.Ignite.Core.csproj

commit b70f05892938282b4cb77ee5c14c7458bbd36fe2
Author: Pavel Tupitsyn 
Date:   2017-09-06T14:31:37Z

Merge branch 'master' into ignite-5896

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/odbc/SqlListenerNioListener.java
#   modules/platforms/dotnet/Apache.Ignite.Core/Apache.Ignite.Core.csproj

commit 6aeed1f1e2840eeec9739d25b604b490f43e38d6
Author: Pavel Tupitsyn 
Date:   2017-09-06T14:44:56Z

Fix protocol version in tests

commit 38047fbc477afb6f4df7cd047c9ba483bcb3054c
Author: Pavel Tupitsyn 
Date:   2017-09-08T13:59:20Z

Merge branch 'master' into ignite-5896

commit bc7bb37d5f318c65594deb2e0f4750fad6711223
Author: Pavel Tupitsyn 
Date:   2017-09-08T14:14:29Z

IGNITE-6244 .NET: Thin client: ScanQuery

commit f10d7a79cb2552b1463a64cdbeed89549d1b8c84
Author: Pavel Tupitsyn 
Date:   2017-09-08T14:25:52Z

Remove unused imports

commit 628aaf6b2e6fad16c5804165b7bfb4c42375e390
Author: Pavel Tupitsyn 
Date:   2017-09-08T14:32:54Z

Cleanup

commit d4021b56c90565e9344652d96c6eac77f595f838
Author: devozerov 
Date:   2017-09-11T07:45:11Z

Merge branch 'master' into ignite-5896

commit c4970f59e8b09b881ed04bbe2bc45b98f2d4e19d
Author: devozerov 
Date:   2017-09-11T07:45:50Z

Merge branch 'ignite-5896' into ignite-6244

commit bf4401bb30f3ef69477fdb72bb8159f57d9fc64b
Author: Pavel Tupitsyn 
Date:   2017-09-11T10:55:08Z

Close all cursors on disconnect

commit 5cd12f7ebbba83e3dc148520813cf21d502e70a5
Author: Pavel Tupitsyn 
Date:   2017-09-11T11:17:00Z

Remove GetAll support

commit 576a71cb2d905c1d42695d92986cc68d1dc0d755
Author: Pavel Tupitsyn 
Date:   2017-09-11T11:39:09Z

Rework tests for GetAll

commit 070fdfa7877353d503ea95ac8123a873eafe290e
Author: Pavel Tupitsyn 
Date:   2017-09-11T12:01:11Z

Refactor cursor paging

commit 2a026de9641dd9b8f40646309635c2a4cd145fc5
Author: Pavel Tupitsyn 
Date:   2017-09-11T12:05:25Z

Include first page in scan query response

commit 232aad7bd84200d3bb810b09d32e079d82922e16
Author: Pavel Tupitsyn 

Re: MVCC configuration

[Vladimir Ozerov]

This could be something like "preferredMvccCoordinator".

On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> >
> > I agree that we need coordinator nodes, but I do not understand why can't
> > we reuse some cache nodes for it? Why do we need to ask user to start up
> > yet another type of node?
> >
>
> Dmitriy,
>
> My understanding is that Semyon does not deny a cache node to be used as a
> coordinator. This property will allow to optionally have a *dedicated* node
> serving as a coordinator to improve cluster throughput under heavy load.
>

Re: Exception handling in thin client: should we pass stack traces to the client?

[Vladimir Ozerov]

Thanks, folks!

Excellent catch - we should not decide whether to allow it or not, but
rather make it configurable.

On Tue, Sep 19, 2017 at 10:43 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> I think the protocol should allow both, and the behavior should be either
> configurable or enabled via a system property. Every web server known to me
> allows exposing this information for debugging purposes.
>
> 2017-09-19 10:20 GMT+03:00 Vladimir Ozerov :
>
> > Igniters,
> >
> > We had a discussion about how to propagate error information from cluster
> > nodes to the client. My opinion is that we should pass a kind of vendor
> > code plus optional error message, if vendor code is not very specific.
> >
> > Alternative idea is to pass the whole stack trace as well. I agree that
> > this is very useful for debugging purposes, but on the other hand IMO it
> > imposes security risk. By sending invalid requests to the server user
> might
> > get sensitive information about server configuration, such as it's
> version,
> > version of the underlying database, frameworks etc.. This information may
> > help attacker to apply some version-specific attacks. This is precise
> > reason why default error pages of web servers with stack traces are
> always
> > replaces with some stubs.
> >
> > This is why I think we should not include stack traces.
> >
> > What do you think?
> >
> > Vladimir.
> >
>

Re: Exception handling in thin client: should we pass stack traces to the client?

[Dmitry Pavlov]

Hi Vladimir,

All of these arguments are relevant. I suggest to provide full stacktrace
at least by server option. This is also common practice on web servers.

Sincerely,
Dmitriy Pavlov

вт, 19 сент. 2017 г. в 10:20, Vladimir Ozerov :

> Igniters,
>
> We had a discussion about how to propagate error information from cluster
> nodes to the client. My opinion is that we should pass a kind of vendor
> code plus optional error message, if vendor code is not very specific.
>
> Alternative idea is to pass the whole stack trace as well. I agree that
> this is very useful for debugging purposes, but on the other hand IMO it
> imposes security risk. By sending invalid requests to the server user might
> get sensitive information about server configuration, such as it's version,
> version of the underlying database, frameworks etc.. This information may
> help attacker to apply some version-specific attacks. This is precise
> reason why default error pages of web servers with stack traces are always
> replaces with some stubs.
>
> This is why I think we should not include stack traces.
>
> What do you think?
>
> Vladimir.
>

Re: Exception handling in thin client: should we pass stack traces to the client?

[Alexey Goncharuk]

I think the protocol should allow both, and the behavior should be either
configurable or enabled via a system property. Every web server known to me
allows exposing this information for debugging purposes.

2017-09-19 10:20 GMT+03:00 Vladimir Ozerov :

> Igniters,
>
> We had a discussion about how to propagate error information from cluster
> nodes to the client. My opinion is that we should pass a kind of vendor
> code plus optional error message, if vendor code is not very specific.
>
> Alternative idea is to pass the whole stack trace as well. I agree that
> this is very useful for debugging purposes, but on the other hand IMO it
> imposes security risk. By sending invalid requests to the server user might
> get sensitive information about server configuration, such as it's version,
> version of the underlying database, frameworks etc.. This information may
> help attacker to apply some version-specific attacks. This is precise
> reason why default error pages of web servers with stack traces are always
> replaces with some stubs.
>
> This is why I think we should not include stack traces.
>
> What do you think?
>
> Vladimir.
>

Re: MVCC configuration

[Alexey Goncharuk]

>
> I agree that we need coordinator nodes, but I do not understand why can't
> we reuse some cache nodes for it? Why do we need to ask user to start up
> yet another type of node?
>

Dmitriy,

My understanding is that Semyon does not deny a cache node to be used as a
coordinator. This property will allow to optionally have a *dedicated* node
serving as a coordinator to improve cluster throughput under heavy load.

Exception handling in thin client: should we pass stack traces to the client?

[Vladimir Ozerov]

Igniters,

We had a discussion about how to propagate error information from cluster
nodes to the client. My opinion is that we should pass a kind of vendor
code plus optional error message, if vendor code is not very specific.

Alternative idea is to pass the whole stack trace as well. I agree that
this is very useful for debugging purposes, but on the other hand IMO it
imposes security risk. By sending invalid requests to the server user might
get sensitive information about server configuration, such as it's version,
version of the underlying database, frameworks etc.. This information may
help attacker to apply some version-specific attacks. This is precise
reason why default error pages of web servers with stack traces are always
replaces with some stubs.

This is why I think we should not include stack traces.

What do you think?

Vladimir.

Re: Ignite PDS WAL analysis with human readable records

[Dmitry Pavlov]

I've updated issue description.

Igniters,

who have already faced with necessity to look into WAL content and/or
latest records?

Sincerely,
Dmitriy Pavlov

пн, 18 сент. 2017 г. в 23:23, Dmitriy Setrakyan :

> Dmitriy, can you please describe your idea in the ticket?
>
> Here is the link for easier access:
> https://issues.apache.org/jira/browse/IGNITE-6421
>
> D.
>
> On Mon, Sep 18, 2017 at 7:02 AM, Dmitry Pavlov 
> wrote:
>
> > Dmitriy,
> >
> > Thank you for your feedback I've created IGNITE-6421 for this change.
> >
> > Is it common practice to use property file in Ignite utils? I'd like to
> > support parse and display data entries and its fields, but it is required
> > to configure several folders: 'binary_meta' and 'marshaller' folders.
> >
> > My idea is to provide .conf file to configure reader once.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 18 сент. 2017 г. в 16:10, Dmitriy Setrakyan :
> >
> > > Thanks!
> > >
> > > After looking at the ticket, the way to launch this utility seems
> > complex.
> > > Does this utility have a shell script? If not, we should add one with
> > > proper command line options, like we do for other scripts.
> > >
> > > D.
> > >
> > > On Mon, Sep 18, 2017 at 4:05 AM, Eduard Shangareev <
> > > eduard.shangar...@gmail.com> wrote:
> > >
> > > > Example of output (will add to wiki):
> > > >
> > > > [W] InitNewPageRecord [newPageId=0001001c0067,
> > super=PageDeltaRecord
> > > > [grpId=2141373875, pageId=0001001c0067, super=WALRecord [size=41,
> > > > chainSize=0, pos=FileWALPointer [idx=1, fileOffset=25406, len=41,
> > > > forceFlush=false], type=INIT_NEW_PAGE_RECORD]]]
> > > > [W] InitNewPageRecord [newPageId=0001001c0068,
> > super=PageDeltaRecord
> > > > [grpId=2141373875, pageId=0001001c0068, super=WALRecord [size=41,
> > > > chainSize=0, pos=FileWALPointer [idx=1, fileOffset=25447, len=41,
> > > > forceFlush=false], type=INIT_NEW_PAGE_RECORD]]]
> > > > [W] PagesListAddPageRecord [dataPageId=0001001c002a,
> > > > super=PageDeltaRecord [grpId=2141373875, pageId=0001001c0068,
> > > > super=WALRecord [size=37, chainSize=0, pos=FileWALPointer [idx=1,
> > > > fileOffset=25488, len=37, forceFlush=false],
> > type=PAGES_LIST_ADD_PAGE]]]
> > > > [W] PageSnapshot [fullPageId = FullPageId [pageId=0001001c002a,
> > > > effectivePageId=001c002a, grpId=2141373875], page = [
> > > > Header [
> > > > type=1 (DataPageIO),
> > > > ver=1,
> > > > crc=0,
> > > > pageId=281595235794986(offset=0, flags=1, partId=28, index=42)
> > > > ],
> > > > DataPageIO [
> > > > 0001001c002a [4049, 4002, 3955, 3908, 3861, 3814, 3767, 3720,
> 3673,
> > > > 3626][][free=3552, actualFree=-544
> > > > ]],
> > > > super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer
> [idx=1,
> > > > fileOffset=25525, len=4125, forceFlush=false], type=PAGE_RECORD]]]
> > > > [W] InsertRecord [idx=12, io=CacheIdAwareDataLeafIO[ver=1],
> > > > rightId=, super=PageDeltaRecord [grpId=2141373875,
> > > > pageId=0001001c0004, super=WALRecord [size=59, chainSize=0,
> > > > pos=FileWALPointer [idx=1, fileOffset=29650, len=59,
> forceFlush=false],
> > > > type=BTREE_PAGE_INSERT]]]
> > > > [W] DataRecord [writeEntries=[DataEntry [cacheId=-1368047377,
> > op=CREATE,
> > > > writeVer=GridCacheVersion [topVer=116892426, order=1505412445450,
> > > > nodeOrder=2], partId=28, partCnt=69]], super=WALRecord [size=100,
> > > > chainSize=0, pos=FileWALPointer [idx=1, fileOffset=29709, len=100,
> > > > forceFlush=false], type=DATA_RECORD]]
> > > > [W] PageSnapshot [fullPageId = FullPageId [pageId=0001000a,
> > > > effectivePageId=000a, grpId=2141373875], page = [
> > > > Header [
> > > > type=20 (PagePartitionCountersIO),
> > > > ver=1,
> > > > crc=0,
> > > > pageId=281474976710666(offset=0, flags=1, partId=0, index=10)
> > > > ],
> > > > PagePartitionCounters [
> > > > count=1,
> > > > lastFlag=true,
> > > > nextCountersPageId=,
> > > > size={
> > > > -1368047377=313
> > > > }
> > > > ]],
> > > > super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer
> [idx=1,
> > > > fileOffset=29809, len=4125, forceFlush=false], type=PAGE_RECORD]]]
> > > > [W] PageSnapshot [fullPageId = FullPageId [pageId=000100020010,
> > > > effectivePageId=00020010, grpId=2141373875], page = [
> > > > Header [
> > > > type=20 (PagePartitionCountersIO),
> > > > ver=1,
> > > > crc=0,
> > > > pageId=281483566645264(offset=0, flags=1, partId=2, index=16)
> > > > ],
> > > > PagePartitionCounters [
> > > > count=1,
> > > > lastFlag=true,
> > > > nextCountersPageId=,
> > > > size={
> > > > -1368047377=313
> > > > }
> > > > ]],
> > > > super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer
> [idx=1,
> > > > fileOffset=33934, len=4125, forceFlush=false], type=PAGE_RECORD]]]
> > > > [W] PageSnapshot [fullPageId = FullPageId [pageId=00010005000d,
> > > > 

[jira] [Created] (IGNITE-6432) SpringCacheManager#getCacheNames() is empty

[Valeri Chibaev (JIRA)]

Valeri Chibaev created IGNITE-6432:
--

 Summary: SpringCacheManager#getCacheNames() is empty
 Key: IGNITE-6432
 URL: https://issues.apache.org/jira/browse/IGNITE-6432
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.1
Reporter: Valeri Chibaev


Application ignite started with org.apache.ignite.IgniteSpringBean, then 
SpringCacheManager created
{code}

  

  

  



{code}
If I'll check cacheManager.getCacheNames() - it is empty.
But if I check cacheManager.getCache("refdata.labelType") metrics and repeat 
call cacheManager.getCacheNames() - it is not empty.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6431) Web Console: "Partition loss policy" field is duplicated

[Dmitry Karachentsev (JIRA)]

Dmitry Karachentsev created IGNITE-6431:
---

 Summary: Web Console: "Partition loss policy" field is duplicated
 Key: IGNITE-6431
 URL: https://issues.apache.org/jira/browse/IGNITE-6431
 Project: Ignite
  Issue Type: Bug
  Components: UI
Affects Versions: 2.1
Reporter: Dmitry Karachentsev
Assignee: Vasiliy Sisko






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)