Re: Future of Ignite transactions

[Vladimir Ozerov]

Dima,

IgniteCache.withReadForUpdate() :-)

On Fri, Sep 29, 2017 at 3:29 AM, Dmitriy Setrakyan 
wrote:

> But what if blocking on reads is needed? Also, how about pessimistic
> read-only transactions? Do we plan to support them?
>
> D.
>
> On Thu, Sep 28, 2017 at 6:44 AM, Vladimir Ozerov 
> wrote:
>
> > Alex,
> >
> > MVCC concept and blocking/non-blocking behavior are not inter-related.
> You
> > can have both blocking and non-blocking writes with MVCC and without it.
> > The thing is that users *want* blocking behavior as it is easy to reason
> > about. This is why even MVCC-based RDBMS vendors (Oracle, PostgreSQL) do
> > block on writes.
> >
> > OPTIMSITIC mode and non-blocking approaches are for rare enthusiasts. It
> is
> > not in priority for us.
> >
> > On Thu, Sep 28, 2017 at 1:59 PM, Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> wrote:
> >
> > > Do we have a plan to support multi-version concurrency control ?
> > >
> > > Proper implementation allows to achieve better isolation without
> blocking
> > > for the cost of additional resource consumption.
> > >
> > > The feature can be configurable per transaction.
> > >
> > > 6) What if I want to acquire write locks only for some specific reads
> > > within same transaction ?
> > >
> > >
> > > 2017-09-27 5:52 GMT+03:00 Dmitriy Setrakyan :
> > >
> > > > OK, if we must change the current behavior, let's discuss the new
> > design.
> > > > My comments/questions are below...
> > > >
> > > > On Mon, Sep 25, 2017 at 12:09 PM, Vladimir Ozerov <
> > voze...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > Sorry for late reply. I had a chat with several Ignite veterans
> > today.
> > > We
> > > > > tried to design transactional SQL for Ignite. One of our questions
> > was
> > > > how
> > > > > to align SQL transactions with current Ignite transactions. And we
> > > > failed.
> > > > > And then we came to conclusion that current transaction API is
> > > unusable.
> > > > We
> > > > > have 6 pairs of modes from API standpoint and 4 real modes. This is
> > > very
> > > > > counterintuitive and cannot be mapped to any transactional
> framework
> > > our
> > > > > users are familiar with.
> > > > >
> > > > > So we thought how new tx API might looks like, and here is the
> draft.
> > > > >
> > > > > 1) Define new enum *TransactionIsolationLevel *(to avoid clashes
> with
> > > > > current enum) with three standard modes - READ_COMMITTED,
> > > > REPEATABLE_READ,
> > > > > SERIALIZABLE.
> > > > >
> > > >
> > > > If it is the same values as we have today, why create a new enum?
> > > >
> > > >
> > > > > 2) Define new enum *TransactionHint* - READ_ONLY,
> OPTIMISTIC_LOCKING
> > > > >
> > > >
> > > > The word *hint* means no guarantee. When it comes to transactions, we
> > > must
> > > > have guarantees. Also, what happens to PESSIMISTIC locking?
> > > >
> > > >
> > > > > 3) No more OPTIMISTIC and PESSIMISTIC. Seriously.
> > > > >
> > > >
> > > > But you are still proposing OPTIMISTIC_LOCKING above. So, we have
> > > > OPTIMISTIC without PESSIMISTIC?
> > > >
> > > >
> > > > > 4) Reads never acuire locks
> > > > > 5) Writes always acquire locks
> > > > >
> > > >
> > > > Hm... what locks? I think we are getting to deep into the weeds here.
> > > Isn't
> > > > it our internal implementation detail to acquire locks or not? For
> > > example,
> > > > MVCC approach we are working on will be virtually lock-free for all
> > > > scenarios.
> > > >
> > > > I would focus on transaction behavior, not locks.
> > > >
> > > >
> > > >
> > > > > 6) *IgniteCache.withReadForUpdate()* will return special facade
> > which
> > > > will
> > > > > obtain locks on reads. This is analogue of SELECT FOR UPDATE in
> SQL.
> > > > >
> > > >
> > > > This syntax is very ugly in databases and we do not need to carry
> this
> > > over
> > > > to Ignite. All it means is that a pessimistic lock is acquired. To be
> > > > honest, I would rather start a pessimistic transaction to get the
> same
> > > > behavior.
> > > >
> > > >
> > > > > 7) *TransactionHint.READ_ONLY* - forces transaction to throw an
> > > exception
> > > > > on any update
> > > > >
> > > >
> > > > I really like the READ_ONLY flag. Allows us to put many optimizations
> > in
> > > > place.
> > > >
> > > >
> > > > > 8) *TransactionHint.OPTIMISTIC_LOCKING* - turns transaction into
> our
> > > > > current deadlock-free OPTIMISTIC/SERIALIZABLE mode. Applicable only
> > to
> > > > > SERIALIZABLE isolation level.
> > > > >
> > > >
> > > > Is there a PESSIMISTIC/SERIALIZABLE mode?
> > > >
> > > >
> > > > > 9) Define new API methods:
> > > > > - IgniteTransactions.txStart(TransactionIsolationLevel isolation)
> > > > > - IgniteTransactions.txStart(TransactionIsolationLevel isolation,
> > > > > TransactionHint... hints)
> > > >
> > > > 10) Deprecate old TX start methods
> > > > >
> > > > > As a result we will have simple, clean and extensible API. Which

Re: [DISCUSS] Ignite Update Checker

[Konstantin Boudnik]

Sorry guys - I neglected the fact that our lists don't permit
attachments. I have put the screenshot to an external server [1]

[1] https://imgur.com/a/p9FJ9

Thank you!
--
  With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.


On Thu, Sep 28, 2017 at 1:37 PM, Denis Magda  wrote:
> Cos,
>
> The screenshot was not attached. Could you share it some other way (google 
> drive, etc.)? I’ve never seen any commercial on the site.
>
> —
> Denis
>
>> On Sep 28, 2017, at 7:23 AM, Konstantin Boudnik  wrote:
>>
>> I don't see an issue with node version either.
>>
>> Just one more, and it might be slightly irrelevant, issue though... I looked 
>> at the Ignite's site and found the following ads and trackers (that are 
>> indeed getting disabled by my browser).
>> Why are googleads or doubleclick are permitted?
>>
>>
>>
>> Thanks,
>>   Cos
>>
>>
>> --
>>   With regards,
>> Konstantin (Cos) Boudnik
>> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>>
>> Disclaimer: Opinions expressed in this email are those of the author, and do 
>> not necessarily represent the views of any company the author might be 
>> affiliated with at the moment of writing.
>>
>> On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan > > wrote:
>> On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov > >
>> wrote:
>>
>> > Folks,
>> >
>> > Can we add version of current node to web request? This way we will better
>> > understand version distribution, what might help us with certain API
>> > update/deprecate decisions
>> > E.g. http://ignite.apache.org/latest.cgi=2.2.0 
>> > 
>>
>>
>> Vladimir, I personally do not see a problem with that, as sending the
>> current version to the update checker seems very innocent to me. At the
>> same time, it will allow us to examine the usage of each release and make
>> decisions about dropping backward compatibility or spotting bugs for a
>> certain release.
>>
>> Cos, Raul, any thoughts?
>>
>>
>> >
>> >
>> > Vladimir.
>> >
>> > On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan > > >
>> > wrote:
>> >
>> > > I think it is safe to assume at this point that everyone is in general
>> > > agreement, since there are no active objections.
>> > >
>> > > I have filed a ticket for the 2.3 release. Let's try to make it happen:
>> > > https://issues.apache.org/jira/browse/IGNITE-6305 
>> > > 
>> > >
>> > > D.
>> > >
>> > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>> > dsetrak...@apache.org >
>> > > wrote:
>> > >
>> > > >
>> > > >
>> > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani > > > > >
>> > > > wrote:
>> > > >
>> > > >> Yeah, I guess that's doable as well and requires less management
>> > effort
>> > > >> than my suggestion. We could use events [1] to store payload data
>> > (e.g.
>> > > >> IP,
>> > > >> version, etc.)
>> > > >
>> > > >
>> > > > Yes, we could use events or some other similar API provided by GA.
>> > > >
>> > > >
>> > > >> What the download page CGI developed in? PHP?
>> > > >>
>> > > >
>> > > > To be honest, no clue. I guess someone in the community can figure it
>> > > out:
>> > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html 
>> > > > 
>> > > >
>> > > >
>> > > >> However, I'm not sure whether storing this data in a 3rd party
>> > (Google)
>> > > is
>> > > >> compliant with the ASF policy. I guess it's no biggie, but if there's
>> > > >> doubt
>> > > >> in the PMC, it's better to ask legal@.
>> > > >
>> > > >
>> > > > I am not sure there is anything to ask about. The whole Ignite website
>> > is
>> > > > GA enabled, and all we are doing is accessing a standard web page from
>> > > the
>> > > > Ignite web site. The information gathered from GA is available only to
>> > > the
>> > > > Ignite PMC. Frankly, I think legal@ will be very confused by this
>> > > > question.
>> > > >
>> > > > Even ASF website itself uses GA: https://www.apache.org/ 
>> > > > 
>> > > > foundation/policies/privacy.html
>> > > >
>> > > >
>> > > >> I think Cos said it's OK; maybe Roman can pitch in.
>> > > >>
>> > > >
>> > > >  Sure, would be nice to hear from Roman as well.
>> > > >
>> > > >
>> > > >> Cheers.
>> > > >>
>> > > >> [1]
>> > > >> https://developers.google.com/analytics/devguides/collection 
>> > > >> 
>> > > >> /analyticsjs/events
>> > > 

Re: Future of Ignite transactions

[Dmitriy Setrakyan]

But what if blocking on reads is needed? Also, how about pessimistic
read-only transactions? Do we plan to support them?

D.

On Thu, Sep 28, 2017 at 6:44 AM, Vladimir Ozerov 
wrote:

> Alex,
>
> MVCC concept and blocking/non-blocking behavior are not inter-related. You
> can have both blocking and non-blocking writes with MVCC and without it.
> The thing is that users *want* blocking behavior as it is easy to reason
> about. This is why even MVCC-based RDBMS vendors (Oracle, PostgreSQL) do
> block on writes.
>
> OPTIMSITIC mode and non-blocking approaches are for rare enthusiasts. It is
> not in priority for us.
>
> On Thu, Sep 28, 2017 at 1:59 PM, Alexei Scherbakov <
> alexey.scherbak...@gmail.com> wrote:
>
> > Do we have a plan to support multi-version concurrency control ?
> >
> > Proper implementation allows to achieve better isolation without blocking
> > for the cost of additional resource consumption.
> >
> > The feature can be configurable per transaction.
> >
> > 6) What if I want to acquire write locks only for some specific reads
> > within same transaction ?
> >
> >
> > 2017-09-27 5:52 GMT+03:00 Dmitriy Setrakyan :
> >
> > > OK, if we must change the current behavior, let's discuss the new
> design.
> > > My comments/questions are below...
> > >
> > > On Mon, Sep 25, 2017 at 12:09 PM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > Sorry for late reply. I had a chat with several Ignite veterans
> today.
> > We
> > > > tried to design transactional SQL for Ignite. One of our questions
> was
> > > how
> > > > to align SQL transactions with current Ignite transactions. And we
> > > failed.
> > > > And then we came to conclusion that current transaction API is
> > unusable.
> > > We
> > > > have 6 pairs of modes from API standpoint and 4 real modes. This is
> > very
> > > > counterintuitive and cannot be mapped to any transactional framework
> > our
> > > > users are familiar with.
> > > >
> > > > So we thought how new tx API might looks like, and here is the draft.
> > > >
> > > > 1) Define new enum *TransactionIsolationLevel *(to avoid clashes with
> > > > current enum) with three standard modes - READ_COMMITTED,
> > > REPEATABLE_READ,
> > > > SERIALIZABLE.
> > > >
> > >
> > > If it is the same values as we have today, why create a new enum?
> > >
> > >
> > > > 2) Define new enum *TransactionHint* - READ_ONLY, OPTIMISTIC_LOCKING
> > > >
> > >
> > > The word *hint* means no guarantee. When it comes to transactions, we
> > must
> > > have guarantees. Also, what happens to PESSIMISTIC locking?
> > >
> > >
> > > > 3) No more OPTIMISTIC and PESSIMISTIC. Seriously.
> > > >
> > >
> > > But you are still proposing OPTIMISTIC_LOCKING above. So, we have
> > > OPTIMISTIC without PESSIMISTIC?
> > >
> > >
> > > > 4) Reads never acuire locks
> > > > 5) Writes always acquire locks
> > > >
> > >
> > > Hm... what locks? I think we are getting to deep into the weeds here.
> > Isn't
> > > it our internal implementation detail to acquire locks or not? For
> > example,
> > > MVCC approach we are working on will be virtually lock-free for all
> > > scenarios.
> > >
> > > I would focus on transaction behavior, not locks.
> > >
> > >
> > >
> > > > 6) *IgniteCache.withReadForUpdate()* will return special facade
> which
> > > will
> > > > obtain locks on reads. This is analogue of SELECT FOR UPDATE in SQL.
> > > >
> > >
> > > This syntax is very ugly in databases and we do not need to carry this
> > over
> > > to Ignite. All it means is that a pessimistic lock is acquired. To be
> > > honest, I would rather start a pessimistic transaction to get the same
> > > behavior.
> > >
> > >
> > > > 7) *TransactionHint.READ_ONLY* - forces transaction to throw an
> > exception
> > > > on any update
> > > >
> > >
> > > I really like the READ_ONLY flag. Allows us to put many optimizations
> in
> > > place.
> > >
> > >
> > > > 8) *TransactionHint.OPTIMISTIC_LOCKING* - turns transaction into our
> > > > current deadlock-free OPTIMISTIC/SERIALIZABLE mode. Applicable only
> to
> > > > SERIALIZABLE isolation level.
> > > >
> > >
> > > Is there a PESSIMISTIC/SERIALIZABLE mode?
> > >
> > >
> > > > 9) Define new API methods:
> > > > - IgniteTransactions.txStart(TransactionIsolationLevel isolation)
> > > > - IgniteTransactions.txStart(TransactionIsolationLevel isolation,
> > > > TransactionHint... hints)
> > >
> > > 10) Deprecate old TX start methods
> > > >
> > > > As a result we will have simple, clean and extensible API. Which can
> be
> > > > explained to users in 5 minutes, instead of current half an hour. And
> > > which
> > > > is perfectly aligned with upcoming transactional SQL.
> > > >
> > >
> > > The API does not look clean yet. Still requires lots of work.
> > >
> > >
> > > >
> > > > Thoughts?
> > > >
> > > >
> > > > On Thu, Sep 7, 2017 at 6:48 AM, Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > Vova,
> 

Re: Issues if -Djava.net.preferIPv4Stack=true is not set

[Dmitriy Setrakyan]

On Thu, Sep 28, 2017 at 7:08 AM, Yakov Zhdanov  wrote:

> The issue has already been picked up by VK.
>
> Dmitry, after some thought I find it a little bit weird to set system
> property automatically without any way to restore default behavior.
>

I am not sure I understand. Can you please explain?

Re: Default CacheConfiguration instance can't be serialized

[Dmitry Pavlov]

Hi Nikolay,

It is strange, because Cache Configuration is saved under
StoreConfiguration into persistent store cache_data file.

Did you find an anwser?

Sincerely,
Dmitriy Pavlov

вт, 26 сент. 2017 г. в 19:02, Николай Ижиков :

> Hello, Guys.
>
> I found that CacheConfiguration which implements Serializable contains not
> serializable fields by default.
>
> CacheConfiguration -> affMapper -> CacheDefaultBinaryAffinityKeyMapper ->
> proc -> CacheObjectBinaryProcessorImpl.
>
> Is this a bug or I miss something?
>
> Simple reproducer:
>
> ```
> public class SimpleSerializableTet extends GridCommonAbstractTest {
> public void testRemoteFilterFactoryClient() throws Exception {
> Ignite g = startGrid(0);
>
> IgniteCache cache =
> g.getOrCreateCache("TEST_CACHE");
> CacheConfiguration ccfg =
> cache.getConfiguration(CacheConfiguration.class);
>
> assertTrue(ccfg instanceof Serializable);
> assertTrue(ccfg.getAffinityMapper() instanceof Serializable);
>
> try (ByteArrayOutputStream bos = new ByteArrayOutputStream()) {
> ObjectOutput out = new ObjectOutputStream(bos);
> out.writeObject(ccfg);
> out.flush();
> }
> }
> }
>
> ```
> Exception is:
>
> ```
> java.io.NotSerializableException:
>
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
> ```
>
>
>
> --
> Nikolay Izhikov
> nizhikov@gmail.com
>

[GitHub] ignite pull request #2776: IGNITE-6502: Print warning if -Djava.net.preferIP...

[vk23]

GitHub user vk23 opened a pull request:

https://github.com/apache/ignite/pull/2776

IGNITE-6502: Print warning if -Djava.net.preferIPv4Stack=true is not set

Print warning if -Djava.net.preferIPv4Stack=true is not set

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vk23/ignite ignite-6502

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2776.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2776


commit 294ab1c55d086ed42a92133463d373f5ae90807f
Author: vk 
Date:   2017-09-28T19:37:33Z

IGNITE-6502: Print warning if -Djava.net.preferIPv4Stack=true is not set




---

[jira] [Created] (IGNITE-6530) Deadlock in checkpointReadLock method in GridCacheDatabaseSharedManager

[Joel Lang (JIRA)]

Joel Lang created IGNITE-6530:
-

 Summary: Deadlock in checkpointReadLock method in 
GridCacheDatabaseSharedManager
 Key: IGNITE-6530
 URL: https://issues.apache.org/jira/browse/IGNITE-6530
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.2
Reporter: Joel Lang


Just started receiving a great deal of warnings about possible starvation in 
stripped pool.

The stack trace shows it happens in checkpointReadLock() in 
GridCacheDatabaseSharedManager.

Here are the log messages:

{noformat}
2017-09-28 13:15:12 [grid-timeout-worker-#15%mbe%] WARN  
o.a.ignite.internal.util.typedef.G - >>> Possible starvation in striped pool.
Thread name: sys-stripe-4-#5%mbe%
Queue: [Message closure [msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, 
topicOrd=8, ordered=false, timeout=0, skipOnTimeout=false, 
msg=GridNearAtomicSingleUpdateRequest [key=BinaryObjectImpl [arr= true, 
ctx=false, start=0], parent=GridNearAtomicAbstractUpdateRequest [res=null, 
flags=]
Deadlock: true
Completed: 3212
Thread [name="sys-stripe-4-#5%mbe%", id=19, state=WAITING, blockCnt=12, 
waitCnt=5835]
Lock 
[object=java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@2e1993dd, 
ownerName=null, ownerId=-1]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown
 Source)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(Unknown 
Source)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(Unknown 
Source)
at 
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(Unknown Source)
at 
o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager.checkpointReadLock(GridCacheDatabaseSharedManager.java:847)
at 
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1770)
at 
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1686)
at 
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateRequest(GridDhtAtomicCache.java:3063)
at 
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$400(GridDhtAtomicCache.java:129)
at 
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:265)
at 
o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:260)
at 
o.a.i.i.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1042)
at 
o.a.i.i.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:561)
at 
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
at 
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
at 
o.a.i.i.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
at 
o.a.i.i.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
at 
o.a.i.i.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
at 
o.a.i.i.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
at 
o.a.i.i.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at 
o.a.i.i.managers.communication.GridIoManager$9.run(GridIoManager.java:1097)
at o.a.i.i.util.StripedExecutor$Stripe.run(StripedExecutor.java:483)
at java.lang.Thread.run(Unknown Source)

2017-09-28 13:15:12 [grid-timeout-worker-#15%mbe%] WARN  
o.a.ignite.internal.util.typedef.G - >>> Possible starvation in striped pool.
Thread name: sys-stripe-5-#6%mbe%
Queue: [Message closure [msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, 
topicOrd=8, ordered=false, timeout=0, skipOnTimeout=false, 
msg=GridNearAtomicSingleUpdateRequest [key=BinaryObjectImpl [arr= true, 
ctx=false, start=0], parent=GridNearAtomicAbstractUpdateRequest [res=null, 
flags=]
Deadlock: true
Completed: 3524
Thread [name="sys-stripe-5-#6%mbe%", id=20, state=WAITING, blockCnt=3, 
waitCnt=6730]
Lock 
[object=java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@2e1993dd, 
ownerName=null, ownerId=-1]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown
 Source)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(Unknown 
Source)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(Unknown 
Source)
 

[GitHub] ignite pull request #2775: IGNITE 6285: Enhance persistent store paths

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/2775

IGNITE 6285: Enhance persistent store paths



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6285-3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2775.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2775


commit b150519f05c99e71be83af9d5107ae599c90f05f
Author: dpavlov 
Date:   2017-09-26T13:31:54Z

IGNITE-6285: Enhance persistent store paths logging on start

commit 53e803baf89e23ec7fbfe9aece31e131460be1dd
Author: dpavlov 
Date:   2017-09-26T16:11:09Z

IGNITE-6285: Enhance persistent store paths: new tests and folder resolver 
was moved to grid kernal context

commit 3c4a6600f158097179f2cd646596d64e152f47d6
Author: dpavlov 
Date:   2017-09-26T18:20:11Z

IGNITE-6285: Enhance persistent store paths: intermediate commit, first 
locking done

commit 9a9c43b672a24609c6454d5f5bdabd0d7f4b69f9
Author: dpavlov 
Date:   2017-09-27T12:40:14Z

IGNITE-6285: Enhance persistent store paths: transferring file lock to 
activate phase

commit d2d72a8a40691d3df196d4efdcd2427b8fc0e218
Author: dpavlov 
Date:   2017-09-27T13:06:09Z

IGNITE-6285: Enhance persistent store paths: protecting from start without 
activate

commit 22779aeaedd1e559677dc061474b3e0bb9ed8257
Author: dpavlov 
Date:   2017-09-27T13:28:02Z

IGNITE-6285: Enhance persistent store paths: new tests, unused code 
removal, javadoc

commit d4e2e19a897ed0383ee3131420713f64992ff2c9
Author: dpavlov 
Date:   2017-09-27T14:57:18Z

IGNITE-6285: Enhance persistent store paths: old style consistent id 
implemented as priority

commit 75ef089a211b75bcfc5a1e90d0c3bc4f67076a17
Author: dpavlov 
Date:   2017-09-27T16:22:55Z

IGNITE-6285: Enhance persistent store paths: retry to list folder on each 
lock try

commit 5f1ccc39498593ca4a64938e91cc062399e09a05
Author: dpavlov 
Date:   2017-09-27T17:16:50Z

IGNITE-6285: Enhance persistent store paths: global lock for root folder 
done

commit f8359d1b60348d43dab79b586dc3d5d58c101d26
Author: dpavlov 
Date:   2017-09-27T18:41:00Z

IGNITE-6285: Enhance persistent store paths: new tests were added, folder 
name padding, deactivate problem demonstrated

commit 41f01d3ce8ba298b7e18ec27cd77dd53ffed14d1
Author: dpavlov 
Date:   2017-09-28T14:01:05Z

IGNITE-6285: Enhance persistent store paths: locking was moved to resolver 
in case new style folders mode is used

commit f56693a081dafa98441d78f6265aae19842cc8ea
Author: dpavlov 
Date:   2017-09-28T14:39:43Z

IGNITE-6285: Enhance persistent store paths: warn about old style storage 
folder

commit 8e843685de6e63304edb6aae6a0467da6d1f07ad
Author: dpavlov 
Date:   2017-09-28T15:52:40Z

IGNITE-6285: Enhance persistent store paths: standalone WAL reader test was 
fixed

commit b4b023793a619bb735c214b0f5feceb527e7b352
Author: dpavlov 
Date:   2017-09-28T17:06:55Z

IGNITE-6285: Enhance persistent store paths: print size and date for 
observed folder

commit 4888bbaf44353dd188cf7a01f14010be63ed8525
Author: Pavel Tupitsyn 
Date:   2017-09-25T16:09:29Z

.NET: Fix code coverage - exclude remote-only class

commit 8cb7f8044bcfb4c30584e632d620aff6ffa32c70
Author: Pavel Tupitsyn 
Date:   2017-09-26T08:37:31Z

IGNITE-6494 .NET: Fix CacheConfiguration.WriteSynchronizationMode default 
value

This closes #2741

commit beebc11e9e28dd8cf9e24837160a4a804c0c53ab
Author: sboikov 
Date:   2017-09-26T09:39:28Z

Removed unused internal getTopologySafe method.

commit e16e9d9bb5efb2559494b5b243bcda24d5303b05
Author: Aleksei Scherbakov 
Date:   2017-09-26T13:07:06Z

IGNITE-6491 Fixed IgniteTopologyValidatorGridSplitCacheTest

commit 53a2ea4449883d492eddd63ea2be34b0afacbb25
Author: Pavel Tupitsyn 
Date:   2017-09-26T16:26:02Z

IGNITE-5898 .NET: Fix QueryDmlExample and BinaryModeExample

Enable distributed joins when selecting data

commit a3545fcc2f7d3bb43ef0276685cae2bbdbd96042
Author: Pavel Tupitsyn 
Date:   2017-09-26T17:00:52Z

IGNITE-5606 .NET: Test various data types as messaging topic and message

commit b6232fbe2130e35d44b4bc63995e62db42f92399
Author: Igor Sapego 
Date:   2017-09-27T06:43:58Z

IGNITE-6468: ODBC: More tests for SQLGetInfo. This closes #2733.

commit 

[GitHub] ignite pull request #2774: IGNITE-6416

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/2774

IGNITE-6416



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6416

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2774.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2774


commit 52d9168e5879470161b4a72735bfcd849dfa13c6
Author: Alexander Paschenko 
Date:   2017-09-28T17:56:23Z

IGNITE-6416 Dynamic columns tests refactoring.

commit e7d3eec3b7ff5dcefc396ffab62455271db70618
Author: Alexander Paschenko 
Date:   2017-09-28T18:00:21Z

Minor.




---

[GitHub] ignite pull request #2752: IGNITE-6285: Enhance persistent store paths loggi...

[dspavlov]

Github user dspavlov closed the pull request at:

https://github.com/apache/ignite/pull/2752


---

Re: [DISCUSS] Ignite Update Checker

[Denis Magda]

Cos, 

The screenshot was not attached. Could you share it some other way (google 
drive, etc.)? I’ve never seen any commercial on the site.

—
Denis

> On Sep 28, 2017, at 7:23 AM, Konstantin Boudnik  wrote:
> 
> I don't see an issue with node version either. 
> 
> Just one more, and it might be slightly irrelevant, issue though... I looked 
> at the Ignite's site and found the following ads and trackers (that are 
> indeed getting disabled by my browser).
> Why are googleads or doubleclick are permitted?
> 
> 
> ​
> Thanks,
>   Cos
> ​
> 
> --
>   With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
> 
> Disclaimer: Opinions expressed in this email are those of the author, and do 
> not necessarily represent the views of any company the author might be 
> affiliated with at the moment of writing.
> 
> On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan  > wrote:
> On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov  >
> wrote:
> 
> > Folks,
> >
> > Can we add version of current node to web request? This way we will better
> > understand version distribution, what might help us with certain API
> > update/deprecate decisions
> > E.g. http://ignite.apache.org/latest.cgi=2.2.0 
> > 
> 
> 
> Vladimir, I personally do not see a problem with that, as sending the
> current version to the update checker seems very innocent to me. At the
> same time, it will allow us to examine the usage of each release and make
> decisions about dropping backward compatibility or spotting bugs for a
> certain release.
> 
> Cos, Raul, any thoughts?
> 
> 
> >
> >
> > Vladimir.
> >
> > On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan  > >
> > wrote:
> >
> > > I think it is safe to assume at this point that everyone is in general
> > > agreement, since there are no active objections.
> > >
> > > I have filed a ticket for the 2.3 release. Let's try to make it happen:
> > > https://issues.apache.org/jira/browse/IGNITE-6305 
> > > 
> > >
> > > D.
> > >
> > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org >
> > > wrote:
> > >
> > > >
> > > >
> > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani  > > > >
> > > > wrote:
> > > >
> > > >> Yeah, I guess that's doable as well and requires less management
> > effort
> > > >> than my suggestion. We could use events [1] to store payload data
> > (e.g.
> > > >> IP,
> > > >> version, etc.)
> > > >
> > > >
> > > > Yes, we could use events or some other similar API provided by GA.
> > > >
> > > >
> > > >> What the download page CGI developed in? PHP?
> > > >>
> > > >
> > > > To be honest, no clue. I guess someone in the community can figure it
> > > out:
> > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html 
> > > > 
> > > >
> > > >
> > > >> However, I'm not sure whether storing this data in a 3rd party
> > (Google)
> > > is
> > > >> compliant with the ASF policy. I guess it's no biggie, but if there's
> > > >> doubt
> > > >> in the PMC, it's better to ask legal@.
> > > >
> > > >
> > > > I am not sure there is anything to ask about. The whole Ignite website
> > is
> > > > GA enabled, and all we are doing is accessing a standard web page from
> > > the
> > > > Ignite web site. The information gathered from GA is available only to
> > > the
> > > > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > > > question.
> > > >
> > > > Even ASF website itself uses GA: https://www.apache.org/ 
> > > > 
> > > > foundation/policies/privacy.html
> > > >
> > > >
> > > >> I think Cos said it's OK; maybe Roman can pitch in.
> > > >>
> > > >
> > > >  Sure, would be nice to hear from Roman as well.
> > > >
> > > >
> > > >> Cheers.
> > > >>
> > > >> [1]
> > > >> https://developers.google.com/analytics/devguides/collection 
> > > >> 
> > > >> /analyticsjs/events
> > > >>
> > > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> > > >> dsetrak...@apache.org >
> > > >> wrote:
> > > >>
> > > >> > Raul,
> > > >> >
> > > >> > Could point about Javascript, it will not work because it executes
> > in
> > > >> the
> > > >> > browser. This means we need a server-side script, like CGI we are
> > > using
> > > >> on
> > > >> > our download page.
> > > >> >
> > > >> > How about this approach. We create something like ignite-version.cgi
> > > >> script
> > > >> > which will invoke a call to GA and then return the latest version.
> > > >> >
> > > >> > This should work, right?
> > > >> >
> 

Re: Warning if no table for BinaryObject

[Mikhail Cherkasov]

Hi Denis,

I filed a ticket: https://issues.apache.org/jira/browse/IGNITE-6528

Thanks,
Mikhail.

On Tue, Sep 26, 2017 at 6:58 PM, Denis Magda  wrote:

> Mikhail,
>
> Could you prepare a properly described ticket encompassing the suggested
> warning? Let’s do the improvement for the upcoming release.
>
> —
> Denis
>
> > On Sep 26, 2017, at 7:23 AM, Mikhail Cherkasov 
> wrote:
> >
> > Hi Dmitry,
> >
> > The problem is that we don't have strong typing when defining QueryEntriy
> > and a user can use an arbitrary string id to
> > define a type, but he should use the same string id to obtain binary
> object
> > builder, however, people sometimes confusing this.
> > So the user can define QueryEntity with value type:
> > queryEntity.setValueType("MyCoolName") and
> > later put to cache the following binary object:
> > ignite.binary.toBinary(value), but this object won't be indexed, because
> > ignite.binary.toBinary uses class name as string id while indexing
> expects
> > to find "MyCoolName" as id.
> >
> > There's no bug in ignite about this, but I think the warning can make
> > users' life easy and
> > make ignite configuration more friendly to new users.
> >
> > Thanks,
> > Mikhail.
> >
> >
> >
> >
> > On Tue, Sep 26, 2017 at 8:15 AM, Yakov Zhdanov 
> wrote:
> >
> >> Pavel, we can optionally enforce that, but not by default.
> >>
> >> Dmitry, the problem is the following. User configures Ignite cache and
> >> query entity, does several puts than empty results for SQL. The reason
> is
> >> very simple - typo in class name or simply different classes in
> >> configuration and of actual cache entry. The fix is pretty simple and
> make
> >> product more usable.
> >>
> >> --Yakov
> >>
> >
> >
> >
> > --
> > Thanks,
> > Mikhail.
>
>


-- 
Thanks,
Mikhail.

[jira] [Created] (IGNITE-6529) JDBC drivers don't provide correct metadata about columns' nullability

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-6529:
---

 Summary: JDBC drivers don't provide correct metadata about  
columns' nullability
 Key: IGNITE-6529
 URL: https://issues.apache.org/jira/browse/IGNITE-6529
 Project: Ignite
  Issue Type: Bug
  Components: jdbc, sql
Reporter: Alexander Paschenko


For complete implementation of IGNITE-5648, we have to make JDBC drivers return 
correct nullability flag for {{NOT NULL}} columns - currently corresponding 
param of {{QueryField}} is ignored.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6528) Warning if no table for BinaryObject

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-6528:
-

 Summary: Warning if no table for BinaryObject
 Key: IGNITE-6528
 URL: https://issues.apache.org/jira/browse/IGNITE-6528
 Project: Ignite
  Issue Type: Improvement
  Components: binary, cache, sql
Reporter: Mikhail Cherkasov


I've seen several times that due wrong cache configuration people can't find 
data in cache and blame Ignite that it's buggy and doesn't work.

And it's very difficult to find an error in the code, especially if you don't 
have reach experience with Ignite.

The problem is that we don't have strong typing when defining QueryEntriy and a 
user can use an arbitrary string id to
define a type, but he should use the same string id to obtain binary object 
builder, however, people sometimes confusing this.
So the user can define QueryEntity with value type:  
queryEntity.setValueType("MyCoolName") and 
later put to cache the following binary object: ignite.binary.toBinary(value), 
but this object won't be indexed, because
ignite.binary.toBinary uses class name as string id while indexing expects to 
find "MyCoolName" as id.

The example is simple and the error is obvious when you see this two lines 
close to each other, however, in real life, cache definition and data ingestion 
are separated by tons of code.

We can save a lot of man-hours for our users if Ignite will print a warning If 
a cache has a configured QE and user puts BinaryObject with typeName which 
doesn't correspond to any QE.

The warning should be printed only once, something like:
[WARN] No table is found for %typeName% binary object.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2773: IGNITE-6005: Add ability to close datastructure i...

[nizhikov]

GitHub user nizhikov opened a pull request:

https://github.com/apache/ignite/pull/2773

IGNITE-6005: Add ability to close datastructure in interrupted thread.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nizhikov/ignite IGNITE-6005

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2773.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2773


commit cee2914a07e84740ba8502f26b5d3bb266b9fc2f
Author: Nikolay Izhikov 
Date:   2017-09-28T16:50:36Z

IGNITE-6005: Add ability to close datastructure in interrupted thread.




---

[GitHub] ignite pull request #2772: Ignite 6512

[EdShangGG]

GitHub user EdShangGG opened a pull request:

https://github.com/apache/ignite/pull/2772

Ignite 6512



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6512

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2772.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2772


commit 5bd47baaa0d87d565aca91cb55da0cdef5695958
Author: Ivan Rakov 
Date:   2017-09-07T13:58:21Z

IGNITE-6512 Need a possibility to start caches without discarding their 
'restarting' state

commit 459d33ddd00e9efe618dda4a6cc54ccee0f47c1c
Author: Eduard Shangareev 
Date:   2017-09-27T21:37:54Z

IGNITE-6512 Need a possibility to start caches without discarding their 
'restarting' state




---

[GitHub] ignite pull request #2771: IGNITE-5784 .NET: QueryIndex.InlineSize

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/2771

IGNITE-5784 .NET: QueryIndex.InlineSize



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-5784

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2771.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2771


commit 0a0966926882185d8bc3f1a127331d78cbd885f2
Author: Pavel Tupitsyn 
Date:   2017-09-28T12:24:27Z

IGNITE-5784 .NET: QueryIndex.InlineSize

commit 4e3ccc50fe982ec1087e8f4b768f86458bdbba78
Author: Pavel Tupitsyn 
Date:   2017-09-28T12:47:28Z

wip

commit 6c7bdcdfad5730ea4a1770372eb00c285522851d
Author: Pavel Tupitsyn 
Date:   2017-09-28T12:49:23Z

wip

commit 47e15e593d4c9618953c3bc6c2c4d07114769ec1
Author: Pavel Tupitsyn 
Date:   2017-09-28T12:51:06Z

Read-write properties

commit c080958789377b97e07ada313c5ee03bcc18a236
Author: Pavel Tupitsyn 
Date:   2017-09-28T12:52:25Z

fix propagation

commit 31e3982a24e941064c3373de1f958373ef1a00eb
Author: Pavel Tupitsyn 
Date:   2017-09-28T12:56:32Z

wip

commit 19cdb384debec83cf3937a141105eca31f918b2a
Author: Pavel Tupitsyn 
Date:   2017-09-28T13:30:56Z

read-write Java

commit aaf286d8179438262bad87dd4ffca10ce7344d49
Author: Pavel Tupitsyn 
Date:   2017-09-28T13:41:27Z

wip tests

commit f7c3130481cd4edbc35b85bf853d199b6ab82be4
Author: Pavel Tupitsyn 
Date:   2017-09-28T13:56:24Z

Fix attributes

commit e36b2fa4b27ae60f3a12c6fd026a621d38c0a7c5
Author: Pavel Tupitsyn 
Date:   2017-09-28T13:58:02Z

cleanup

commit 57c9a39fae4399fa1993cdce356b42882c0545f9
Author: Pavel Tupitsyn 
Date:   2017-09-28T14:56:55Z

wip tests

commit 99027190b57d4923c03d775a47dae3ec8cc28bec
Author: Pavel Tupitsyn 
Date:   2017-09-28T15:02:08Z

Extract predefined config to a file

commit 5eca207b1d9e8697eddfbffeae795f9da01a950e
Author: Pavel Tupitsyn 
Date:   2017-09-28T15:07:24Z

wip tests

commit d8f0e58d23f41df7dc0764c6a803b4705f07b4f1
Author: Pavel Tupitsyn 
Date:   2017-09-28T15:09:38Z

wip schema

commit 5a12ae1382ef0f31689be2f7c7872d8c81a4c7a1
Author: Pavel Tupitsyn 
Date:   2017-09-28T15:10:53Z

wip tests

commit 19c888f54704fd15b277dd45697dcec61ed81c76
Author: Pavel Tupitsyn 
Date:   2017-09-28T15:27:22Z

all done




---

[jira] [Created] (IGNITE-6527) Deadlock detection work incorrect with deadlocks that are not the cause of the timeout.

[Vitaliy Biryukov (JIRA)]

Vitaliy Biryukov created IGNITE-6527:


 Summary: Deadlock detection work incorrect with deadlocks that are 
not the cause of the timeout.
 Key: IGNITE-6527
 URL: https://issues.apache.org/jira/browse/IGNITE-6527
 Project: Ignite
  Issue Type: Bug
Reporter: Vitaliy Biryukov
Assignee: Vitaliy Biryukov






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Persistence per memory policy configuration

[Yakov Zhdanov]

StorageConfiguration + StorageRegionConfiguration

--Yakov

[GitHub] ignite pull request #2736: ignite-5097 Trick for running test via Maven on T...

[daradurvs]

Github user daradurvs closed the pull request at:

https://github.com/apache/ignite/pull/2736


---

Re: Persistence per memory policy configuration

[Vladimir Ozerov]

Guys,

But what is exact desicion? :-) I saw two final options:

1) StorageConfiguration + StorageRegionConfiguration
2) DataStorageConfiguration + DataRegionConfiguration

Which one we choose?

On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov  wrote:

> > I guess it is safe to assume that at this point we came to a consensus?
>
> Alex, I think so. Let's carve it in stone (code).
>
> --Yakov
>

Re: [DISCUSS] Ignite Update Checker

[Konstantin Boudnik]

I don't see an issue with node version either.

Just one more, and it might be slightly irrelevant, issue though... I
looked at the Ignite's site and found the following ads and trackers (that
are indeed getting disabled by my browser).
Why are googleads or doubleclick are permitted?


​
Thanks,
  Cos
​

--
  With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any company the author might be
affiliated with at the moment of writing.

On Tue, Sep 26, 2017 at 3:21 PM, Dmitriy Setrakyan 
wrote:

> On Tue, Sep 26, 2017 at 6:20 AM, Vladimir Ozerov 
> wrote:
>
> > Folks,
> >
> > Can we add version of current node to web request? This way we will
> better
> > understand version distribution, what might help us with certain API
> > update/deprecate decisions
> > E.g. http://ignite.apache.org/latest.cgi=2.2.0
>
>
> Vladimir, I personally do not see a problem with that, as sending the
> current version to the update checker seems very innocent to me. At the
> same time, it will allow us to examine the usage of each release and make
> decisions about dropping backward compatibility or spotting bugs for a
> certain release.
>
> Cos, Raul, any thoughts?
>
>
> >
> >
> > Vladimir.
> >
> > On Fri, Sep 8, 2017 at 7:06 AM, Dmitriy Setrakyan  >
> > wrote:
> >
> > > I think it is safe to assume at this point that everyone is in general
> > > agreement, since there are no active objections.
> > >
> > > I have filed a ticket for the 2.3 release. Let's try to make it happen:
> > > https://issues.apache.org/jira/browse/IGNITE-6305
> > >
> > > D.
> > >
> > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > > >
> > > >
> > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
> raul@evosent.com>
> > > > wrote:
> > > >
> > > >> Yeah, I guess that's doable as well and requires less management
> > effort
> > > >> than my suggestion. We could use events [1] to store payload data
> > (e.g.
> > > >> IP,
> > > >> version, etc.)
> > > >
> > > >
> > > > Yes, we could use events or some other similar API provided by GA.
> > > >
> > > >
> > > >> What the download page CGI developed in? PHP?
> > > >>
> > > >
> > > > To be honest, no clue. I guess someone in the community can figure it
> > > out:
> > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> > > >
> > > >
> > > >> However, I'm not sure whether storing this data in a 3rd party
> > (Google)
> > > is
> > > >> compliant with the ASF policy. I guess it's no biggie, but if
> there's
> > > >> doubt
> > > >> in the PMC, it's better to ask legal@.
> > > >
> > > >
> > > > I am not sure there is anything to ask about. The whole Ignite
> website
> > is
> > > > GA enabled, and all we are doing is accessing a standard web page
> from
> > > the
> > > > Ignite web site. The information gathered from GA is available only
> to
> > > the
> > > > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > > > question.
> > > >
> > > > Even ASF website itself uses GA: https://www.apache.org/
> > > > foundation/policies/privacy.html
> > > >
> > > >
> > > >> I think Cos said it's OK; maybe Roman can pitch in.
> > > >>
> > > >
> > > >  Sure, would be nice to hear from Roman as well.
> > > >
> > > >
> > > >> Cheers.
> > > >>
> > > >> [1]
> > > >> https://developers.google.com/analytics/devguides/collection
> > > >> /analyticsjs/events
> > > >>
> > > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> > > >> dsetrak...@apache.org>
> > > >> wrote:
> > > >>
> > > >> > Raul,
> > > >> >
> > > >> > Could point about Javascript, it will not work because it executes
> > in
> > > >> the
> > > >> > browser. This means we need a server-side script, like CGI we are
> > > using
> > > >> on
> > > >> > our download page.
> > > >> >
> > > >> > How about this approach. We create something like
> ignite-version.cgi
> > > >> script
> > > >> > which will invoke a call to GA and then return the latest version.
> > > >> >
> > > >> > This should work, right?
> > > >> >
> > > >> > D.
> > > >> >
> > > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> > raul@evosent.com
> > > >
> > > >> > wrote:
> > > >> >
> > > >> > > Hey Dmitriy and all
> > > >> > >
> > > >> > > Also, since we have GA enabled for the website, we can track how
> > > many
> > > >> > times
> > > >> > > > this page was accessed, which will be equal to the number of
> > > starts.
> > > >> > This
> > > >> > > > way, the counter information is tracked and monitored by the
> > > Ignite
> > > >> > PMC.
> > > >> > >
> > > >> > >
> > > >> > > Unfortunately this won't work because GA is loaded via
> Javascript
> > on
> > > >> the
> > > >> > > browser, so Google will never receive the page hit.
> > > >> > >
> > > >> > > Given the constraints, the most 

[jira] [Created] (IGNITE-6526) Ignite 2.x capacity planning guide

[Denis Magda (JIRA)]

Denis Magda created IGNITE-6526:
---

 Summary: Ignite 2.x capacity planning guide
 Key: IGNITE-6526
 URL: https://issues.apache.org/jira/browse/IGNITE-6526
 Project: Ignite
  Issue Type: Task
Reporter: Denis Magda


Current capacity planning guide [1] is too high level and should be elaborated 
considering durable memory's internals:
- memory pages overhead.
- per-entry overhead 
(http://apache-ignite-users.70518.x6.nabble.com/Re-Memory-Overhead-per-entry-in-Apache-Ignite-td9498.html).
- space occupied for indexing needs.
- free lists
- etc.

The page has to include estimates for the Ignite Native Persistence:
- entry size and its overheads.
- index size and overheads.
- data files overheads.
- estimated WAL size and how to shrink it basing on checkpointing settings.


[1] https://apacheignite.readme.io/docs/capacity-planning



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6525) .NET: BinaryObject.Deserialize to custom type

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6525:
--

 Summary: .NET: BinaryObject.Deserialize to custom type
 Key: IGNITE-6525
 URL: https://issues.apache.org/jira/browse/IGNITE-6525
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Pavel Tupitsyn


Current {{BinaryObject.Deserialize()}}  method takes a generic argument, but 
only uses it to cast the final result. Actual type is determined according to 
{{typeId}}.

Add {{forceType}} argument to override {{typeId}} logic and use the provided 
type instead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Persistence per memory policy configuration

[Yakov Zhdanov]

> I guess it is safe to assume that at this point we came to a consensus?

Alex, I think so. Let's carve it in stone (code).

--Yakov

Re: Issues if -Djava.net.preferIPv4Stack=true is not set

[Yakov Zhdanov]

The issue has already been picked up by VK.

Dmitry, after some thought I find it a little bit weird to set system
property automatically without any way to restore default behavior.

--Yakov

[jira] [Created] (IGNITE-6524) .NET: Explicit dynamic type registration

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6524:
--

 Summary: .NET: Explicit dynamic type registration
 Key: IGNITE-6524
 URL: https://issues.apache.org/jira/browse/IGNITE-6524
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Pavel Tupitsyn


Introduce {{IBinary.RegisterType(Type type)}} (similar to existing 
{{RegisterEnum}}) to allow explicit type registration in cluster.

This may be useful when we already have binary objects in cluster and want to 
start working with them in deserialized form for the first time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mockito 1.9.5 and JUnit 4.11 incompatibility

[Aleksei Zaitsev]

Hi, guys.

There is a known issue, that mockito 1.9.5 and JUnit 4.11 are not compatible: 
https://code.google.com/archive/p/mockito/issues/397

In our project we have these versions of libraries, so when someone wants to 
write "red" unit test, like this one:

import org.junit.Test;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.MatcherAssert.assertThat;

public class FailureTests {
@Test
public void reproduceMockitoAndJUnit() {
int expected = 8;
int actual = 2 + 2 * 2;

assertThat(actual, is(expected));
}
}

He gets an error:
java.lang.NoSuchMethodError: 
org.hamcrest.Matcher.describeMismatch(Ljava/lang/Object;Lorg/hamcrest/Description;)V

at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
at 
org.apache.ignite.XmlConverterTest.xmlBuildsCorrectlyForNonEmptyClassWithEmbeddedIgniteClass(XmlConverterTest.java:58)
...

instead of normal error message:
java.lang.AssertionError: 
Expected: is <8>
 but: was <6>


Briefly, in JUnit 4.11 was introduced harmcrest library 1.3, which adds a new 
method describeMismatch and since the shadowing interface org.hamcrest.Matcher 
in mockito-all does not have this method, we get the error mentioned above.

I want to create a Jira issue for this and fix it.

And one more question. Considering intention to migration to Java 8(or even 9) 
are we going to update JUnit to version 5 and mockito to 2.* versions? If yes, 
I could also create tickets for this and do this later.

[jira] [Created] (IGNITE-6523) .NET: QueryField.NotNull

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-6523:
--

 Summary: .NET: QueryField.NotNull
 Key: IGNITE-6523
 URL: https://issues.apache.org/jira/browse/IGNITE-6523
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Affects Versions: 2.3
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.3


Add {{QueryEntity.NotNull}} property to reflect changes in Java: IGNITE-6509



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2769: IGNITE-6509

[devozerov]

Github user devozerov closed the pull request at:

https://github.com/apache/ignite/pull/2769


---

Re: Future of Ignite transactions

[Vladimir Ozerov]

Alex,

MVCC concept and blocking/non-blocking behavior are not inter-related. You
can have both blocking and non-blocking writes with MVCC and without it.
The thing is that users *want* blocking behavior as it is easy to reason
about. This is why even MVCC-based RDBMS vendors (Oracle, PostgreSQL) do
block on writes.

OPTIMSITIC mode and non-blocking approaches are for rare enthusiasts. It is
not in priority for us.

On Thu, Sep 28, 2017 at 1:59 PM, Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> Do we have a plan to support multi-version concurrency control ?
>
> Proper implementation allows to achieve better isolation without blocking
> for the cost of additional resource consumption.
>
> The feature can be configurable per transaction.
>
> 6) What if I want to acquire write locks only for some specific reads
> within same transaction ?
>
>
> 2017-09-27 5:52 GMT+03:00 Dmitriy Setrakyan :
>
> > OK, if we must change the current behavior, let's discuss the new design.
> > My comments/questions are below...
> >
> > On Mon, Sep 25, 2017 at 12:09 PM, Vladimir Ozerov 
> > wrote:
> >
> > > Folks,
> > >
> > > Sorry for late reply. I had a chat with several Ignite veterans today.
> We
> > > tried to design transactional SQL for Ignite. One of our questions was
> > how
> > > to align SQL transactions with current Ignite transactions. And we
> > failed.
> > > And then we came to conclusion that current transaction API is
> unusable.
> > We
> > > have 6 pairs of modes from API standpoint and 4 real modes. This is
> very
> > > counterintuitive and cannot be mapped to any transactional framework
> our
> > > users are familiar with.
> > >
> > > So we thought how new tx API might looks like, and here is the draft.
> > >
> > > 1) Define new enum *TransactionIsolationLevel *(to avoid clashes with
> > > current enum) with three standard modes - READ_COMMITTED,
> > REPEATABLE_READ,
> > > SERIALIZABLE.
> > >
> >
> > If it is the same values as we have today, why create a new enum?
> >
> >
> > > 2) Define new enum *TransactionHint* - READ_ONLY, OPTIMISTIC_LOCKING
> > >
> >
> > The word *hint* means no guarantee. When it comes to transactions, we
> must
> > have guarantees. Also, what happens to PESSIMISTIC locking?
> >
> >
> > > 3) No more OPTIMISTIC and PESSIMISTIC. Seriously.
> > >
> >
> > But you are still proposing OPTIMISTIC_LOCKING above. So, we have
> > OPTIMISTIC without PESSIMISTIC?
> >
> >
> > > 4) Reads never acuire locks
> > > 5) Writes always acquire locks
> > >
> >
> > Hm... what locks? I think we are getting to deep into the weeds here.
> Isn't
> > it our internal implementation detail to acquire locks or not? For
> example,
> > MVCC approach we are working on will be virtually lock-free for all
> > scenarios.
> >
> > I would focus on transaction behavior, not locks.
> >
> >
> >
> > > 6) *IgniteCache.withReadForUpdate()* will return special facade which
> > will
> > > obtain locks on reads. This is analogue of SELECT FOR UPDATE in SQL.
> > >
> >
> > This syntax is very ugly in databases and we do not need to carry this
> over
> > to Ignite. All it means is that a pessimistic lock is acquired. To be
> > honest, I would rather start a pessimistic transaction to get the same
> > behavior.
> >
> >
> > > 7) *TransactionHint.READ_ONLY* - forces transaction to throw an
> exception
> > > on any update
> > >
> >
> > I really like the READ_ONLY flag. Allows us to put many optimizations in
> > place.
> >
> >
> > > 8) *TransactionHint.OPTIMISTIC_LOCKING* - turns transaction into our
> > > current deadlock-free OPTIMISTIC/SERIALIZABLE mode. Applicable only to
> > > SERIALIZABLE isolation level.
> > >
> >
> > Is there a PESSIMISTIC/SERIALIZABLE mode?
> >
> >
> > > 9) Define new API methods:
> > > - IgniteTransactions.txStart(TransactionIsolationLevel isolation)
> > > - IgniteTransactions.txStart(TransactionIsolationLevel isolation,
> > > TransactionHint... hints)
> >
> > 10) Deprecate old TX start methods
> > >
> > > As a result we will have simple, clean and extensible API. Which can be
> > > explained to users in 5 minutes, instead of current half an hour. And
> > which
> > > is perfectly aligned with upcoming transactional SQL.
> > >
> >
> > The API does not look clean yet. Still requires lots of work.
> >
> >
> > >
> > > Thoughts?
> > >
> > >
> > > On Thu, Sep 7, 2017 at 6:48 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >
> > > wrote:
> > >
> > > > Vova,
> > > >
> > > > Thanks for doing the research. The changes you are suggesting are a
> bit
> > > too
> > > > bold, so let's discuss them in some more detail...
> > > >
> > > > On Wed, Sep 6, 2017 at 4:51 AM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > We are moving towards DBMS system. None of them has a notion of
> > > > > OPTIMISTIC/PESSIMISTIC transactions. Instead, they work as follows:
> > > > > 1) Reads (SELECT) do not acquire exclusive row 

[GitHub] ignite pull request #2770: IGNITE-6513 introduce system property for manage ...

[DmitriyGovorukhin]

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/2770

IGNITE-6513 introduce system property for manage wal serializer version

-DIGNITE_WAL_SERIALIZER_VERSION=2

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGNITE-6513

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2770.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2770


commit d921a453ed48681781c830bac56f73d33b3b3ca0
Author: Dmitriy Govorukhin 
Date:   2017-09-28T13:05:57Z

IGNITE-6513 introduce system property for manage wal serializer version 
(IGNITE_WAL_SERIALIZER_VERSION)




---

[GitHub] ignite pull request #2618: IGNITE-6046 Multiple SQL statements in one JDBC c...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2618


---

[GitHub] ignite pull request #2769: IGNITE-6509

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/2769

IGNITE-6509



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6509

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2769.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2769


commit 2ee59b208ac0c7d4665bf190c38bf7286aeffe3e
Author: devozerov 
Date:   2017-09-28T08:37:58Z

Returned not null field.




---

[GitHub] ignite pull request #2768: ignite-5714 refactoring proposal for unused remot...

[voipp]

GitHub user voipp opened a pull request:

https://github.com/apache/ignite/pull/2768

ignite-5714 refactoring proposal for unused remoteCandidate()



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/voipp/ignite ignite-5714-refactoring-proposal

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2768.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2768


commit 317ba55678e52e586054e046674a887debf26809
Author: voipp 
Date:   2017-09-28T12:04:35Z

refactoring proposal for unused remoteCandidate()




---

[jira] [Created] (IGNITE-6522) Binary metadata is invalid after creation on BinaryConfiguration.classNames with wildcard

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6522:


 Summary: Binary metadata is invalid after creation on 
BinaryConfiguration.classNames with wildcard
 Key: IGNITE-6522
 URL: https://issues.apache.org/jira/browse/IGNITE-6522
 Project: Ignite
  Issue Type: Bug
  Components: binary
Affects Versions: 2.1
Reporter: Taras Ledkov


The enum flag and enum's fields are calculated invalid on iterating through 
classes specified with wildcard {{.*}} at the 
{{BinaryConfiguration.classNames}}

See: 
{{BinaryContext#configure()}}
{{typeCfg}} is created for type with wildcard. Enum flag & fields are gathered 
for type with wildcard instead of each type (inside the loop: {{{for (String 
clsName0 : classesInPackage(pkgName))}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6521) Review default JVM options for better performance

[Alexandr Kuramshin (JIRA)]

Alexandr Kuramshin created IGNITE-6521:
--

 Summary: Review default JVM options for better performance
 Key: IGNITE-6521
 URL: https://issues.apache.org/jira/browse/IGNITE-6521
 Project: Ignite
  Issue Type: Improvement
  Components: general, visor
Affects Versions: 2.1
Reporter: Alexandr Kuramshin
Assignee: Alexandr Kuramshin


Non-optimal recommendations are present in ignite startup scrips

{noformat}
::
:: Uncomment the following GC settings if you see spikes in your throughput due 
to Garbage Collection.
::
:: set JVM_OPTS=%JVM_OPTS% -XX:+UseParNewGC -XX:+UseConcMarkSweepGC 
-XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m
:: set JVM_OPTS=%JVM_OPTS% -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 
-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60
{noformat}

Some utilities (like Visor) are hanged up in continuous GCs when connected to 
large clusters (above one hundred nodes). Even after using large heap (about 32 
Gb).

I'd like to propose to remove this lines and modify default JVM_OPTS as follows

{noformat}
set JVM_OPTS=-Xms1g -Xmx8g -XX:+UseG1GC -server -XX:+AggressiveOpts 
-XX:MaxPermSize=256m
{noformat}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Future of Ignite transactions

[Alexei Scherbakov]

Do we have a plan to support multi-version concurrency control ?

Proper implementation allows to achieve better isolation without blocking
for the cost of additional resource consumption.

The feature can be configurable per transaction.

6) What if I want to acquire write locks only for some specific reads
within same transaction ?


2017-09-27 5:52 GMT+03:00 Dmitriy Setrakyan :

> OK, if we must change the current behavior, let's discuss the new design.
> My comments/questions are below...
>
> On Mon, Sep 25, 2017 at 12:09 PM, Vladimir Ozerov 
> wrote:
>
> > Folks,
> >
> > Sorry for late reply. I had a chat with several Ignite veterans today. We
> > tried to design transactional SQL for Ignite. One of our questions was
> how
> > to align SQL transactions with current Ignite transactions. And we
> failed.
> > And then we came to conclusion that current transaction API is unusable.
> We
> > have 6 pairs of modes from API standpoint and 4 real modes. This is very
> > counterintuitive and cannot be mapped to any transactional framework our
> > users are familiar with.
> >
> > So we thought how new tx API might looks like, and here is the draft.
> >
> > 1) Define new enum *TransactionIsolationLevel *(to avoid clashes with
> > current enum) with three standard modes - READ_COMMITTED,
> REPEATABLE_READ,
> > SERIALIZABLE.
> >
>
> If it is the same values as we have today, why create a new enum?
>
>
> > 2) Define new enum *TransactionHint* - READ_ONLY, OPTIMISTIC_LOCKING
> >
>
> The word *hint* means no guarantee. When it comes to transactions, we must
> have guarantees. Also, what happens to PESSIMISTIC locking?
>
>
> > 3) No more OPTIMISTIC and PESSIMISTIC. Seriously.
> >
>
> But you are still proposing OPTIMISTIC_LOCKING above. So, we have
> OPTIMISTIC without PESSIMISTIC?
>
>
> > 4) Reads never acuire locks
> > 5) Writes always acquire locks
> >
>
> Hm... what locks? I think we are getting to deep into the weeds here. Isn't
> it our internal implementation detail to acquire locks or not? For example,
> MVCC approach we are working on will be virtually lock-free for all
> scenarios.
>
> I would focus on transaction behavior, not locks.
>
>
>
> > 6) *IgniteCache.withReadForUpdate()* will return special facade which
> will
> > obtain locks on reads. This is analogue of SELECT FOR UPDATE in SQL.
> >
>
> This syntax is very ugly in databases and we do not need to carry this over
> to Ignite. All it means is that a pessimistic lock is acquired. To be
> honest, I would rather start a pessimistic transaction to get the same
> behavior.
>
>
> > 7) *TransactionHint.READ_ONLY* - forces transaction to throw an exception
> > on any update
> >
>
> I really like the READ_ONLY flag. Allows us to put many optimizations in
> place.
>
>
> > 8) *TransactionHint.OPTIMISTIC_LOCKING* - turns transaction into our
> > current deadlock-free OPTIMISTIC/SERIALIZABLE mode. Applicable only to
> > SERIALIZABLE isolation level.
> >
>
> Is there a PESSIMISTIC/SERIALIZABLE mode?
>
>
> > 9) Define new API methods:
> > - IgniteTransactions.txStart(TransactionIsolationLevel isolation)
> > - IgniteTransactions.txStart(TransactionIsolationLevel isolation,
> > TransactionHint... hints)
>
> 10) Deprecate old TX start methods
> >
> > As a result we will have simple, clean and extensible API. Which can be
> > explained to users in 5 minutes, instead of current half an hour. And
> which
> > is perfectly aligned with upcoming transactional SQL.
> >
>
> The API does not look clean yet. Still requires lots of work.
>
>
> >
> > Thoughts?
> >
> >
> > On Thu, Sep 7, 2017 at 6:48 AM, Dmitriy Setrakyan  >
> > wrote:
> >
> > > Vova,
> > >
> > > Thanks for doing the research. The changes you are suggesting are a bit
> > too
> > > bold, so let's discuss them in some more detail...
> > >
> > > On Wed, Sep 6, 2017 at 4:51 AM, Vladimir Ozerov 
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > We are moving towards DBMS system. None of them has a notion of
> > > > OPTIMISTIC/PESSIMISTIC transactions. Instead, they work as follows:
> > > > 1) Reads (SELECT) do not acquire exclusive row locks
> > > > 2) Exclusive lock on read could be forced explicitly (SELECT ... FOR
> > > > UPDATE)
> > > > 3) Writes do acuire explicit row locks
> > > > 4) Locks are always acquired immediately once statement is executed
> > > > 5) The strictest concurrency level - typically SERIALIZABLE - rely on
> > > > so-called *range locks* (or *predicate locks*) to track dependencies
> > > > between transactions. Some vendors throw an exception in case of
> > > conflict -
> > > > these are ones where snapshot-based MVCC is used - PostgreSQL,
> Oracle.
> > > > Others do aggressive locking - ones where two-phase locking algorithm
> > is
> > > > used - SQL Server, MySQL.
> > > >
> > > > As you see, there is no concept of PESSIMISTIC/OPTIMISTIC modes.
> > Instead,
> > > > all updates are "PESSIMISTIC", 

Re: Monitoring of active transactions

[Sasha Belyak]

It's very useful, but I often we need to get list of hang transaction when
exchange stopped by some reason and in this case utility, based on client
node won't help. I rewrite it as ComputeTask with default constructor, add
jar into ignite libs, activate http rest api and now it can be used from
console and no need to update cluster topology:

curl '
http://127.0.0.1:8080/ignite?cmd=exe=org.apache.ignite.txviewer.RestCollectTxInfoTask=false=100
'

{"successStatus":0,"sessionToken":null,"error":null,"response":{"id":"~98391a83-3d76-4e5e-b0c3-185cf2bd4acd","finished":true,"error":null,"result":[{"nearXidVersion":"GridCacheVersion
[topVer=118063514, order=1506583525449,
nodeOrder=2]","nodeId":"baa0237e-707c-4b69-abb6-555a2fc17762","nodeString":"TcpDiscoveryNode
[id=baa0237e-707c-4b69-abb6-555a2fc17762, addrs=[0:0:0:0:0:0:0:1%1,
10.0.3.1, 10.38.176.253, 10.42.1.107, 127.0.0.1, 172.17.0.1], sockAddrs=[/
127.0.0.1:0, /10.42.1.107:0, /0:0:0:0:0:0:0:1%1:0, /172.17.0.1:0, /
10.38.176.253:0, /10.0.3.1:0], discPort=0, order=2, intOrder=0,
lastExchangeTime=1506583525683, loc=true, ver=2.1.5#20170922-sha1:6452201d,
isClient=true]","threadId":1,"startTime":"Thu Sep 28 14:25:27 NOVT
2017","entries":[{"cache":"txCache","key":"1","value":"1","operation":"CREATE"}]}]}}

Even better if this tool can use binary rest too. I mean that we should be
able to run this collecting task from:
1) http rest api by curl/wget (but must deploy class somehow before, by
peerClassLoad with Continues mode or by adding it to application classpash)
Good for admin's console scripting.
2) binary rest api by some java tool (with instant peerClassLoading). Good
for investigation on any grid configuration.
3) maybe, by client node as it implemented now (can't see any adwantages)


2017-09-16 5:35 GMT+07:00 Dmitry Pavlov :

> Hi Ilya,
>
> I can help with including this utility into build/release, I've recenty
> finished same steps for PDS WAL analysing tool for converting records to
> human readable format.
> Please feel free to contact me.
>
> Sincerely,
> Dmitriy Pavlov
>
> пт, 15 сент. 2017 г. в 6:37, Dmitriy Setrakyan :
>
> > It seems that the community (including me) really would like to see this
> > feature in Ignite.
> >
> > Ilya, can you create a ticket and submit it for review?
> >
> > D.
> >
> > On Fri, Sep 8, 2017 at 7:15 AM, Anton Vinogradov  wrote:
> >
> > > Ilya,
> > >
> > > We extremely need this!
> > >
> > > Txs and Locks info should be collected on each cluster hang.
> > > We already have an issue related to this problem -
> > > https://issues.apache.org/jira/browse/IGNITE-4937
> > >
> > > Nikolay,
> > >
> > > Good point,
> > > but, seems you should start separate thread to discuss this.
> > >
> > > On Fri, Sep 8, 2017 at 4:28 PM, Dmitry Pavlov 
> > > wrote:
> > >
> > > > Hi Ilya,
> > > >
> > > > I'm definitely +1 for including the utility in the product. Perfect
> > > > contribution.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > пт, 8 сент. 2017 г. в 14:28, Ilya Lantukh :
> > > >
> > > > > Igniters,
> > > > >
> > > > > According to our current design and implementation, unclosed
> > > transaction
> > > > or
> > > > > unreleased lock can hang ignite cluster forever. This is logical,
> and
> > > > with
> > > > > correct usage of those mechanics such issue should never happen, in
> > > real
> > > > > world developers can make mistakes and leave transaction open. We
> > have
> > > a
> > > > > feature "transaction timeout", but turns out it doesn't work in all
> > > cases
> > > > > (see https://issues.apache.org/jira/browse/IGNITE-6181). Even if
> all
> > > > known
> > > > > issues are fixed, there is still a lot of room for mistake and
> > > incorrect
> > > > > usage.
> > > > >
> > > > > To make it possible for Ignite users to discover such problem and
> > trace
> > > > it
> > > > > to a particular part of code, I've created a very simple utility
> that
> > > > > collects and prints information about long running transactions for
> > the
> > > > > whole cluster. It is available here:
> > > > > https://github.com/ilantukh/IgniteTxViewer.
> > > > >
> > > > > One might expect such monitoring utilities to be included in Ignite
> > > > > codebase. Personally, I think that such information should be
> > available
> > > > > from public API, without using of additional applications or diving
> > > into
> > > > > Ignite internals.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ilya
> > > > >
> > > >
> > >
> >
>

[GitHub] ignite pull request #2767: IGNITE-6519 Race in SplitAwareTopologyValidator o...

[akuramshingg]

GitHub user akuramshingg opened a pull request:

https://github.com/apache/ignite/pull/2767

IGNITE-6519 Race in SplitAwareTopologyValidator on activator and server 
node join



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6519

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2767.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2767


commit b06b1f9de5814a778e7e42a3295f3fce5886ccdd
Author: Alexandr Kuramshin 
Date:   2017-09-28T08:57:10Z

IGNITE-6519 Race in SplitAwareTopologyValidator on activator and server 
node join




---

Re: Persistence per memory policy configuration

[Alexey Goncharuk]

My vote also goes for 1. I guess it is safe to assume that at this point we
came to a consensus?

2017-09-27 21:52 GMT+03:00 Denis Magda :

> Vote for 1.
>
> —
> Denis
>
> > On Sep 26, 2017, at 11:23 PM, Vladimir Ozerov 
> wrote:
> >
> > Folks,
> >
> > Let me summarize current naming ideas one more time:
> >
> > 1) [StorageConfiguration - StorageRegionConfiguration]
> > 2) [DurableMemoryConfiguration - DataRegionConfiguration]
> > 3) [DurableMemoryConfiguration - DurableMemoryRegionConfiguration] -
> out of
> > question, as "durable memory region" is too misleading.
> >
> > My vote for p.1. Short, simple and intuitive.
> >
> > Vladimir.
> >
> > On Tue, Sep 26, 2017 at 10:16 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> >> On Tue, Sep 26, 2017 at 7:33 AM, Dmitry Pavlov 
> >> wrote:
> >>
> >>> Hi Dmitriy, thank you for reply. Do you agree Memory Policy already
> >> became
> >>> Ignite's term? We call this configuration now
> >> MemoryPolicy(Configuration),
> >>> can we call new configuration elments by their existings name? We can
> >> avoid
> >>> introduction of second Ignite's term in that case.
> >>>
> >>
> >> The refactoring is about merging memory and persistence configuration
> under
> >> the same umbrella. The term "MemoryPolicy" does not make sense anymore,
> >> given that it now also includes persistent configuration as well.
> >>
> >>
> >>>
> >>> вт, 26 сент. 2017 г. в 17:27, Dmitriy Setrakyan  >:
> >>>
>  Dmitriy, we are not renaming classes, we are refactoring them. Prior
> to
>  this design, it was impossible to set persistence configuration on
>  per-cache basis. With this new design, users will be able to configure
> >>> some
>  caches to be in-memory only and others to be on disk.
> 
>  Given that we are already refactoring, it only makes sense to pick
> >>> better,
>  more appropriate names.
> 
>  D.
> 
>  On Tue, Sep 26, 2017 at 7:20 AM, Dmitry Pavlov  >
>  wrote:
> 
> > Vladimir, it is not clear for me, why we need to rename existing
> > configuration classes. Could you explain? And if we can't get
> >> consensus
> > now, should we pospond solution?
> >
> > My idea is that user needs this feature more than elegant names in
> > configuration.
> >
> > Moreover once MemoryPolicyConfiguration was introduced as Ignite term
> >>> it
>  is
> > simpler to keep it as is, than create new terms.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 26 сент. 2017 г. в 16:59, Vladimir Ozerov  >>> :
> >
> >> I do not understand why we should delay with renames. Yes, it will
>  cause
> >> questions, so we will have to put additional efforts to docs and
> > JavaDocs.
> >> But the earlier we do that, the better.
> >>
> >> On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <
> >>> dpavlov@gmail.com>
> >> wrote:
> >>
> >>> Hi Igniters, sorry for late response. I didn't catch idea of
>  renaming.
> >>> PersistentStoreConfiguration is intuitive, and
> > MemoryPolicyConfiguration
> >> is
> >>> intuitive also.
> >>>
> >>> If we rename these classes now, it will bring more questions to
> >>> user
> >> list.
> >>> Users may be confused by old and new names and by trying to match
> >>> it.
> >> More
> >>> issues can came from XML configs that users already have.
> >>>
> >>> Can we postpone the renaming? I suggest to finish 'persistence
> >> per
> > memory
> >>> policy' task without renaming, and create separate JIRA issue for
> >> creating
> >>> future decision?
> >>>
> >>> вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <
> >> alexey.goncha...@gmail.com
>  :
> >>>
>  I do not like DurableMemoryConfiguration, because it's quite
> > confusing
> >> -
> >>> we
>  configure in-memory caches using DurableMemory class, which
> > immediately
>  suggests that everything will be persisted. I am not sure if
> >> this
>  is
> > a
>  right wording choice for the documentation either. I would go
> >>> with
>  DataStoreConfiguration and DataRegionConfiguration.
> 
>  --AG
> 
>  2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <
> >>> dsetrak...@apache.org
> > :
> 
> > Given that we already have a notion of CacheStore which comes
>  from
> >>> JCache
> > spec, I think having other stores may get confusing. I like
> > DurableMemoryConfiguration.
> >
> > Other opinions?
> >
> > D.
> >
> > On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
> >>> voze...@gridgain.com>
> > wrote:
> >
> >> Dima, let's finalize the design first.
> 

[GitHub] ignite pull request #2660: IGNITE-6375

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2660


---

[jira] [Created] (IGNITE-6520) IgniteCacheGetRestartTest.testGetRestartReplicated fails sometimes (timeout)

[Andrey Kuznetsov (JIRA)]

Andrey Kuznetsov created IGNITE-6520:


 Summary: IgniteCacheGetRestartTest.testGetRestartReplicated fails 
sometimes (timeout)
 Key: IGNITE-6520
 URL: https://issues.apache.org/jira/browse/IGNITE-6520
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.2
Reporter: Andrey Kuznetsov
Assignee: Andrey Kuznetsov






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Apache Ignite 2.3 release

[Vladimir Ozerov]

Igniters,

I created our standard release page for AI 2.3 [1]. As nobody had
objections on dates, I propose to dedicate next week for stabilization and
start testing then.
Also I created release branch ignite-2.3. Let's make sure that effective of
next Tuesday, 3 Oct, we merge only release-related changes to this branch.

Vladimir.

[1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.3

On Thu, Sep 28, 2017 at 10:10 AM, Vladimir Ozerov 
wrote:

> Hi Pavel, Sergey.
>
> All mentioned tickets definitely important for the release. Let's do our
> best to fit them.
>
> On Mon, Sep 25, 2017 at 11:06 AM, schernolyas  > wrote:
>
>> Hi Vladimir!
>>
>> Could you add to future release by PR
>> (http://apache-ignite-developers.2346864.n4.nabble.com/
>> GitHub-ignite-pull-request-2737-IGNITE-6286-fix-org-h2-
>> jdbc-JdbcSQLException-td22625.html)?
>> The PR is very important for project Hibernate OGM .
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>
>
>

Re: Apache Ignite 2.3 release

[Vladimir Ozerov]

Hi Pavel, Sergey.

All mentioned tickets definitely important for the release. Let's do our
best to fit them.

On Mon, Sep 25, 2017 at 11:06 AM, schernolyas 
wrote:

> Hi Vladimir!
>
> Could you add to future release by PR
> (http://apache-ignite-developers.2346864.n4.nabble.com/GitHub-ignite-pull-
> request-2737-IGNITE-6286-fix-org-h2-jdbc-JdbcSQLException-td22625.html)?
> The PR is very important for project Hibernate OGM .
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[jira] [Created] (IGNITE-6519) Race in SplitAwareTopologyValidator on activator and server node join

[Alexandr Kuramshin (JIRA)]

Alexandr Kuramshin created IGNITE-6519:
--

 Summary: Race in SplitAwareTopologyValidator on activator and 
server node join
 Key: IGNITE-6519
 URL: https://issues.apache.org/jira/browse/IGNITE-6519
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.1
Reporter: Alexandr Kuramshin
Assignee: Alexandr Kuramshin


The following sequence may occur:

1. {{SplitAwareTopologyValidator}} detects split, gets {{NOTVALID}} and returns 
false from {{validate()}}

2. Activator node joins and {{SplitAwareTopologyValidator}} gets {{REPAIRED}}

3. Server node joins from other DC and it makes {{SplitAwareTopologyValidator}} 
gets {{VALID}}

4. Then the server node left the cluster and {{SplitAwareTopologyValidator}} 
should return false from {{validate()}} in cause of next split

But current implementation makes {{SplitAwareTopologyValidator}} 
auto-{{REPAIRED}}. Actually if the activator node will being forgotten to leave 
the cluster it may automatically repair a split many times. But it supposed to 
be manual operation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2766: Ignite 2.3.1

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/2766

Ignite 2.3.1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.3.1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2766.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2766


commit d22631e647280eff9f661dd7cfd492a6ba7460af
Author: tledkov-gridgain 
Date:   2017-08-23T12:46:23Z

IGNITE-6169: Fixed thin JDBC driver compatibility problem.

commit ac94426ced2cee5df2c993029e2a17ea5a9aa398
Author: Ilya Kasnacheev 
Date:   2017-08-23T13:26:58Z

IGNITE-4643: Fixed NPE in  JdbcDatabaseMetadata.getIndexInfo(). This closes 
#2481.

commit 0c5dca9a807c8f024e3477c1b18ddb3f237124b2
Author: sboikov 
Date:   2017-08-23T13:44:04Z

ignite-6124 Added missed initialization of merged join exchanges in 
GridDhtPartitionsExchangeFuture.onBecomeCoordinator

commit eee638d3da9a5dfbc7b43fd56eb4f72328c13a31
Author: sboikov 
Date:   2017-08-23T13:44:27Z

Merge remote-tracking branch 'origin/master'

commit 4fe8f762c2dc1b62bdcddd606b66dc67215b50a6
Author: sboikov 
Date:   2017-08-23T15:29:52Z

ignite-6174 Temporary changed test until issue not fixed

commit 46ec148ca9ae7ecc668c2c0bb9547140d05d68e2
Author: Yury Babak 
Date:   2017-08-23T17:53:08Z

ignite-5280 SparseDistributedMatrix refactoring

Signed-off-by: Andrey Gura 

commit e1bf8d7badee88c405b398ff5abb7e75bfaf2cde
Author: Alexey Kuznetsov 
Date:   2017-08-24T07:57:36Z

IGNITE-6136 Fixed version for demo.

commit 3ab523cd0f9c8bf962e6069596002204bcc49886
Author: sboikov 
Date:   2017-08-24T11:09:12Z

ignite-6124 DataStreamerImpl: do not wait for exchange future inside cache 
gateway.

commit 24306badde225ee4b3edec53b2ae8d3e1c0bff8d
Author: EdShangGG 
Date:   2017-08-24T16:15:24Z

ignite-6175 JVM Crash in Ignite Binary Objects Simple Mapper Basic suite

Signed-off-by: Andrey Gura 

commit e1eb1b97f541f944492f4860d33ad7899aafef87
Author: Andrey Novikov 
Date:   2017-08-25T03:48:15Z

IGNITE-5200 Web Console: Don't cache generated chunks in production.

commit 79d47f87383411319254c9182e09c30158cbd72d
Author: sboikov 
Date:   2017-08-25T08:07:57Z

ignite-4003 Preliminary changes in GridNioServer for async connect

commit 1c9d80a540cbce0a9d9a65e3fac2e06f53b73f43
Author: Igor Seliverstov 
Date:   2017-08-25T09:22:44Z

IGNITE-6101 Try to improve local scans performance

commit 6f279b004990d4a21436a80dd4bbf21c5210c282
Author: Ivan Rakov 
Date:   2017-08-24T15:18:31Z

IGNITE-6178 Make CheckpointWriteOrder.SEQUENTIAL and checkpointingThreads=4 
default in persistent store confguration

commit 02801f8f28e4936b2770a7a96d9d3809cce16d42
Author: Ilya Kasnacheev 
Date:   2017-08-25T12:32:05Z

Fixed "IGNITE-6168 Need SSL client authentication during discovery". This 
closes #2505.

Signed-off-by: nikolay_tikhonov 

commit 418c43889aac293cef8bcd6c5e4b90f85fc30e7c
Author: Ilya Lantukh 
Date:   2017-08-25T10:12:32Z

ignite-6180: restoring marshaller mappings on node start is implemented

Signed-off-by: Andrey Gura 

commit a1fab62c2d44730204e657ea25d1002669df90b1
Author: Ivan Rakov 
Date:   2017-08-25T17:09:43Z

IGNITE-6183 Make "Node crashed in the middle of checkpoint" message softer 
and more informative

Signed-off-by: Andrey Gura 

commit ebfa5557bf197e7cf0e6eebbee15d2f227dfc553
Author: Anton Vinogradov 
Date:   2017-08-28T15:12:25Z

Merge remote-tracking branch 'remotes/origin/ignite-2.1'

commit d20f302036206e6215d3ecea923747014f4e2bcf
Author: tledkov-gridgain 
Date:   2017-08-29T07:53:48Z

IGNITE-6118: JDBC thin driver: added lazy execution flag. This closes #2491.

commit 196aded9b8f51bf5700bb3a8b1b3ed0a1a16c9c6
Author: Andrey V. Mashenkov 
Date:   2017-08-29T09:36:30Z

ignite-5986 Fix failed test in .NET suite.

Signed-off-by: Andrey Gura 

commit 1494db9e020dd6a185b0005382455ff5ecb5246f
Author: Andrey Gura 
Date:   2017-08-29T10:14:02Z

Minor. Typo fixed.

commit 2cf12603c0e7fca84893d6c7341cabd23b737263
Author: Pavel Tupitsyn 
Date:   2017-08-29T11:16:00Z

IGNITE-6050 Fix eternal wait in DataStreamerTest.TestBufferSize