Two quick questions on index page and data page

[John Wilson]

Hi,


   1. An index page contains the hash value of key and a link; where a link
   is page_id + offset. Question: what is this offset? Is it the offset to the
   item in the data page? In other words, Ignite locates the page and the item
   within the page and finally gets the key-value pair by following the item?
   2. The design doc below states that the item serves as an internal
   reference (offset) to the key value pair. Does the item has information
   about the size of the key-value is it pointing to? If not, how does it now
   the size of the key-value.

https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood

Thanks,
John

Re: Service grid redesign

[Dmitriy Setrakyan]

On Tue, Jun 19, 2018 at 1:50 PM, Vyacheslav Daradur 
wrote:

> Hi Dmitriy,
>
> Yes, the task [1] is planned to be implemented once the main tasks
> will be completed.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-8367


Awesome! This is a huge addition to the project.


>
>
> On Tue, Jun 19, 2018 at 10:06 PM Dmitriy Setrakyan
>  wrote:
> >
> > Hi Vyacheslav,
> >
> > How about service redeployment in case if user wants to update the code?
> Is
> > this planned?
> >
> > D.
> >
>

Re: Service grid redesign

[Vyacheslav Daradur]

Hi Dmitriy,

Yes, the task [1] is planned to be implemented once the main tasks
will be completed.

[1] https://issues.apache.org/jira/browse/IGNITE-8367

On Tue, Jun 19, 2018 at 10:06 PM Dmitriy Setrakyan
 wrote:
>
> Hi Vyacheslav,
>
> How about service redeployment in case if user wants to update the code? Is
> this planned?
>
> D.
>
> On Tue, Jun 19, 2018 at 12:10 AM, Vyacheslav Daradur 
> wrote:
>
> > Hi Denis!
> >
> > >> If to assume that Ignite 2.7 gets released in September, what else
> > would you be able to complete by that time?
> >
> > The following tasks will be completed by September:
> > https://issues.apache.org/jira/browse/IGNITE-8361 - Use discovery
> > messages for service deployment
> > https://issues.apache.org/jira/browse/IGNITE-8362 - Collect service
> > deployment results asynchronously on coordinator
> > https://issues.apache.org/jira/browse/IGNITE-3392 - Propagate service
> > deployment results from assigned nodes to initiator
> > https://issues.apache.org/jira/browse/IGNITE-8366 - Replace service
> > instance parameter with a class name in ServiceConfiguration
> >
> > The following tasks will be completed by the end of September:
> > https://issues.apache.org/jira/browse/IGNITE-8363 - Handle topology
> > changes during service deployment
> > https://issues.apache.org/jira/browse/IGNITE-8364 - Propagate deployed
> > services to joining nodes
> >
> > >> Is someone else going to help you with this initiative?
> >
> > Yes, Nikita Amelchev is going to help at least with IGNITE-8366 and
> > with testing.
> >
> > >> Plus, do you think you'll be able to complete the whole IEP by the end
> > of the year?
> >
> > Definitely, I intend to complete the IEP-17 by the end of the year.
> >
> >
> > On Fri, Jun 15, 2018 at 9:29 PM Denis Magda  wrote:
> > >
> > > Hi Vyacheslav,
> > >
> > > If to assume that Ignite 2.7 gets released in September, what else would
> > > you be able to complete by that time? I would see how to promote this
> > from
> > > the community side (articles, webinars, meetups).
> > >
> > > Plus, do you think you'll be able to complete the whole IEP by the end of
> > > the year? Is someone else going to help you with this initiative? That
> > > would be a huge New Year gift to Ignite community.
> > >
> > > --
> > > Denis
> > >
> > > On Thu, Jun 7, 2018 at 11:51 AM Vyacheslav Daradur 
> > > wrote:
> > >
> > > > Hi Igniters, sorry for the delay in replying.
> > > >
> > > > I going to finish the task [1] to the end of this month.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-8361
> > > >
> > > > On Wed, May 16, 2018 at 10:57 PM, Vyacheslav Daradur
> > > >  wrote:
> > > > > Hi, Igniters!
> > > > >
> > > > > I had a discussion about the scope of work of IEP-17 with Denis and
> > > > Pavel.
> > > > >
> > > > > To estimate the time I took 2 weeks for R during which I would do
> > the
> > > > task:
> > > > > https://issues.apache.org/jira/browse/IGNITE-8361
> > > > >
> > > > > Then I will inform the community about the planned work time.
> > > > >
> > > > >
> > > > > On Tue, May 8, 2018 at 9:50 PM, Vyacheslav Daradur <
> > daradu...@gmail.com>
> > > > wrote:
> > > > >> Thanks, Denis! I assigned the task to myself.
> > > > >>
> > > > >> I going to start work next week.
> > > > >>
> > > > >> On Tue, May 8, 2018 at 7:50 PM, Denis Mekhanikov <
> > dmekhani...@gmail.com>
> > > > wrote:
> > > > >>> Hi Vyacheslav!
> > > > >>>
> > > > >>> Thanks for the enthusiasm!
> > > > >>> The first ticket to be implemented here is IGNITE-8361
> > > > >>> .
> > > > >>> I think, until this one is resolved, other tasks can't be started.
> > > > >>>
> > > > >>> Please assign it to yourself, if you feel confident enough.
> > > > >>> Feel free to ask for any help, if you need.
> > > > >>>
> > > > >>> For now it will be enough to perform service deployment and
> > > > initialization
> > > > >>> in the discovery thread.
> > > > >>> Making it asynchronous will be our next step: IGNITE-8362
> > > > >>> 
> > > > >>>
> > > > >>> Denis
> > > > >>>
> > > > >>> вт, 24 апр. 2018 г. в 15:43, Vyacheslav Daradur <
> > daradu...@gmail.com>:
> > > > >>>
> > > >  Hi, Denis M.,
> > > > 
> > > >  I'd like to pick up a ticket from IEP-17 next week.
> > > > 
> > > >  Could you please advise a ticket to start?
> > > > 
> > > >  On Tue, Apr 24, 2018 at 11:47 AM, Dmitriy Setrakyan
> > > >   wrote:
> > > >  > On Tue, Apr 24, 2018, 3:59 PM Denis Mekhanikov <
> > > > dmekhani...@gmail.com>
> > > >  > wrote:
> > > >  >
> > > >  >> Dmitriy,
> > > >  >>
> > > >  >> After the proposed changes are made the utility cache won't be
> > > > needed at
> > > >  >> all.
> > > >  >>
> > > >  >
> > > >  > I was rather talking about prioritization. In my view, first and
> > > > foremost
> > > >  > we must fix deployment before anything else.
> > > >  >
> 

Re: Quick questions on segments and page map buckets

[Eduard Shangareev]

Hi,

1. It looks weird, yeah. Need to ask Sergey, who has changed it last time.

2. Because we could reuse memory. For example, after cache destroy or
something like that.

On Tue, Jun 19, 2018 at 9:58 PM, John Wilson 
wrote:

> Hi,
>
> Two quick questions:
>
>
>1. The design documentation here,
>https://cwiki.apache.org/confluence/display/IGNITE/
> Ignite+Durable+Memory+-+under+the+hood,
>states that the default segment count is equal to the number of logical
>cores available in the underlying machine. However, the segments array
> in
>PageMemory indicates that the maximum number of segments is: 1 <<
> SEG_BITS.
>Since SEG_BITS = 4, the max # segments is 16. Did I miss something here?
>2. Reading the code in PageMemoryNoStoreImp, it looks like pages are
>allocated segment sequentially in a bump-the-pointer strategy where the
>first 8 bytes of a segment hold a pointer to the index of the last
>allocated page. If this is true, then I don't understand the point of
>having a page map buckets. Why not use a simple arithmetic index *
> pageSize
>to get the offset of a page?
>
> Thanks.
> John
>

Quick questions on segments and page map buckets

[John Wilson]

Hi,

Two quick questions:


   1. The design documentation here,
   
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood,
   states that the default segment count is equal to the number of logical
   cores available in the underlying machine. However, the segments array in
   PageMemory indicates that the maximum number of segments is: 1 << SEG_BITS.
   Since SEG_BITS = 4, the max # segments is 16. Did I miss something here?
   2. Reading the code in PageMemoryNoStoreImp, it looks like pages are
   allocated segment sequentially in a bump-the-pointer strategy where the
   first 8 bytes of a segment hold a pointer to the index of the last
   allocated page. If this is true, then I don't understand the point of
   having a page map buckets. Why not use a simple arithmetic index * pageSize
   to get the offset of a page?

Thanks.
John

[GitHub] ignite pull request #4209: IGNITE-8808 Improve control.sh --tx command to sh...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4209


---

[GitHub] ignite pull request #4094: IGNITE-8554

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4094


---

[GitHub] ignite pull request #4227: IGNITE-8768 Implemented eviction stopping to prev...

[Jokser]

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/4227

IGNITE-8768 Implemented eviction stopping to prevent possible JVM crush



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8768

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4227.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4227


commit 2cb2cf9c1053a28f636b94936a685344fa29e1b5
Author: Pavel Kovalenko 
Date:   2018-06-18T15:55:56Z

IGNITE-8768 WIP

commit 17d33eb98490e5a714f88e295b89ac79dddbee43
Author: Pavel Kovalenko 
Date:   2018-06-19T14:26:23Z

IGNITE-8768 Implemented eviction stop during cache group stopping. Reworked 
evictor class.

commit 5a3e74a9b4a2bf93c865c4eeb94cf914d2da69e2
Author: Pavel Kovalenko 
Date:   2018-06-19T14:37:34Z

IGNITE-8768 Complete eviction future before log error.




---

[jira] [Created] (IGNITE-8836) NullPointerException during Ignition.start prevents JVM shutdown

[Anton Kurbanov (JIRA)]

Anton Kurbanov created IGNITE-8836:
--

 Summary: NullPointerException during Ignition.start prevents JVM 
shutdown
 Key: IGNITE-8836
 URL: https://issues.apache.org/jira/browse/IGNITE-8836
 Project: Ignite
  Issue Type: Bug
Reporter: Anton Kurbanov
 Fix For: 1.9


The exception like below leaves node in stopping state forever:

java.lang.NullPointerException: null
at 
java.util.concurrent.ConcurrentLinkedQueue.checkNotNull(ConcurrentLinkedQueue.java:914)
at 
java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:327)
at 
java.util.concurrent.ConcurrentLinkedQueue.add(ConcurrentLinkedQueue.java:297)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateResponse.addFailedKey(GridNearAtomicUpdateResponse.java:347)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicSingleUpdateFuture.addFailedKeys(GridDhtAtomicSingleUpdateFuture.java:166)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:446)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:56)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:345)
at 
org.apache.ignite.internal.processors.cache.GridCacheMvccManager.cancelClientFutures(GridCacheMvccManager.java:388)
at 
org.apache.ignite.internal.processors.cache.GridCacheMvccManager.onStop(GridCacheMvccManager.java:370)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:956)
at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2095)
at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2041)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2397)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2360)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:1871)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8835) Do not skip distributed phase of 2-phase partition release if there are some caches to stop / modify

[Pavel Kovalenko (JIRA)]

Pavel Kovalenko created IGNITE-8835:
---

 Summary: Do not skip distributed phase of 2-phase partition 
release if there are some caches to stop / modify
 Key: IGNITE-8835
 URL: https://issues.apache.org/jira/browse/IGNITE-8835
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.5
Reporter: Pavel Kovalenko
Assignee: Pavel Kovalenko
 Fix For: 2.6


If we don't perform distributed 2-phase in case of cache stop, we can lost some 
transactional updates from primary to backup.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8834) SqlQuery not throwing exception after partition loss events

[Anton Kurbanov (JIRA)]

Anton Kurbanov created IGNITE-8834:
--

 Summary: SqlQuery not throwing exception after partition loss 
events
 Key: IGNITE-8834
 URL: https://issues.apache.org/jira/browse/IGNITE-8834
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.4
Reporter: Anton Kurbanov


 

Precondition: 3 server nodes, client listening for partition loss events, 
partitioned cache with backups = 1, partition loss policy = READ_ONLY_SAFE, 
stream some data. Kill 2 nodes, call:
{code:java}
SqlQuery query = new SqlQuery<>(Integer.class, "where 
_key>0");
query.setLocal(false);
List> list = cache.query(query).getAll();
{code}
Cache configuration:
{code:java}
public IgniteConfiguration getCfg() {
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setConsistentId(cfg.getIgniteInstanceName());
cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST);

TcpDiscoverySpi discovery = new TcpDiscoverySpi();
TcpDiscoveryVmIpFinder finder = new TcpDiscoveryVmIpFinder();
finder.setAddresses(Arrays.asList("127.0.0.1:47500..47509"));
discovery.setIpFinder(finder);
cfg.setDiscoverySpi(discovery);

QueryEntity queryEntity = new QueryEntity();

queryEntity.setKeyType(Integer.class.getName());
queryEntity.setValueType(Integer.class.getName());

cfg.setCacheConfiguration(new CacheConfiguration(CACHE)
.setCacheMode(CacheMode.PARTITIONED)
.setBackups(1)
.setAffinity(new RendezvousAffinityFunction())
.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC)
.setPartitionLossPolicy(PartitionLossPolicy.READ_ONLY_SAFE)
.setQueryEntities(Collections.singletonList(queryEntity)));

DataStorageConfiguration storageCfg = new DataStorageConfiguration();
storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(true);
cfg.setDataStorageConfiguration(storageCfg);

return cfg;
}
{code}
Query is expected to fail, but succeeds and returns entries from partitions 
that are alive.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8833) IgniteCache.isLocalLocked method has unexpected behivior in case of several nodes started in one JVM in different threads

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8833:
--

 Summary: IgniteCache.isLocalLocked method has unexpected behivior 
in case of several nodes started in one JVM in different threads
 Key: IGNITE-8833
 URL: https://issues.apache.org/jira/browse/IGNITE-8833
 Project: Ignite
  Issue Type: Bug
  Components: cache, documentation
Affects Versions: 2.5
Reporter: Andrey Aleksandrov
 Fix For: 2.6
 Attachments: ThreadLockedTest.java

According to specification:

[https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#isLocalLocked-K-boolean-]


Checks if specified key is locked.This is a local in-VM operation and does not 
involve any network trips or access to persistent storage in any way.

Parameters:

{{key}} - Key to check.

{{byCurrThread}} - If {{true}} method will check that current thread owns a 
lock on this key, other vise will check that any thread on any node owns a lock 
on this key.

Returns:{{True}} if lock is owned by some node.

In the attached test we start one node in the main thread and another node from 
the second thread. In second node we take a lock but in main thread 
isLocalLocked shows that no thread held the lock.

However tryLock works ok. So the behavior of the isLocalLocked method should be 
described in this case or fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4226: IGNITE-4939 Receive event before cache initialize...

[ezhuravl]

GitHub user ezhuravl opened a pull request:

https://github.com/apache/ignite/pull/4226

IGNITE-4939 Receive event before cache initialized fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4939

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4226.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4226


commit 704f0fda313f559b810da9278fc120916c80795e
Author: ezhuravl 
Date:   2018-06-18T09:19:39Z

IGNITE-4939 Receive event before cache initialized fix




---

[GitHub] ignite pull request #4206: IGNITE-8769: JVM crash in Basic1 suite in master ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4206


---

[jira] [Created] (IGNITE-8832) Detecting hanging of coordinator during processing local partition maps messages

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-8832:
---

 Summary: Detecting hanging of coordinator during processing local 
partition maps messages
 Key: IGNITE-8832
 URL: https://issues.apache.org/jira/browse/IGNITE-8832
 Project: Ignite
  Issue Type: Improvement
  Components: general
Reporter: Sergey Chugunov


After coordinator gathered local partition maps from all other server nodes it 
should prepare full partition map and send it back.

If for some reason (e.g. bug in code) coordinator fails to prepare full map it 
should detect this situation and shut down itself.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4225: IGNITE-8827 disable WAL logging

[DmitriyGovorukhin]

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/4225

IGNITE-8827 disable WAL logging



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8827

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4225.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4225


commit 2250a923330127b2819bf50b4faeb40764f01762
Author: Dmitriy Govorukhin 
Date:   2018-06-19T11:57:48Z

IGNITE-8827 disable WAL logging




---

[GitHub] ignite pull request #4224: MarshallerMappingFileStore: Incorrect locks on fi...

[SpiderRus]

GitHub user SpiderRus opened a pull request:

https://github.com/apache/ignite/pull/4224

MarshallerMappingFileStore: Incorrect locks on files



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8831

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4224.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4224


commit 7bf8bba177daddac672bdcd4a4839a7a3b1e4f39
Author: Alexey Stelmak 
Date:   2018-06-19T12:08:20Z

Update locks




---

[jira] [Created] (IGNITE-8831) MarshallerMappingFileStore: Incorrect locks on files

[Alexey Stelmak (JIRA)]

Alexey Stelmak created IGNITE-8831:
--

 Summary: MarshallerMappingFileStore: Incorrect locks on files
 Key: IGNITE-8831
 URL: https://issues.apache.org/jira/browse/IGNITE-8831
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Stelmak






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8830) Non-coordinator node unable to finish local exchange should detect it and stop

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-8830:
---

 Summary: Non-coordinator node unable to finish local exchange 
should detect it and stop
 Key: IGNITE-8830
 URL: https://issues.apache.org/jira/browse/IGNITE-8830
 Project: Ignite
  Issue Type: Improvement
Reporter: Sergey Chugunov


At the final stage of Partition Map Exchange coordinator node sends full 
partitions map to all other nodes.

If some node fails to apply this message and finish its local exchange it won't 
be able to operate correctly.
To prevent this node should be able to check status of this exchange on 
coordinator (by sending some diagnostic message). If the exchange has already 
finished on coordinator, node should stop.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8829) Some configuration properties of TcpCommunicationSpi does not annotated appropriately

[Vladislav Pyatkov (JIRA)]

Vladislav Pyatkov created IGNITE-8829:
-

 Summary: Some configuration properties of TcpCommunicationSpi does 
not annotated appropriately
 Key: IGNITE-8829
 URL: https://issues.apache.org/jira/browse/IGNITE-8829
 Project: Ignite
  Issue Type: Bug
Reporter: Vladislav Pyatkov


When I checked all properties of TcpCommunicationSpi, I have found an issue 
with getting all configuration properties from code. Because a part of them not 
be a configured property, but a part of a real SPI life.

I should was rid of these issues - all configurable properties must annotate as 
{{IgniteSpiConfiguration}}, but it not done for each.

I have found at least two properties for which not be done:
{{connectionsPerNode}}
{{usePairedConnections}}

and one property which not appropriate contract (it have only setter, but not 
getter):
{{addressResolver}}

Need to revised all properties CommunicationSpi and correct them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8828) Detecting and stopping unresponsive nodes during Partition Map Exchange

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-8828:
---

 Summary: Detecting and stopping unresponsive nodes during 
Partition Map Exchange
 Key: IGNITE-8828
 URL: https://issues.apache.org/jira/browse/IGNITE-8828
 Project: Ignite
  Issue Type: Improvement
Reporter: Sergey Chugunov


During PME process coordinator (1) gathers local partition maps from all nodes 
and (2) sends calculated full partition map back to all nodes in the topology.

However if one or more nodes fail to send local information on step 1 for any 
reason, PME process hangs blocking all operations. The only solution will be to 
manually identify and stop nodes which failed to send info to coordinator.

This should be done by coordinator itself: in case it didn't receive in time 
local partition maps from any nodes, it should check that stopping these nodes 
won't lead to data loss and then stop them forcibly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Entries are not loaded into cache during cache store loading

[Aleksey Kuznetsov]

Hi, Nikolay.

Yes.

вт, 19 июн. 2018 г. в 13:18, Nikolay Izhikov :

> Hello, Aleksey.
>
> By "cache store" you mean 3-d party cache store [1] ?
>
> https://apacheignite.readme.io/docs/3rd-party-store
>
> В Вт, 19/06/2018 в 13:10 +0300, Aleksey Kuznetsov пишет:
> > Hi, Igniters!
> >
> > During test bug fixing [1], I stumbled on the following issue:
> > When we perform cache store loading and start several nodes
> > simulateneously, some entries might not be loaded into cache.
> >
> > Entry is loaded in `GridDhtCacheAdapter#loadEntry`, but it fails on
> > `ctx.group().topology().localPartition()`
> > because partition is moved to another node during rebalancing.
> >
> > One possible solution to this issue is to block PME if cache store
> loading
> > is in progress.
> > But there is drawback: PME might hang for a long period and will lead to
> > massive data rebalancing.
> >
> > Another solution is to cancel cache store loading if PME is started.
> > Exception should be thrown to user, so he has got possibility to
> reinitiate
> > cache store loading procedure.
> >
> > Personally, I like first solution more: user might prefer PME hang than
> > canceling cache store loading.
> > I think, we should warn user about PME hang when he starts cache store
> > loading.
> >
> > What do u think on this issue ?
> >
> > [1] : https://issues.apache.org/jira/browse/IGNITE-4210

Re: Entries are not loaded into cache during cache store loading

[Nikolay Izhikov]

Hello, Aleksey.

By "cache store" you mean 3-d party cache store [1] ?

https://apacheignite.readme.io/docs/3rd-party-store

В Вт, 19/06/2018 в 13:10 +0300, Aleksey Kuznetsov пишет:
> Hi, Igniters!
> 
> During test bug fixing [1], I stumbled on the following issue:
> When we perform cache store loading and start several nodes
> simulateneously, some entries might not be loaded into cache.
> 
> Entry is loaded in `GridDhtCacheAdapter#loadEntry`, but it fails on
> `ctx.group().topology().localPartition()`
> because partition is moved to another node during rebalancing.
> 
> One possible solution to this issue is to block PME if cache store loading
> is in progress.
> But there is drawback: PME might hang for a long period and will lead to
> massive data rebalancing.
> 
> Another solution is to cancel cache store loading if PME is started.
> Exception should be thrown to user, so he has got possibility to reinitiate
> cache store loading procedure.
> 
> Personally, I like first solution more: user might prefer PME hang than
> canceling cache store loading.
> I think, we should warn user about PME hang when he starts cache store
> loading.
> 
> What do u think on this issue ?
> 
> [1] : https://issues.apache.org/jira/browse/IGNITE-4210

signature.asc
Description: This is a digitally signed message part

Entries are not loaded into cache during cache store loading

[Aleksey Kuznetsov]

Hi, Igniters!

During test bug fixing [1], I stumbled on the following issue:
When we perform cache store loading and start several nodes
simulateneously, some entries might not be loaded into cache.

Entry is loaded in `GridDhtCacheAdapter#loadEntry`, but it fails on
`ctx.group().topology().localPartition()`
because partition is moved to another node during rebalancing.

One possible solution to this issue is to block PME if cache store loading
is in progress.
But there is drawback: PME might hang for a long period and will lead to
massive data rebalancing.

Another solution is to cancel cache store loading if PME is started.
Exception should be thrown to user, so he has got possibility to reinitiate
cache store loading procedure.

Personally, I like first solution more: user might prefer PME hang than
canceling cache store loading.
I think, we should warn user about PME hang when he starts cache store
loading.

What do u think on this issue ?

[1] : https://issues.apache.org/jira/browse/IGNITE-4210

[GitHub] ignite pull request #4216: IGNITE-8749: after-merge code formatting.

[x-kreator]

Github user x-kreator closed the pull request at:

https://github.com/apache/ignite/pull/4216


---

[GitHub] ignite pull request #4220: IGNITE-8749: storage error handling fix.

[x-kreator]

Github user x-kreator closed the pull request at:

https://github.com/apache/ignite/pull/4220


---

[GitHub] ignite pull request #4223: Ignite-gg-13938

[DmitriyGovorukhin]

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/4223

Ignite-gg-13938



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-13938

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4223.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4223






---

[jira] [Created] (IGNITE-8827) Disable WAL during apply updates on recovery

[Dmitriy Govorukhin (JIRA)]

Dmitriy Govorukhin created IGNITE-8827:
--

 Summary: Disable WAL during apply updates on recovery
 Key: IGNITE-8827
 URL: https://issues.apache.org/jira/browse/IGNITE-8827
 Project: Ignite
  Issue Type: Bug
Reporter: Dmitriy Govorukhin






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Best Effort Affinity for thin clients

[Vladimir Ozerov]

Denis,

Yes, in principle we can extend it. We are going to implement it in
subsequent phases of this IEP.

On Tue, Jun 19, 2018 at 4:30 AM, Dmitriy Setrakyan 
wrote:

> On Mon, Jun 18, 2018 at 11:07 AM, Denis Magda  wrote:
>
> > Folks,
> >
> > Feel that this functionality can be extended to the automatic reconnect,
> > can't it? Presently we require to provide a static list of IPs to be used
> > at a reconnect time. By having a partition map of all the nodes, the thin
> > client should be able to automate this piece.
> >
>
> Not sure if static IP list can be avoided. What Igor is suggesting is that
> we try to pick the best node out of the static IP  list.
>
> D.
>

[GitHub] ignite pull request #4222: Ignite 8826: added missed licences

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4222


---

[GitHub] ignite pull request #4222: Ignite 8826: added missed licences

[ybabak]

GitHub user ybabak opened a pull request:

https://github.com/apache/ignite/pull/4222

Ignite 8826: added missed licences

fix

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8826

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4222.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4222


commit 32f24ea553e0a5ad725c4279c5674134523f1bc7
Author: YuriBabak 
Date:   2018-06-19T08:30:31Z

IGNITE-8826: Added missed licence

fix

commit 3e41124aa6471d9afd01d0dc1bb2bd0adc108777
Author: YuriBabak 
Date:   2018-06-19T08:40:04Z

IGNITE-8826: Added missed licence

fix




---

[jira] [Created] (IGNITE-8825) Ignite - Configuration for IGFS

[Puviarasu (JIRA)]

Puviarasu created IGNITE-8825:
-

 Summary: Ignite - Configuration for IGFS 
 Key: IGNITE-8825
 URL: https://issues.apache.org/jira/browse/IGNITE-8825
 Project: Ignite
  Issue Type: Test
Reporter: Puviarasu


Default configuration not working for IGFS



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Service grid redesign

[Vyacheslav Daradur]

Hi Denis!

>> If to assume that Ignite 2.7 gets released in September, what else would you 
>> be able to complete by that time?

The following tasks will be completed by September:
https://issues.apache.org/jira/browse/IGNITE-8361 - Use discovery
messages for service deployment
https://issues.apache.org/jira/browse/IGNITE-8362 - Collect service
deployment results asynchronously on coordinator
https://issues.apache.org/jira/browse/IGNITE-3392 - Propagate service
deployment results from assigned nodes to initiator
https://issues.apache.org/jira/browse/IGNITE-8366 - Replace service
instance parameter with a class name in ServiceConfiguration

The following tasks will be completed by the end of September:
https://issues.apache.org/jira/browse/IGNITE-8363 - Handle topology
changes during service deployment
https://issues.apache.org/jira/browse/IGNITE-8364 - Propagate deployed
services to joining nodes

>> Is someone else going to help you with this initiative?

Yes, Nikita Amelchev is going to help at least with IGNITE-8366 and
with testing.

>> Plus, do you think you'll be able to complete the whole IEP by the end of 
>> the year?

Definitely, I intend to complete the IEP-17 by the end of the year.


On Fri, Jun 15, 2018 at 9:29 PM Denis Magda  wrote:
>
> Hi Vyacheslav,
>
> If to assume that Ignite 2.7 gets released in September, what else would
> you be able to complete by that time? I would see how to promote this from
> the community side (articles, webinars, meetups).
>
> Plus, do you think you'll be able to complete the whole IEP by the end of
> the year? Is someone else going to help you with this initiative? That
> would be a huge New Year gift to Ignite community.
>
> --
> Denis
>
> On Thu, Jun 7, 2018 at 11:51 AM Vyacheslav Daradur 
> wrote:
>
> > Hi Igniters, sorry for the delay in replying.
> >
> > I going to finish the task [1] to the end of this month.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-8361
> >
> > On Wed, May 16, 2018 at 10:57 PM, Vyacheslav Daradur
> >  wrote:
> > > Hi, Igniters!
> > >
> > > I had a discussion about the scope of work of IEP-17 with Denis and
> > Pavel.
> > >
> > > To estimate the time I took 2 weeks for R during which I would do the
> > task:
> > > https://issues.apache.org/jira/browse/IGNITE-8361
> > >
> > > Then I will inform the community about the planned work time.
> > >
> > >
> > > On Tue, May 8, 2018 at 9:50 PM, Vyacheslav Daradur 
> > wrote:
> > >> Thanks, Denis! I assigned the task to myself.
> > >>
> > >> I going to start work next week.
> > >>
> > >> On Tue, May 8, 2018 at 7:50 PM, Denis Mekhanikov 
> > wrote:
> > >>> Hi Vyacheslav!
> > >>>
> > >>> Thanks for the enthusiasm!
> > >>> The first ticket to be implemented here is IGNITE-8361
> > >>> .
> > >>> I think, until this one is resolved, other tasks can't be started.
> > >>>
> > >>> Please assign it to yourself, if you feel confident enough.
> > >>> Feel free to ask for any help, if you need.
> > >>>
> > >>> For now it will be enough to perform service deployment and
> > initialization
> > >>> in the discovery thread.
> > >>> Making it asynchronous will be our next step: IGNITE-8362
> > >>> 
> > >>>
> > >>> Denis
> > >>>
> > >>> вт, 24 апр. 2018 г. в 15:43, Vyacheslav Daradur :
> > >>>
> >  Hi, Denis M.,
> > 
> >  I'd like to pick up a ticket from IEP-17 next week.
> > 
> >  Could you please advise a ticket to start?
> > 
> >  On Tue, Apr 24, 2018 at 11:47 AM, Dmitriy Setrakyan
> >   wrote:
> >  > On Tue, Apr 24, 2018, 3:59 PM Denis Mekhanikov <
> > dmekhani...@gmail.com>
> >  > wrote:
> >  >
> >  >> Dmitriy,
> >  >>
> >  >> After the proposed changes are made the utility cache won't be
> > needed at
> >  >> all.
> >  >>
> >  >
> >  > I was rather talking about prioritization. In my view, first and
> > foremost
> >  > we must fix deployment before anything else.
> >  >
> >  > D.
> > 
> > 
> > 
> >  --
> >  Best Regards, Vyacheslav D.
> > 
> > >>
> > >>
> > >>
> > >> --
> > >> Best Regards, Vyacheslav D.
> > >
> > >
> > >
> > > --
> > > Best Regards, Vyacheslav D.
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
> >



-- 
Best Regards, Vyacheslav D.

[GitHub] ignite pull request #4221: IGNITE-8428 Web Console: Implemented connection t...

[akuznetsov-gridgain]

GitHub user akuznetsov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/4221

IGNITE-8428 Web Console: Implemented connection to secured cluster.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8428

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4221.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4221


commit a70d0335e661bbfb69c68b8a38c10c289dab261f
Author: Alexey Kuznetsov 
Date:   2018-06-19T04:47:03Z

IGNITE-8428 Web Console: Implemented connection to secured cluster.

commit ba452276d918fb258fa48b50b32303c47e19561c
Author: Alexey Kuznetsov 
Date:   2018-06-19T06:29:06Z

IGNITE-8428 Minor cleanup.




---