[jira] [Commented] (IGNITE-12285) Remove boilerplate code in test PluginProvider implementations.
[
https://issues.apache.org/jira/browse/IGNITE-12285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019891#comment-17019891
]
Ignite TC Bot commented on IGNITE-12285:
{panel:title=Branch: [pull/6969/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4945092buildTypeId=IgniteTests24Java8_RunAll]
> Remove boilerplate code in test PluginProvider implementations.
> ---
>
> Key: IGNITE-12285
> URL: https://issues.apache.org/jira/browse/IGNITE-12285
> Project: Ignite
> Issue Type: Improvement
>Reporter: PetrovMikhail
>Assignee: PetrovMikhail
>Priority: Minor
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> It's needed to remove boilerplate code in test PluginProvider implementations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12554) Ignite instance couldn't be restarted if it's parent thread group has been called destroy once
[
https://issues.apache.org/jira/browse/IGNITE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019796#comment-17019796
]
ha faculty commented on IGNITE-12554:
-
Hi Ivan,
Unfortunately our in-house platform which is using a 3rd party framework
which don’t allow using the system threadgroup. Thanks for your reply. Let me
see if assign a new DFLT_GRP instance every time when ignition start will
resolve it or not
> Ignite instance couldn't be restarted if it's parent thread group has been
> called destroy once
> --
>
> Key: IGNITE-12554
> URL: https://issues.apache.org/jira/browse/IGNITE-12554
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.7.6
>Reporter: ha faculty
>Priority: Major
> Attachments: NodeRestartTesting.java
>
>
> I am using an inhouse platform which manage the life cycle of ignite instance
> by calling ignition.start() and ignition.stop() from a web application.
> it could be started without issue for the first time. however, if it stop the
> ignite by calling ignition.stop() and followed by destroying the parent
> thread group which starting it.
> Calling ignition.start() in the 2nd time will throw the following exception
> Exception in thread "kickOff" java.lang.IllegalThreadStateExceptionException
> in thread "kickOff" java.lang.IllegalThreadStateException at
> java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at
> java.lang.Thread.init(Thread.java:405) at
> java.lang.Thread.init(Thread.java:349) at
> java.lang.Thread.(Thread.java:599) at
> org.apache.ignite.thread.IgniteThread.(IgniteThread.java:96) at
> org.apache.ignite.internal.util.StripedExecutor$Stripe.start(StripedExecutor.java:474)
> at
> org.apache.ignite.internal.util.StripedExecutor.(StripedExecutor.java:121)
> at
> org.apache.ignite.internal.util.StripedExecutor.(StripedExecutor.java:80)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1842)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:589) at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:565) at
> org.apache.ignite.Ignition.start(Ignition.java:305) at
> com.faculty.poc.ignite.app.NodeRestartTesting.run(NodeRestartTesting.java:60)
> at java.lang.Thread.run(Thread.java:748)
> Please find my demo code for this issue.
> Walk through into Ignite code, it should be caused by a static thread group
> created in IgniteThread class. (private static final ThreadGroup DFLT_GRP =
> new ThreadGroup("ignite")
> Destroyed the parent thread group which start the ignite instance will result
> in causing the DFLT_GRP go into destroyed state. Therefore, when Ignite was
> start ifor the 2nd time, this static thread group has been destroyed and no
> longer able to run the ignite.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Deleted] (IGNITE-12556) Online Dev Tools
[ https://issues.apache.org/jira/browse/IGNITE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Kasnacheev deleted IGNITE-12556: - > Online Dev Tools > > > Key: IGNITE-12556 > URL: https://issues.apache.org/jira/browse/IGNITE-12556 > Project: Ignite > Issue Type: Bug > Environment: https://onlinedevtools.in > check this link > useful web developer tools link > https://onlinedevtools.in/lodash_underscore_alternative > https://onlinedevtools.in/online/sqlformatter > https://onlinedevtools.in/online/npmpackageanalyzer > https://onlinedevtools.in/curl > https://onlinedevtools.in/crontab > https://onlinedevtools.in/devnews > https://diffviewer.onlinedevtools.in >Reporter: Stephen >Priority: Major > > https://onlinedevtools.in > check this link > useful web developer tools link > https://onlinedevtools.in/lodash_underscore_alternative > https://onlinedevtools.in/online/sqlformatter > https://onlinedevtools.in/online/npmpackageanalyzer > https://onlinedevtools.in/curl > https://onlinedevtools.in/crontab > https://onlinedevtools.in/devnews > https://diffviewer.onlinedevtools.in -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12549) Scan query/iterator on a replicated cache may get wrong results
[
https://issues.apache.org/jira/browse/IGNITE-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019552#comment-17019552
]
Sergey Kosarev edited comment on IGNITE-12549 at 1/20/20 3:20 PM:
--
[~Pavlukhin], there is a difference.
I've found the second problem in the method:
{{org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl#projection}}
there is stacktrace:
projection:628, IgniteCacheProxyImpl
(org.apache.ignite.internal.processors.cache)
query:809, IgniteCacheProxyImpl (org.apache.ignite.internal.processors.cache)
query:412, GatewayProtectedCacheProxy
(org.apache.ignite.internal.processors.cache)
it looks like IgniteCacheProxyImpl#projection somewhat duplicates logic of
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#nodes()
and both have the same error - lacks the check for moving partitions.
was (Author: macrergate):
[~Pavlukhin], there is a difference.
I've found the second problem in the method:
{{org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl#projection}}
there is stacktrace:
projection:628, IgniteCacheProxyImpl
(org.apache.ignite.internal.processors.cache)
query:809, IgniteCacheProxyImpl (org.apache.ignite.internal.processors.cache)
query:412, GatewayProtectedCacheProxy
(org.apache.ignite.internal.processors.cache)
it looks like {{IgniteCacheProxyImpl#projection somewhat duplicates logic of
}}org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#nodes()
and both have the same error - lacks the check for moving partitions.
> Scan query/iterator on a replicated cache may get wrong results
> ---
>
> Key: IGNITE-12549
> URL: https://issues.apache.org/jira/browse/IGNITE-12549
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.7.6
>Reporter: Sergey Kosarev
>Priority: Critical
> Fix For: 2.8
>
>
> Case 1
> 1. start server node 1
> 2. create and fill replicated cache with RebalanceMode.Async (as by default)
> 3. start servr node 2
> 4. immediately execute scan query on the replicated cache((or just iterate
> the cache)) on the node 2
> It can get empty or partial results. (if rebalance on node 2 is finished)
> Case 2
> 1. start server node 1
> 2. create and fill replicated cache with RebalanceMode.Async (as by default)
> 3. start client node 2
> 4. start server node 3
> 5. immediately execute scan query on the replicated cache((or just iterate
> the cache)) on the client node 2
> It can get empty or partial results. (if rebalance on node 2 is not finished
> and query is mapped on the node 2)
> It looks like problem in the
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#nodes()
> case REPLICATED:
> if (prj != null || part != null)
> return nodes(cctx, prj, part);
> if (cctx.affinityNode())
> return *Collections.singletonList(cctx.localNode())*;
> Collection affNodes = nodes(cctx, null, null);
> return affNodes.isEmpty() ? affNodes :
> *Collections.singletonList(F.rand(affNodes))*;
> case PARTITIONED:
> return nodes(cctx, prj, part);
> which is executed in
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#executeScanQuery.
> If executed on a just started node it obviously returns the local node
> disregarding was it rebalanced or not.
> If executed on a client it returns a random affinity node, so it also can be
> not yet rebalanced node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12549) Scan query/iterator on a replicated cache may get wrong results
[
https://issues.apache.org/jira/browse/IGNITE-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019552#comment-17019552
]
Sergey Kosarev commented on IGNITE-12549:
-
[~Pavlukhin], there is a difference.
I've found the second problem in the method:
{{org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl#projection}}
there is stacktrace:
projection:628, IgniteCacheProxyImpl
(org.apache.ignite.internal.processors.cache)
query:809, IgniteCacheProxyImpl (org.apache.ignite.internal.processors.cache)
query:412, GatewayProtectedCacheProxy
(org.apache.ignite.internal.processors.cache)
it looks like {{IgniteCacheProxyImpl#projection somewhat duplicates logic of
}}org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#nodes()
and both have the same error - lacks the check for moving partitions.
> Scan query/iterator on a replicated cache may get wrong results
> ---
>
> Key: IGNITE-12549
> URL: https://issues.apache.org/jira/browse/IGNITE-12549
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.7.6
>Reporter: Sergey Kosarev
>Priority: Critical
> Fix For: 2.8
>
>
> Case 1
> 1. start server node 1
> 2. create and fill replicated cache with RebalanceMode.Async (as by default)
> 3. start servr node 2
> 4. immediately execute scan query on the replicated cache((or just iterate
> the cache)) on the node 2
> It can get empty or partial results. (if rebalance on node 2 is finished)
> Case 2
> 1. start server node 1
> 2. create and fill replicated cache with RebalanceMode.Async (as by default)
> 3. start client node 2
> 4. start server node 3
> 5. immediately execute scan query on the replicated cache((or just iterate
> the cache)) on the client node 2
> It can get empty or partial results. (if rebalance on node 2 is not finished
> and query is mapped on the node 2)
> It looks like problem in the
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#nodes()
> case REPLICATED:
> if (prj != null || part != null)
> return nodes(cctx, prj, part);
> if (cctx.affinityNode())
> return *Collections.singletonList(cctx.localNode())*;
> Collection affNodes = nodes(cctx, null, null);
> return affNodes.isEmpty() ? affNodes :
> *Collections.singletonList(F.rand(affNodes))*;
> case PARTITIONED:
> return nodes(cctx, prj, part);
> which is executed in
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#executeScanQuery.
> If executed on a just started node it obviously returns the local node
> disregarding was it rebalanced or not.
> If executed on a client it returns a random affinity node, so it also can be
> not yet rebalanced node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12505) Improve logging of data regions on startup
[
https://issues.apache.org/jira/browse/IGNITE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Kasnacheev updated IGNITE-12505:
-
Ignite Flags: (was: Release Notes Required)
> Improve logging of data regions on startup
> --
>
> Key: IGNITE-12505
> URL: https://issues.apache.org/jira/browse/IGNITE-12505
> Project: Ignite
> Issue Type: Improvement
> Components: general
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Major
> Labels: usability
> Fix For: 2.9
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> First we have IgniteConfiguration printed (quiet=false):
> {quote}2019-07-24 02:33:33.918[INFO
> ][Thread-139][o.a.i.i.IgniteKernal%GridNodeName] IgniteConfiguration [...
> dfltDataRegConf=DataRegionConfiguration [name=mem_plc, maxSize=635655159808,
> initSize=268435456, swapPath=null, pageEvictionMode=DISABLED,
> evictionThreshold=0.9, emptyPagesPoolSize=100, metricsEnabled=true,
> metricsSubIntervalCount=5, metricsRateTimeInterval=1000,
> persistenceEnabled=true, checkpointPageBufSize=17179869184],
> storagePath=/ssd/data, checkpointFreq=3, lockWaitTime=1,
> checkpointThreads=4, checkpointWriteOrder=SEQUENTIAL, walHistSize=2147483647,
> walSegments=10, walSegmentSize=1073741824, walPath=/ssd/data/wal,
> walArchivePath=/sas/wal_archive, metricsEnabled=false, walMode=LOG_ONLY,
> walTlbSize=131072, walBuffSize=5242880, walFlushFreq=2000,
> walFsyncDelay=1000, walRecordIterBuffSize=67108864,
> alwaysWriteFullPages=false,
> fileIOFactory=org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory@3612c49a,
> metricsSubIntervalCnt=5, metricsRateTimeInterval=6,
> walAutoArchiveAfterInactivity=-1, writeThrottlingEnabled=false,
> walCompactionEnabled=true, walCompactionLevel=1], ...]{quote}
> Then we have all configured Data Regions printed per IGNITE-8803 (quiet=true):
> {quote} [11:30:36] Data Regions Configured:
> [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
> persistence=false, lazyMemoryAllocation=true]
> [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
> persistence=false, lazyMemoryAllocation=true]{quote}
> Then we print number of Data Regions that were initialized as per
> IGNITE-7196, but not regions themselves (quiet=false):
> Configured data regions initialized successfully [total=4]
> I propose to keep the first one (IgniteConfiguration), remove the second one
> (Data Regions Configured), and promote the last one to quiet mode while also
> outputting the regions themselves like this:
> {quote} [11:30:36] Data Regions Initialized Successfully: 4
> [11:30:36] ^-- plcWithMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
> persistence=true, lazyMemoryAllocation=true]
> [11:30:36] ^-- plcNoMetrics [initSize=256,0 MiB, maxSize=6,3 GiB,
> persistence=true, lazyMemoryAllocation=true]
> [11:30:36] ^-- sysMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB,
> persistence=true, lazyMemoryAllocation=false]
> [11:30:36] ^-- volatileMemPlc [initSize=40,0 MiB, maxSize=100,0 MiB,
> persistence=false, lazyMemoryAllocation=true]{quote}
> Also clearly mark default region and system regions as such.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12530) Pages list caching can cause IgniteOOME when checkpoint is triggered by "too many dirty pages" reason
[
https://issues.apache.org/jira/browse/IGNITE-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019491#comment-17019491
]
Ivan Rakov commented on IGNITE-12530:
-
[~alex_pl] Indeed, I didn't catch it. Thanks.
Looks good to me.
> Pages list caching can cause IgniteOOME when checkpoint is triggered by "too
> many dirty pages" reason
> -
>
> Key: IGNITE-12530
> URL: https://issues.apache.org/jira/browse/IGNITE-12530
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When a checkpoint is triggered, we need some amount of page memory to store
> pages list on-heap cache.
> If data region is too small, a checkpoint is triggered by "too many dirty
> pages" reason and pages list cache is rather big, we can get
> IgniteOutOfMemoryException.
> Reproducer:
> {code:java}
> @Override protected IgniteConfiguration getConfiguration(String name) throws
> Exception {
> IgniteConfiguration cfg = super.getConfiguration(name);
> cfg.setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new DataRegionConfiguration()
> .setPersistenceEnabled(true)
> .setMaxSize(50 * 1024 * 1024)
> ));
> return cfg;
> }
> @Test
> public void testUpdatesNotFittingIntoMemoryRegion() throws Exception {
> IgniteEx ignite = startGrid(0);
> ignite.cluster().active(true);
> ignite.getOrCreateCache(DEFAULT_CACHE_NAME);
> try (IgniteDataStreamer streamer =
> ignite.dataStreamer(DEFAULT_CACHE_NAME)) {
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, new byte[i % 2048]);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12556) Online Dev Tools
[ https://issues.apache.org/jira/browse/IGNITE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019483#comment-17019483 ] Stephen commented on IGNITE-12556: -- https://onlinedevtools.in check this link useful web developer tools link https://onlinedevtools.in/lodash_underscore_alternative https://onlinedevtools.in/online/sqlformatter https://onlinedevtools.in/online/npmpackageanalyzer https://onlinedevtools.in/curl https://onlinedevtools.in/crontab https://onlinedevtools.in/devnews https://diffviewer.onlinedevtools.in > Online Dev Tools > > > Key: IGNITE-12556 > URL: https://issues.apache.org/jira/browse/IGNITE-12556 > Project: Ignite > Issue Type: Bug > Environment: https://onlinedevtools.in > check this link > useful web developer tools link > https://onlinedevtools.in/lodash_underscore_alternative > https://onlinedevtools.in/online/sqlformatter > https://onlinedevtools.in/online/npmpackageanalyzer > https://onlinedevtools.in/curl > https://onlinedevtools.in/crontab > https://onlinedevtools.in/devnews > https://diffviewer.onlinedevtools.in >Reporter: Stephen >Priority: Major > > https://onlinedevtools.in > check this link > useful web developer tools link > https://onlinedevtools.in/lodash_underscore_alternative > https://onlinedevtools.in/online/sqlformatter > https://onlinedevtools.in/online/npmpackageanalyzer > https://onlinedevtools.in/curl > https://onlinedevtools.in/crontab > https://onlinedevtools.in/devnews > https://diffviewer.onlinedevtools.in -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12556) Online Dev Tools
Stephen created IGNITE-12556: Summary: Online Dev Tools Key: IGNITE-12556 URL: https://issues.apache.org/jira/browse/IGNITE-12556 Project: Ignite Issue Type: Bug Environment: https://onlinedevtools.in check this link useful web developer tools link https://onlinedevtools.in/lodash_underscore_alternative https://onlinedevtools.in/online/sqlformatter https://onlinedevtools.in/online/npmpackageanalyzer https://onlinedevtools.in/curl https://onlinedevtools.in/crontab https://onlinedevtools.in/devnews https://diffviewer.onlinedevtools.in Reporter: Stephen https://onlinedevtools.in check this link useful web developer tools link https://onlinedevtools.in/lodash_underscore_alternative https://onlinedevtools.in/online/sqlformatter https://onlinedevtools.in/online/npmpackageanalyzer https://onlinedevtools.in/curl https://onlinedevtools.in/crontab https://onlinedevtools.in/devnews https://diffviewer.onlinedevtools.in -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12556) Online Dev Tools
[ https://issues.apache.org/jira/browse/IGNITE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019481#comment-17019481 ] Stephen commented on IGNITE-12556: -- https://onlinedevtools.in check this link useful web developer tools link https://onlinedevtools.in/lodash_underscore_alternative https://onlinedevtools.in/online/sqlformatter https://onlinedevtools.in/online/npmpackageanalyzer https://onlinedevtools.in/curl https://onlinedevtools.in/crontab https://onlinedevtools.in/devnews https://diffviewer.onlinedevtools.in > Online Dev Tools > > > Key: IGNITE-12556 > URL: https://issues.apache.org/jira/browse/IGNITE-12556 > Project: Ignite > Issue Type: Bug > Environment: https://onlinedevtools.in > check this link > useful web developer tools link > https://onlinedevtools.in/lodash_underscore_alternative > https://onlinedevtools.in/online/sqlformatter > https://onlinedevtools.in/online/npmpackageanalyzer > https://onlinedevtools.in/curl > https://onlinedevtools.in/crontab > https://onlinedevtools.in/devnews > https://diffviewer.onlinedevtools.in >Reporter: Stephen >Priority: Major > > https://onlinedevtools.in > check this link > useful web developer tools link > https://onlinedevtools.in/lodash_underscore_alternative > https://onlinedevtools.in/online/sqlformatter > https://onlinedevtools.in/online/npmpackageanalyzer > https://onlinedevtools.in/curl > https://onlinedevtools.in/crontab > https://onlinedevtools.in/devnews > https://diffviewer.onlinedevtools.in -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12530) Pages list caching can cause IgniteOOME when checkpoint is triggered by "too many dirty pages" reason
[
https://issues.apache.org/jira/browse/IGNITE-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019474#comment-17019474
]
Ignite TC Bot commented on IGNITE-12530:
{panel:title=Branch: [pull/7245/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4938592buildTypeId=IgniteTests24Java8_RunAll]
> Pages list caching can cause IgniteOOME when checkpoint is triggered by "too
> many dirty pages" reason
> -
>
> Key: IGNITE-12530
> URL: https://issues.apache.org/jira/browse/IGNITE-12530
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When a checkpoint is triggered, we need some amount of page memory to store
> pages list on-heap cache.
> If data region is too small, a checkpoint is triggered by "too many dirty
> pages" reason and pages list cache is rather big, we can get
> IgniteOutOfMemoryException.
> Reproducer:
> {code:java}
> @Override protected IgniteConfiguration getConfiguration(String name) throws
> Exception {
> IgniteConfiguration cfg = super.getConfiguration(name);
> cfg.setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new DataRegionConfiguration()
> .setPersistenceEnabled(true)
> .setMaxSize(50 * 1024 * 1024)
> ));
> return cfg;
> }
> @Test
> public void testUpdatesNotFittingIntoMemoryRegion() throws Exception {
> IgniteEx ignite = startGrid(0);
> ignite.cluster().active(true);
> ignite.getOrCreateCache(DEFAULT_CACHE_NAME);
> try (IgniteDataStreamer streamer =
> ignite.dataStreamer(DEFAULT_CACHE_NAME)) {
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, new byte[i % 2048]);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12554) Ignite instance couldn't be restarted if it's parent thread group has been called destroy once
[
https://issues.apache.org/jira/browse/IGNITE-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019459#comment-17019459
]
Ivan Pavlukhin commented on IGNITE-12554:
-
[~hafaculty], currently I can suggest only a workaround. Is it possible for you
to enforce {{org.apache.ignite.thread.IgniteThread#DFLT_GRP}} initialization in
a system (root) ThreadGroup?
Unfortunately, cannot answer from scratch what would be the proper fix. Perhaps
special DFLT_GRP is not really needed..
> Ignite instance couldn't be restarted if it's parent thread group has been
> called destroy once
> --
>
> Key: IGNITE-12554
> URL: https://issues.apache.org/jira/browse/IGNITE-12554
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.7.6
>Reporter: ha faculty
>Priority: Major
> Attachments: NodeRestartTesting.java
>
>
> I am using an inhouse platform which manage the life cycle of ignite instance
> by calling ignition.start() and ignition.stop() from a web application.
> it could be started without issue for the first time. however, if it stop the
> ignite by calling ignition.stop() and followed by destroying the parent
> thread group which starting it.
> Calling ignition.start() in the 2nd time will throw the following exception
> Exception in thread "kickOff" java.lang.IllegalThreadStateExceptionException
> in thread "kickOff" java.lang.IllegalThreadStateException at
> java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at
> java.lang.Thread.init(Thread.java:405) at
> java.lang.Thread.init(Thread.java:349) at
> java.lang.Thread.(Thread.java:599) at
> org.apache.ignite.thread.IgniteThread.(IgniteThread.java:96) at
> org.apache.ignite.internal.util.StripedExecutor$Stripe.start(StripedExecutor.java:474)
> at
> org.apache.ignite.internal.util.StripedExecutor.(StripedExecutor.java:121)
> at
> org.apache.ignite.internal.util.StripedExecutor.(StripedExecutor.java:80)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1842)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:589) at
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:565) at
> org.apache.ignite.Ignition.start(Ignition.java:305) at
> com.faculty.poc.ignite.app.NodeRestartTesting.run(NodeRestartTesting.java:60)
> at java.lang.Thread.run(Thread.java:748)
> Please find my demo code for this issue.
> Walk through into Ignite code, it should be caused by a static thread group
> created in IgniteThread class. (private static final ThreadGroup DFLT_GRP =
> new ThreadGroup("ignite")
> Destroyed the parent thread group which start the ignite instance will result
> in causing the DFLT_GRP go into destroyed state. Therefore, when Ignite was
> start ifor the 2nd time, this static thread group has been destroyed and no
> longer able to run the ignite.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12530) Pages list caching can cause IgniteOOME when checkpoint is triggered by "too many dirty pages" reason
[
https://issues.apache.org/jira/browse/IGNITE-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019446#comment-17019446
]
Stanilovsky Evgeny commented on IGNITE-12530:
-
!screenshot-2.png! new fix and test seem ok.
> Pages list caching can cause IgniteOOME when checkpoint is triggered by "too
> many dirty pages" reason
> -
>
> Key: IGNITE-12530
> URL: https://issues.apache.org/jira/browse/IGNITE-12530
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When a checkpoint is triggered, we need some amount of page memory to store
> pages list on-heap cache.
> If data region is too small, a checkpoint is triggered by "too many dirty
> pages" reason and pages list cache is rather big, we can get
> IgniteOutOfMemoryException.
> Reproducer:
> {code:java}
> @Override protected IgniteConfiguration getConfiguration(String name) throws
> Exception {
> IgniteConfiguration cfg = super.getConfiguration(name);
> cfg.setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new DataRegionConfiguration()
> .setPersistenceEnabled(true)
> .setMaxSize(50 * 1024 * 1024)
> ));
> return cfg;
> }
> @Test
> public void testUpdatesNotFittingIntoMemoryRegion() throws Exception {
> IgniteEx ignite = startGrid(0);
> ignite.cluster().active(true);
> ignite.getOrCreateCache(DEFAULT_CACHE_NAME);
> try (IgniteDataStreamer streamer =
> ignite.dataStreamer(DEFAULT_CACHE_NAME)) {
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, new byte[i % 2048]);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12530) Pages list caching can cause IgniteOOME when checkpoint is triggered by "too many dirty pages" reason
[
https://issues.apache.org/jira/browse/IGNITE-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stanilovsky Evgeny updated IGNITE-12530:
Attachment: screenshot-2.png
> Pages list caching can cause IgniteOOME when checkpoint is triggered by "too
> many dirty pages" reason
> -
>
> Key: IGNITE-12530
> URL: https://issues.apache.org/jira/browse/IGNITE-12530
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When a checkpoint is triggered, we need some amount of page memory to store
> pages list on-heap cache.
> If data region is too small, a checkpoint is triggered by "too many dirty
> pages" reason and pages list cache is rather big, we can get
> IgniteOutOfMemoryException.
> Reproducer:
> {code:java}
> @Override protected IgniteConfiguration getConfiguration(String name) throws
> Exception {
> IgniteConfiguration cfg = super.getConfiguration(name);
> cfg.setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new DataRegionConfiguration()
> .setPersistenceEnabled(true)
> .setMaxSize(50 * 1024 * 1024)
> ));
> return cfg;
> }
> @Test
> public void testUpdatesNotFittingIntoMemoryRegion() throws Exception {
> IgniteEx ignite = startGrid(0);
> ignite.cluster().active(true);
> ignite.getOrCreateCache(DEFAULT_CACHE_NAME);
> try (IgniteDataStreamer streamer =
> ignite.dataStreamer(DEFAULT_CACHE_NAME)) {
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, new byte[i % 2048]);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12549) Scan query/iterator on a replicated cache may get wrong results
[ https://issues.apache.org/jira/browse/IGNITE-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019443#comment-17019443 ] Ivan Pavlukhin commented on IGNITE-12549: - [~macrergate], seems very common to IGNITE-12482. Can it be fixed similarly? > Scan query/iterator on a replicated cache may get wrong results > --- > > Key: IGNITE-12549 > URL: https://issues.apache.org/jira/browse/IGNITE-12549 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.7.6 >Reporter: Sergey Kosarev >Priority: Critical > Fix For: 2.8 > > > Case 1 > 1. start server node 1 > 2. create and fill replicated cache with RebalanceMode.Async (as by default) > 3. start servr node 2 > 4. immediately execute scan query on the replicated cache((or just iterate > the cache)) on the node 2 > It can get empty or partial results. (if rebalance on node 2 is finished) > Case 2 > 1. start server node 1 > 2. create and fill replicated cache with RebalanceMode.Async (as by default) > 3. start client node 2 > 4. start server node 3 > 5. immediately execute scan query on the replicated cache((or just iterate > the cache)) on the client node 2 > It can get empty or partial results. (if rebalance on node 2 is not finished > and query is mapped on the node 2) > It looks like problem in the > org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#nodes() > case REPLICATED: > if (prj != null || part != null) > return nodes(cctx, prj, part); > if (cctx.affinityNode()) > return *Collections.singletonList(cctx.localNode())*; > Collection affNodes = nodes(cctx, null, null); > return affNodes.isEmpty() ? affNodes : > *Collections.singletonList(F.rand(affNodes))*; > case PARTITIONED: > return nodes(cctx, prj, part); > which is executed in > org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter#executeScanQuery. > If executed on a just started node it obviously returns the local node > disregarding was it rebalanced or not. > If executed on a client it returns a random affinity node, so it also can be > not yet rebalanced node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12543) When put List>, the data was increased much larger.
[
https://issues.apache.org/jira/browse/IGNITE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019435#comment-17019435
]
Ivan Pavlukhin commented on IGNITE-12543:
-
[~redcomet], thank you for reporting this issue. In fact there are multiple
limitations in BinaryObject serialization protocol. Initially the protocol was
designed to be universal and able to serialize objects of any complexity
(nested objects, circular references). The downside is that the protocol is not
efficient sometimes.
Will a workaround with wrapping a list by a custom wrapper class work for you?
{code}
public class Wrapper {
private T object;
...
}
{code}
> When put List>, the data was increased much larger.
>
>
> Key: IGNITE-12543
> URL: https://issues.apache.org/jira/browse/IGNITE-12543
> Project: Ignite
> Issue Type: Bug
> Components: thin client
>Affects Versions: 2.6
>Reporter: LEE PYUNG BEOM
>Priority: Major
>
> I use Ignite 2.6 version of Java Thin Client.
>
> When I put data in the form List>,
> The size of the original 200KB data was increased to 50MB when inquired by
> Ignite servers.
> On the Heap Dump, the list element was repeatedly accumulated, increasing the
> data size.
>
> When I checked org.apacheignite.internal.binary.BinaryWriterExImpl.java
> doWriteBinaryObject() method,
> {code:java}
> // org.apacheignite.internal.binary.BinaryWriterExImpl.java
> public void doWriteBinaryObject(@Nullable BinaryObjectImpl po) {
> if (po == null)
> out.writeByte(GridBinaryMarshaller.NULL);
> else {
> byte[] poArr = po.array();
> out.unsafeEnsure(1 + 4 + poArr.length +4);
> out.unsafeWriteByte(GridBinaryMarshaller.BINARY_OBJ);
> out.unsafeWriteInt(poArr.length);
> out.writeByteArray(poArr);
> out.unsafeWriteInt(po.start());
> }
> }
> {code}
>
> The current Ignite implementation for storing data in the form
> List> is:
> In the Marshalling stage, for example, data the size of List(5
> members) is:
> As many as 10*5 of the list's elements are duplicated.
> If the above data contains five objects of 200KB size, ten by one,
> 50 iterations are stored and 200K*10**5 = 100MB of data is used for cache and
> transfer.
> As a result of this increase in data size, it is confirmed that the failure
> of OOM, GC, etc. is caused by occupying Heap memory.
> Unnecessarily redundant data is used for cache storage and network transport.
> When looking up cache data, only some of the data at the top is read based on
> file location information from the entire data, so that normal data is
> retrieved.
> The way we're implemented today is safe from basic behavior, but we're
> wasting memory and network unnecessarily using inefficient algorithms
> This can have very serious consequences. Please check.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12096) Used memory metric does not contract when cache is emptied - FreeList REUSE_BUCKET is not accounted for
[
https://issues.apache.org/jira/browse/IGNITE-12096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019423#comment-17019423
]
Ivan Pavlukhin commented on IGNITE-12096:
-
[~coliin], could you please create a pull request to include your test? Some
points to take into account:
# It is usually handy to create a subclass of {{GridCommonAbstractTest}} when
adding a new test class.
# It is required to run tests on TeamCity before merging code patches.
# More details about contribution process
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
Do not hesitate to ask in case of any questions.
> Used memory metric does not contract when cache is emptied - FreeList
> REUSE_BUCKET is not accounted for
> ---
>
> Key: IGNITE-12096
> URL: https://issues.apache.org/jira/browse/IGNITE-12096
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.7
>Reporter: Colin Cassidy
>Priority: Major
> Fix For: 2.8
>
>
> When using the Ignite metrics API to measure available memory, the usage
> figures appear to be accurate while memory is being consumed - but when
> memory is freed the metrics do not drop. They appear to report that memory
> has not been freed up, even though it has.
> A reliable estimate of memory consumption is very important for solutions
> that don't use native persistence - as this is the only controllable way of
> avoiding a critical OOM condition.
> Reproducer below. This affects Ignite 2.7+.
> {{public class MemoryTest2 { }}
> {{ private static final String CACHE_NAME = "cache"; }}
> {{ private static final String DEFAULT_MEMORY_REGION = "Default_Region";
> }}
> {{ private static final long MEM_SIZE = 100L * 1024 * 1024; }}
> {{ @Test }}
> {{ public void testOOM() throws InterruptedException { }}
> {{ try (Ignite ignite = startIgnite("IgniteMemoryMonitorTest1")) { }}
> {{ fillDataRegion(ignite); }}
> {{ CacheConfiguration cfg = new }}
> {{CacheConfiguration<>(CACHE_NAME); }}
> {{ cfg.setStatisticsEnabled(true); }}
> {{ IgniteCache cache = }}
> {{ignite.getOrCreateCache(cfg); }}
> {{ // Clear all entries from the cache to free up memory }}
> {{ memUsed(ignite); }}
> {{ cache.clear(); }}
> {{ cache.removeAll(); }}
> {{ cache.put("Key", "Value"); }}
> {{ memUsed(ignite); }}
> {{ cache.destroy(); }}
> {{ Thread.sleep(5000); }}
> {{ // Should now report close to 0% but reports 59% still }}
> {{ memUsed(ignite); }}
> {{ } }}
> {{ } }}
> {{ }}
> {{ private Ignite startIgnite(String instanceName) { }}
> {{ IgniteConfiguration cfg = new IgniteConfiguration(); }}
> {{ cfg.setIgniteInstanceName(instanceName); }}
> {{ cfg.setDataStorageConfiguration(createDataStorageConfiguration());
> }}
> {{ cfg.setFailureHandler(new NoOpFailureHandler()); }}
> {{ return Ignition.start(cfg); }}
> {{ } }}
> {{ private DataStorageConfiguration createDataStorageConfiguration() { }}
> {{ return new DataStorageConfiguration() }}
> {{ .setDefaultDataRegionConfiguration( }}
> {{ new DataRegionConfiguration() }}
> {{ .setName(DEFAULT_MEMORY_REGION) }}
> {{ .setInitialSize(MEM_SIZE) }}
> {{ .setMaxSize(MEM_SIZE) }}
> {{ .setMetricsEnabled(true)); }}
> {{ } }}
> {{ private void fillDataRegion(Ignite ignite) { }}
> {{ byte[] megabyte = new byte[1024 * 1024]; }}
> {{ IgniteCache cache = }}
> {{ ignite.getOrCreateCache(CACHE_NAME); }}
> {{ for (int i = 0; i < 50; i++) { }}
> {{ cache.put(i, megabyte); }}
> {{ memUsed(ignite); }}
> {{ } }}
> {{ } }}
> {{ private void memUsed(Ignite ignite) { }}
> {{ DataRegionConfiguration defaultDataRegionCfg = }}
> {{ignite.configuration() }}
> {{ .getDataStorageConfiguration() }}
> {{ .getDefaultDataRegionConfiguration(); }}
> {{ String regionName = defaultDataRegionCfg.getName(); }}
> {{ DataRegionMetrics metrics = ignite.dataRegionMetrics(regionName);
> }}
> {{ float usedMem = metrics.getPagesFillFactor() * }}
> {{metrics.getTotalAllocatedPages() * metrics.getPageSize(); }}
> {{ float pctUsed = 100 * usedMem / defaultDataRegionCfg.getMaxSize();
> }}
> {{ System.out.println("Memory used: " + pctUsed + "%"); }}
> {{ } }}
> {{} }}
--
This message was sent by
[jira] [Commented] (IGNITE-12546) Prevent partitions owned by other nodes switch their state to MOVING due to counter difference on node join.
[ https://issues.apache.org/jira/browse/IGNITE-12546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019407#comment-17019407 ] Ivan Rakov commented on IGNITE-12546: - [~ascherbakov] Looks good. > Prevent partitions owned by other nodes switch their state to MOVING due to > counter difference on node join. > > > Key: IGNITE-12546 > URL: https://issues.apache.org/jira/browse/IGNITE-12546 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.7.6 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > When node joins it's expected that MOVING partitions can only belong to > joining node. > But if somehow counters are desynced other nodes can switch state of owning > partitions to MOVING too, causing spurious rebalancing and assertions on > mapping to moving primary. > Possible solution: exclude other nodes partitions from switching state to > MOVING on node join. > This only affects persistent groups. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12548) Possible tx desync during recovery on near node left.
[ https://issues.apache.org/jira/browse/IGNITE-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019408#comment-17019408 ] Ivan Rakov commented on IGNITE-12548: - [~ascherbakov] Looks good. > Possible tx desync during recovery on near node left. > - > > Key: IGNITE-12548 > URL: https://issues.apache.org/jira/browse/IGNITE-12548 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7.6 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Blocker > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > The problem appears if a transaction is starting to rollback in PREPARED > state for some reason and concurrently near node is left triggering tx > recovery protocol. > Consider having two enlisted keys from different partitions mapped to > different nodes N1 and N2. > Due to race N1 local tx can be rolled back while N2 local tx is committed > breaking tx atomicity guarantee. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12555) .NET: Thin Client: deserializing DateTime fields causes BinaryTypeGet request for every value
[
https://issues.apache.org/jira/browse/IGNITE-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-12555:
Description:
Actual: The following code causes 10 BinaryProcessorClient.GetBinaryType calls
(2 fields, 5 Foo instances). Every call is a server request.
Expected: 0 calls. Binary metadata should be cached after PutAll call.
{code}
public class CacheDateTimeMetaTest : ClientTestBase
{
[Test]
public void TestDateTimeMeta()
{
var data = Enumerable.Range(1, 5)
.Select(x => new Foo
{
Id = x,
StartDate = DateTime.Now.AddHours(x),
EndDate = DateTime.Now.AddDays(x)
});
var cache = Client.GetOrCreateCache("foo");
cache.PutAll(data.Select(x => new KeyValuePair(x.Id, x)));
var res = cache.Query(new ScanQuery()).GetAll();
Assert.AreEqual(cache.GetSize(), res.Count);
}
public class Foo
{
public int Id { get; set; }
public DateTime? StartDate { get; set; }
public DateTime? EndDate { get; set; }
}
}
{code}
This causes huge performance issues.
*Workaround*
* Force Timestamp format for all DateTime values
User list discussion:
http://apache-ignite-users.70518.x6.nabble.com/Getting-all-data-from-cache-via-scan-query-is-taking-lot-of-time-td30949.html
was:
Actual: The following code causes 10 BinaryProcessorClient.GetBinaryType calls
(2 fields, 5 Foo instances). Every call is a server request.
Expected: 0 calls. Binary metadata should be cached after PutAll call.
{code}
public class CacheDateTimeMetaTest : ClientTestBase
{
[Test]
public void TestDateTimeMeta()
{
var data = Enumerable.Range(1, 5)
.Select(x => new Foo
{
Id = x,
StartDate = DateTime.Now.AddHours(x),
EndDate = DateTime.Now.AddDays(x)
});
var cache = Client.GetOrCreateCache("foo");
cache.PutAll(data.Select(x => new KeyValuePair(x.Id, x)));
var res = cache.Query(new ScanQuery()).GetAll();
Assert.AreEqual(cache.GetSize(), res.Count);
}
public class Foo
{
public int Id { get; set; }
public DateTime? StartDate { get; set; }
public DateTime? EndDate { get; set; }
}
}
{code}
User list discussion:
http://apache-ignite-users.70518.x6.nabble.com/Getting-all-data-from-cache-via-scan-query-is-taking-lot-of-time-td30949.html
> .NET: Thin Client: deserializing DateTime fields causes BinaryTypeGet request
> for every value
> -
>
> Key: IGNITE-12555
> URL: https://issues.apache.org/jira/browse/IGNITE-12555
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Affects Versions: 2.4, 2.5, 2.6, 2.7, 2.8, 2.7.5, 2.7.6
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Blocker
> Labels: .NET
> Fix For: 2.8
>
>
> Actual: The following code causes 10 BinaryProcessorClient.GetBinaryType
> calls (2 fields, 5 Foo instances). Every call is a server request.
> Expected: 0 calls. Binary metadata should be cached after PutAll call.
> {code}
> public class CacheDateTimeMetaTest : ClientTestBase
> {
> [Test]
> public void TestDateTimeMeta()
> {
> var data = Enumerable.Range(1, 5)
> .Select(x => new Foo
> {
> Id = x,
> StartDate = DateTime.Now.AddHours(x),
> EndDate = DateTime.Now.AddDays(x)
> });
> var cache = Client.GetOrCreateCache("foo");
> cache.PutAll(data.Select(x => new KeyValuePair(x.Id,
> x)));
> var res = cache.Query(new ScanQuery()).GetAll();
> Assert.AreEqual(cache.GetSize(), res.Count);
> }
> public class Foo
> {
> public int Id { get; set; }
> public DateTime? StartDate { get; set; }
> public DateTime? EndDate { get; set; }
> }
> }
> {code}
> This causes huge performance issues.
> *Workaround*
> * Force Timestamp format for all DateTime values
> User list discussion:
> http://apache-ignite-users.70518.x6.nabble.com/Getting-all-data-from-cache-via-scan-query-is-taking-lot-of-time-td30949.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12555) .NET: Thin Client: deserializing DateTime fields causes BinaryTypeGet request for every value
Pavel Tupitsyn created IGNITE-12555:
---
Summary: .NET: Thin Client: deserializing DateTime fields causes
BinaryTypeGet request for every value
Key: IGNITE-12555
URL: https://issues.apache.org/jira/browse/IGNITE-12555
Project: Ignite
Issue Type: Bug
Components: platforms
Affects Versions: 2.7.6, 2.7.5, 2.7, 2.6, 2.5, 2.4, 2.8
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
Fix For: 2.8
Actual: The following code causes 10 BinaryProcessorClient.GetBinaryType calls
(2 fields, 5 Foo instances). Every call is a server request.
Expected: 0 calls. Binary metadata should be cached after PutAll call.
{code}
public class CacheDateTimeMetaTest : ClientTestBase
{
[Test]
public void TestDateTimeMeta()
{
var data = Enumerable.Range(1, 5)
.Select(x => new Foo
{
Id = x,
StartDate = DateTime.Now.AddHours(x),
EndDate = DateTime.Now.AddDays(x)
});
var cache = Client.GetOrCreateCache("foo");
cache.PutAll(data.Select(x => new KeyValuePair(x.Id, x)));
var res = cache.Query(new ScanQuery()).GetAll();
Assert.AreEqual(cache.GetSize(), res.Count);
}
public class Foo
{
public int Id { get; set; }
public DateTime? StartDate { get; set; }
public DateTime? EndDate { get; set; }
}
}
{code}
User list discussion:
http://apache-ignite-users.70518.x6.nabble.com/Getting-all-data-from-cache-via-scan-query-is-taking-lot-of-time-td30949.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12474) Fix boost compatibility for cpp thin-client-test.
[
https://issues.apache.org/jira/browse/IGNITE-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019382#comment-17019382
]
Ignite TC Bot commented on IGNITE-12474:
{panel:title=Branch: [pull/7242/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4938147buildTypeId=IgniteTests24Java8_RunAll]
> Fix boost compatibility for cpp thin-client-test.
> -
>
> Key: IGNITE-12474
> URL: https://issues.apache.org/jira/browse/IGNITE-12474
> Project: Ignite
> Issue Type: Improvement
> Components: thin client
>Affects Versions: 2.7.6
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Current teamcity_boost implementation leads to errors like:
> "src/teamcity/teamcity_boost.cpp:22:10: fatal error:
> boost/test/unit_test_suite_impl.hpp: No such file or catalog.
> #include "
> dev list discussion [1]
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/Can-t-run-CPP-tests-locally-on-linux-machine-td29597.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12463) Inconsistancy of checkpoint progress future with its state
[ https://issues.apache.org/jira/browse/IGNITE-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019380#comment-17019380 ] Alexey Goncharuk commented on IGNITE-12463: --- This is a good change, thanks for the contribution! Merged to master. > Inconsistancy of checkpoint progress future with its state > --- > > Key: IGNITE-12463 > URL: https://issues.apache.org/jira/browse/IGNITE-12463 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > It needs to reorganize checkpoint futures(start, finish) so they should be > matched to states. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12463) Inconsistancy of checkpoint progress future with its state
[ https://issues.apache.org/jira/browse/IGNITE-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019376#comment-17019376 ] Alexey Goncharuk commented on IGNITE-12463: --- No need for release notes as it is an internal API change. > Inconsistancy of checkpoint progress future with its state > --- > > Key: IGNITE-12463 > URL: https://issues.apache.org/jira/browse/IGNITE-12463 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > It needs to reorganize checkpoint futures(start, finish) so they should be > matched to states. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12463) Inconsistancy of checkpoint progress future with its state
[ https://issues.apache.org/jira/browse/IGNITE-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-12463: -- Ignite Flags: (was: Docs Required,Release Notes Required) > Inconsistancy of checkpoint progress future with its state > --- > > Key: IGNITE-12463 > URL: https://issues.apache.org/jira/browse/IGNITE-12463 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > It needs to reorganize checkpoint futures(start, finish) so they should be > matched to states. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12463) Inconsistancy of checkpoint progress future with its state
[ https://issues.apache.org/jira/browse/IGNITE-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-12463: -- Fix Version/s: 2.9 > Inconsistancy of checkpoint progress future with its state > --- > > Key: IGNITE-12463 > URL: https://issues.apache.org/jira/browse/IGNITE-12463 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > It needs to reorganize checkpoint futures(start, finish) so they should be > matched to states. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12227) Default auto-adjust baseline enabled flag calculated incorrectly in some cases
[ https://issues.apache.org/jira/browse/IGNITE-12227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019374#comment-17019374 ] Alexey Goncharuk commented on IGNITE-12227: --- Looks good to me as well, merged to master. [~mmuzaf], can you cherry-pick the commit to ignite-2.8? > Default auto-adjust baseline enabled flag calculated incorrectly in some cases > -- > > Key: IGNITE-12227 > URL: https://issues.apache.org/jira/browse/IGNITE-12227 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Blocker > Fix For: 2.8 > > Time Spent: 20m > Remaining Estimate: 0h > > baselineAutoAdjustEnabled can be been different on different nodes because of > the calculation of default value happening locally on each node and including > only local configuration. It issue can happen by the following reasons: > * If IGNITE_BASELINE_AUTO_ADJUST_ENABLED flag set to a different value on > different nodes it leads to cluster hanging due to baseline calculation > finishing with the unpredictable state on each node. > * if cluster in mixed mode(included in-memory and persistent nodes) sometimes > flag is set to a different value due to calculation doesn't consider remote > nodes configuration. > Possible solution(both points required): > * Get rid of IGNITE_BASELINE_AUTO_ADJUST_ENABLED and replace it by the > explicit call of IgniteCluster#baselineAutoAdjustEnabled where it > required(test only). > * Calculating default value on the first started node as early as > possible(instead of activation) and this value always should be set to > distributed metastorage(unlike it happening now). It means that instead of > awaiting activation, the default value would be calculated by the first > started node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12463) Inconsistancy of checkpoint progress future with its state
[ https://issues.apache.org/jira/browse/IGNITE-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019355#comment-17019355 ] Sergey Chugunov commented on IGNITE-12463: -- [~akalashnikov], Change looks good to me, lets merge it to master branch. Thank you for contribution! > Inconsistancy of checkpoint progress future with its state > --- > > Key: IGNITE-12463 > URL: https://issues.apache.org/jira/browse/IGNITE-12463 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > It needs to reorganize checkpoint futures(start, finish) so they should be > matched to states. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12227) Default auto-adjust baseline enabled flag calculated incorrectly in some cases
[ https://issues.apache.org/jira/browse/IGNITE-12227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019353#comment-17019353 ] Ivan Bessonov commented on IGNITE-12227: [~akalashnikov] looks good, let's merge it. I hope that there were no new tests with that property in last few days. > Default auto-adjust baseline enabled flag calculated incorrectly in some cases > -- > > Key: IGNITE-12227 > URL: https://issues.apache.org/jira/browse/IGNITE-12227 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Blocker > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > baselineAutoAdjustEnabled can be been different on different nodes because of > the calculation of default value happening locally on each node and including > only local configuration. It issue can happen by the following reasons: > * If IGNITE_BASELINE_AUTO_ADJUST_ENABLED flag set to a different value on > different nodes it leads to cluster hanging due to baseline calculation > finishing with the unpredictable state on each node. > * if cluster in mixed mode(included in-memory and persistent nodes) sometimes > flag is set to a different value due to calculation doesn't consider remote > nodes configuration. > Possible solution(both points required): > * Get rid of IGNITE_BASELINE_AUTO_ADJUST_ENABLED and replace it by the > explicit call of IgniteCluster#baselineAutoAdjustEnabled where it > required(test only). > * Calculating default value on the first started node as early as > possible(instead of activation) and this value always should be set to > distributed metastorage(unlike it happening now). It means that instead of > awaiting activation, the default value would be calculated by the first > started node. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12456) Cluster Data Store grid gets Corrupted for Load test
[
https://issues.apache.org/jira/browse/IGNITE-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019350#comment-17019350
]
Alexey Goncharuk commented on IGNITE-12456:
---
[~ravimsc] from the logs I see that the {{sys-stripe-pool}} is blocked by some
task, but this information is not present in ticket. When an Ignite node
detects a blocked system thread, it will print out a thread dump so we can
diagnose an issue. Can you attach a full thread dump to this ticket? We need a
thread dump from the server nodes.
> Cluster Data Store grid gets Corrupted for Load test
>
>
> Key: IGNITE-12456
> URL: https://issues.apache.org/jira/browse/IGNITE-12456
> Project: Ignite
> Issue Type: Bug
> Components: general
>Affects Versions: 2.7
>Reporter: Ravi Kumar Powli
>Priority: Blocker
> Fix For: 2.8
>
> Attachments: default-config.xml
>
>
> We have Apache Ignite 3 node cluster setup with Amazon S3 Based Discovery.we
> are into AWS cloud environment with Microservice model with 8
> microservices.we are using Ignite for session data store.While performing
> load test we are facing data grid issues if the number of clients reached 40
> , Once data grid got corrupted we will lost the session store data and
> Application will not respond due to session data also corrupted and all the
> instances will auto scaled down to initial size 8.We need to restart Apache
> Ignite to make the Application up.Please find the attached Apache Ignite
> Configuration for your reference.
> This is Impacting the scalability for the micro-services. It is very evident
> that the current state based architecture will not scale beyond a certain TPS
> and the state store, especially Ignite becomes the single point of failure,
> stalling the full Micro-service cluster.
> Apache Ignite Version : 2.7.0
> {code}
> 07:24:46,678][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G]
> Blocked system-critical thread has been detected. This can lead to
> cluster-wide undefined behaviour [threadName=sys-stripe-5,
> blockedFor=21s]07:24:46,678][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G]
> Blocked system-critical thread has been detected. This can lead to
> cluster-wide undefined behaviour [threadName=sys-stripe-5,
> blockedFor=21s][07:24:46,680][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
> [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker
> [name=sys-stripe-5, igniteInstanceName=DataStoreIgniteCache, finished=false,
> heartbeatTs=1575271465499]]]class org.apache.ignite.IgniteException:
> GridWorker [name=sys-stripe-5, igniteInstanceName=DataStoreIgniteCache,
> finished=false, heartbeatTs=1575271465499] at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
> at
> org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
> at
> org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
> at
> org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:24:52,692][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G]
> Blocked system-critical thread has been detected. This can lead to
> cluster-wide undefined behaviour [threadName=ttl-cleanup-worker,
> blockedFor=27s][07:24:52,692][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
> [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker
> [name=ttl-cleanup-worker, igniteInstanceName=DataStoreIgniteCache,
> finished=false, heartbeatTs=1575271465044]]]class
> org.apache.ignite.IgniteException: GridWorker [name=ttl-cleanup-worker,
>
[jira] [Updated] (IGNITE-12456) Cluster Data Store grid gets Corrupted for Load test
[
https://issues.apache.org/jira/browse/IGNITE-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12456:
--
Description:
We have Apache Ignite 3 node cluster setup with Amazon S3 Based Discovery.we
are into AWS cloud environment with Microservice model with 8 microservices.we
are using Ignite for session data store.While performing load test we are
facing data grid issues if the number of clients reached 40 , Once data grid
got corrupted we will lost the session store data and Application will not
respond due to session data also corrupted and all the instances will auto
scaled down to initial size 8.We need to restart Apache Ignite to make the
Application up.Please find the attached Apache Ignite Configuration for your
reference.
This is Impacting the scalability for the micro-services. It is very evident
that the current state based architecture will not scale beyond a certain TPS
and the state store, especially Ignite becomes the single point of failure,
stalling the full Micro-service cluster.
Apache Ignite Version : 2.7.0
{code}
07:24:46,678][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G]
Blocked system-critical thread has been detected. This can lead to
cluster-wide undefined behaviour [threadName=sys-stripe-5,
blockedFor=21s]07:24:46,678][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G]
Blocked system-critical thread has been detected. This can lead to
cluster-wide undefined behaviour [threadName=sys-stripe-5,
blockedFor=21s][07:24:46,680][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
[type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker
[name=sys-stripe-5, igniteInstanceName=DataStoreIgniteCache, finished=false,
heartbeatTs=1575271465499]]]class org.apache.ignite.IgniteException: GridWorker
[name=sys-stripe-5, igniteInstanceName=DataStoreIgniteCache, finished=false,
heartbeatTs=1575271465499] at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
at
org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
at
org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
at
org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:24:52,692][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G]
Blocked system-critical thread has been detected. This can lead to
cluster-wide undefined behaviour [threadName=ttl-cleanup-worker,
blockedFor=27s][07:24:52,692][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
[type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker
[name=ttl-cleanup-worker, igniteInstanceName=DataStoreIgniteCache,
finished=false, heartbeatTs=1575271465044]]]class
org.apache.ignite.IgniteException: GridWorker [name=ttl-cleanup-worker,
igniteInstanceName=DataStoreIgniteCache, finished=false,
heartbeatTs=1575271465044] at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
at
org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
at
org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
at
