[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987594#comment-16987594 ] Stanilovsky Evgeny commented on IGNITE-12374: - 8 * 2128 = already yours 10k ) i`l try to bench it with thick java client, hope results will be soon. > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12413) .NET: XMLDoc does not work when using Ignite NuGet from .NET Core
[ https://issues.apache.org/jira/browse/IGNITE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987179#comment-16987179 ] Pavel Tupitsyn commented on IGNITE-12413: - Merged to master: 64c56bc08c0882ee4ca3ee730651409a3737bd0b > .NET: XMLDoc does not work when using Ignite NuGet from .NET Core > - > > Key: IGNITE-12413 > URL: https://issues.apache.org/jira/browse/IGNITE-12413 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 2.8 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Critical > Labels: .NET > Fix For: 2.8 > > Time Spent: 20m > Remaining Estimate: 0h > > * Create new .NET Core project (2.x or 3.x): dotnet new console > * Install nightly build of 2.8.0: dotnet add package Apache.Ignite -v > 2.8.0-alpha20191118 > * Open the project in any IDE (VS, VSCode, Rider) and start using Ignite APIs > There is no documentation in IDE tooltips. NuGet package is malformed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (IGNITE-12413) .NET: XMLDoc does not work when using Ignite NuGet from .NET Core
[ https://issues.apache.org/jira/browse/IGNITE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn resolved IGNITE-12413. - Resolution: Fixed > .NET: XMLDoc does not work when using Ignite NuGet from .NET Core > - > > Key: IGNITE-12413 > URL: https://issues.apache.org/jira/browse/IGNITE-12413 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 2.8 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Critical > Labels: .NET > Fix For: 2.8 > > Time Spent: 20m > Remaining Estimate: 0h > > * Create new .NET Core project (2.x or 3.x): dotnet new console > * Install nightly build of 2.8.0: dotnet add package Apache.Ignite -v > 2.8.0-alpha20191118 > * Open the project in any IDE (VS, VSCode, Rider) and start using Ignite APIs > There is no documentation in IDE tooltips. NuGet package is malformed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12412) Incomplete check for ABA problem in PageMemoryImpl#PagePool
[
https://issues.apache.org/jira/browse/IGNITE-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987098#comment-16987098
]
Ignite TC Bot commented on IGNITE-12412:
{panel:title=Branch: [pull/7098/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4808374buildTypeId=IgniteTests24Java8_RunAll]
> Incomplete check for ABA problem in PageMemoryImpl#PagePool
> ---
>
> Key: IGNITE-12412
> URL: https://issues.apache.org/jira/browse/IGNITE-12412
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In current implementation, {{PagePool#releasePage}} clears the counter part
> of the returned page ID, which effectively disables the ABA check intended in
> the class. This issue can be rarely reproduced on zOS during checkpoints
> (when pages are being taken and returned to the checkpoint pages pool).
> I managed to write a unit-test to reproduce this issue on x86.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12220) Allow to use cache-related permissions both at system and per-cache levels
[
https://issues.apache.org/jira/browse/IGNITE-12220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986980#comment-16986980
]
Ignite TC Bot commented on IGNITE-12220:
{panel:title=Branch: [pull/6904/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4793595buildTypeId=IgniteTests24Java8_RunAll]
> Allow to use cache-related permissions both at system and per-cache levels
> --
>
> Key: IGNITE-12220
> URL: https://issues.apache.org/jira/browse/IGNITE-12220
> Project: Ignite
> Issue Type: Task
> Components: security
>Affects Versions: 2.7.6
>Reporter: Andrey Kuznetsov
>Assignee: Sergei Ryzhov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Currently, {{CACHE_CREATE}} and {{CACHE_DESTROY}} permissions are enforced to
> be system-level permissions, see for instance
> {{SecurityPermissionSetBuilder#appendCachePermissions}}. This looks
> inflexible: Ignite Security implementations are not able to manage cache
> creation and deletion permissions on per-cache basis (unlike get/put/remove
> permissions). All such limitations should be found and removed in order to
> allow all {{CACHE_*}} permissions to be set both at system and per-cache
> levels.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986940#comment-16986940 ] swy commented on IGNITE-12374: -- unfortunately our product run in single thread and not so easy to refactor it at the moment. We can scale the application but only around 6 to 8 instances, because of synchronisation scaling cannot more than that. > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986938#comment-16986938 ] Ivan Pavlukhin commented on IGNITE-12401: - [~Yuriy_Shuliha], [~ilyak], I dig into the problem and found that the cause of the test failure is improper mapping of text queries on topology when there are MOVING partitions. The scenario in test is as follows: 1. Start 2 nodes. 2. Fill cache with data. 3. Execute text query and check results. And the tricky thing here is that between steps 1 and 2 there are MOVING partitions! Really misleading, because it seems that there should be no data movement as cluster starts without data, but the reality is different. As a result text query loads some entries twice (from both nodes). So, we identified that _mapping_ of text queries on topology works incorrectly. Consequently incorrect results on topologies with MOVING partitions. > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Yuriy Shuliha >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12141) Ignite Spark Integration Support Schema on Table Write
[ https://issues.apache.org/jira/browse/IGNITE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12141: - Labels: (was: await) > Ignite Spark Integration Support Schema on Table Write > -- > > Key: IGNITE-12141 > URL: https://issues.apache.org/jira/browse/IGNITE-12141 > Project: Ignite > Issue Type: Improvement > Components: spark >Affects Versions: 2.8 >Reporter: Manoj G T >Assignee: Alexey Zinoviev >Priority: Critical > Fix For: 2.8 > > Original Estimate: 4h > Remaining Estimate: 4h > > Ignite 2.6 doesn't allow to create table on any schema other than Public > Schema and this is the reason for not supporting "OPTION_SCHEMA" during > Overwrite mode. Now that Ignite supports to create the table in any given > schema it will be great if we can incorporate the changes to support > "OPTION_SCHEMA" during Overwrite mode and make it available as part of next > Ignite release. > > +Related Issue:+ > [https://stackoverflow.com/questions/57782033/apache-ignite-spark-integration-not-working-with-schema-name] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12141) Ignite Spark Integration Support Schema on Table Write
[ https://issues.apache.org/jira/browse/IGNITE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12141: - Fix Version/s: (was: 2.8) 2.9 > Ignite Spark Integration Support Schema on Table Write > -- > > Key: IGNITE-12141 > URL: https://issues.apache.org/jira/browse/IGNITE-12141 > Project: Ignite > Issue Type: Improvement > Components: spark >Affects Versions: 2.8 >Reporter: Manoj G T >Assignee: Alexey Zinoviev >Priority: Critical > Fix For: 2.9 > > Original Estimate: 4h > Remaining Estimate: 4h > > Ignite 2.6 doesn't allow to create table on any schema other than Public > Schema and this is the reason for not supporting "OPTION_SCHEMA" during > Overwrite mode. Now that Ignite supports to create the table in any given > schema it will be great if we can incorporate the changes to support > "OPTION_SCHEMA" during Overwrite mode and make it available as part of next > Ignite release. > > +Related Issue:+ > [https://stackoverflow.com/questions/57782033/apache-ignite-spark-integration-not-working-with-schema-name] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12141) Ignite Spark Integration Support Schema on Table Write
[ https://issues.apache.org/jira/browse/IGNITE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986926#comment-16986926 ] Alexey Zinoviev commented on IGNITE-12141: -- [~mmuzaf] move to 2.9 The problem is clear, no time to fix now > Ignite Spark Integration Support Schema on Table Write > -- > > Key: IGNITE-12141 > URL: https://issues.apache.org/jira/browse/IGNITE-12141 > Project: Ignite > Issue Type: Improvement > Components: spark >Affects Versions: 2.8 >Reporter: Manoj G T >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 2.8 > > Original Estimate: 4h > Remaining Estimate: 4h > > Ignite 2.6 doesn't allow to create table on any schema other than Public > Schema and this is the reason for not supporting "OPTION_SCHEMA" during > Overwrite mode. Now that Ignite supports to create the table in any given > schema it will be great if we can incorporate the changes to support > "OPTION_SCHEMA" during Overwrite mode and make it available as part of next > Ignite release. > > +Related Issue:+ > [https://stackoverflow.com/questions/57782033/apache-ignite-spark-integration-not-working-with-schema-name] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12141) Ignite Spark Integration Support Schema on Table Write
[ https://issues.apache.org/jira/browse/IGNITE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12141: - Affects Version/s: (was: 2.8) > Ignite Spark Integration Support Schema on Table Write > -- > > Key: IGNITE-12141 > URL: https://issues.apache.org/jira/browse/IGNITE-12141 > Project: Ignite > Issue Type: Improvement > Components: spark >Reporter: Manoj G T >Assignee: Alexey Zinoviev >Priority: Critical > Fix For: 2.9 > > Original Estimate: 4h > Remaining Estimate: 4h > > Ignite 2.6 doesn't allow to create table on any schema other than Public > Schema and this is the reason for not supporting "OPTION_SCHEMA" during > Overwrite mode. Now that Ignite supports to create the table in any given > schema it will be great if we can incorporate the changes to support > "OPTION_SCHEMA" during Overwrite mode and make it available as part of next > Ignite release. > > +Related Issue:+ > [https://stackoverflow.com/questions/57782033/apache-ignite-spark-integration-not-working-with-schema-name] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12141) Ignite Spark Integration Support Schema on Table Write
[ https://issues.apache.org/jira/browse/IGNITE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986925#comment-16986925 ] Maxim Muzafarov commented on IGNITE-12141: -- [~zaleslaw] Hello, still no changes? Should we wait for this issue to be included into the 2.8 release? > Ignite Spark Integration Support Schema on Table Write > -- > > Key: IGNITE-12141 > URL: https://issues.apache.org/jira/browse/IGNITE-12141 > Project: Ignite > Issue Type: Improvement > Components: spark >Affects Versions: 2.8 >Reporter: Manoj G T >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 2.8 > > Original Estimate: 4h > Remaining Estimate: 4h > > Ignite 2.6 doesn't allow to create table on any schema other than Public > Schema and this is the reason for not supporting "OPTION_SCHEMA" during > Overwrite mode. Now that Ignite supports to create the table in any given > schema it will be great if we can incorporate the changes to support > "OPTION_SCHEMA" during Overwrite mode and make it available as part of next > Ignite release. > > +Related Issue:+ > [https://stackoverflow.com/questions/57782033/apache-ignite-spark-integration-not-working-with-schema-name] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12188) Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly if there is more then one cache in cache group
[
https://issues.apache.org/jira/browse/IGNITE-12188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986907#comment-16986907
]
Aleksey Plekhanov commented on IGNITE-12188:
[~NSAmelchev], yes we should get a fully working metric by this ticket.
> Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly
> if there is more then one cache in cache group
>
>
> Key: IGNITE-12188
> URL: https://issues.apache.org/jira/browse/IGNITE-12188
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: IEP-35, await
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Initial partitions counter is set to a total number of partitions for cache
> group but decremented each time partition processed for each cache.
> Reproducer:
> {code:java}
> @Test
> public void testIndexRebuildMetrics() throws Exception {
> IgniteEx ignite = startGrid(new IgniteConfiguration()
> .setConsistentId("ignite")
> .setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new
> DataRegionConfiguration().setPersistenceEnabled(true)))
> .setCacheConfiguration(
> new CacheConfiguration Integer>("c1").setGroupName("grp").setIndexedTypes(Integer.class,
> Integer.class),
> new CacheConfiguration Integer>("c2").setGroupName("grp").setIndexedTypes(Integer.class,
> Integer.class)
> ));
> ignite.cluster().active(true);
> for (int i = 0; i < 10_000; i++)
> ignite.cache("c1").put(i, i);
> ignite.cluster().active(false);
> File dir = U.resolveWorkDirectory(U.defaultWorkDirectory(),
> DFLT_STORE_DIR, false);
> U.delete(new File(dir, "ignite/cacheGroup-grp/index.bin"));
> ignite.cluster().active(true);
> doSleep(1_000L);
>
> assertTrue(ignite.context().cache().cache("c1").context().group().metrics().getIndexBuildCountPartitionsLeft()
> >= 0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12393) Striped thread pool queue system view
[
https://issues.apache.org/jira/browse/IGNITE-12393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986904#comment-16986904
]
Amelchev Nikita commented on IGNITE-12393:
--
[~nizhikov], LGTM
> Striped thread pool queue system view
> -
>
> Key: IGNITE-12393
> URL: https://issues.apache.org/jira/browse/IGNITE-12393
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When in the production environment exist some cluster performance issues
> usually it leads to the large striped executor queue size.
> The number of tasks in the queue can observe by
> {StripedExecutorMXBean#getTotalQueueSize} metric. In the case queue size
> becomes large it's useful to have the ability to know what tasks are waiting
> for execution in the thread pool.
> Especially, for dealing with failover scenarios.
> We should create a system views to expose information about striped executor
> services queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986896#comment-16986896 ] Stanilovsky Evgeny commented on IGNITE-12374: - after warmup, results are more better: [estanilovskiy@lab34 ~]$ ./run.sh 50 2000 === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 54 seconds TPS: 1852 === [estanilovskiy@lab34 ~]$ ./run.sh 50 2000 === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 48 seconds TPS: 2083 === [estanilovskiy@lab34 ~]$ ./run.sh 50 2000 === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 47 seconds TPS: 2128 === additionally i fix here : can you refactor code for multi-threaded client connection? > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986889#comment-16986889 ] Stanilovsky Evgeny commented on IGNITE-12374: - [~isapego] may be you can spot additional optimisations in source ? [~yow] i will investigate it too. > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12412) Incomplete check for ABA problem in PageMemoryImpl#PagePool
[
https://issues.apache.org/jira/browse/IGNITE-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986887#comment-16986887
]
Alexey Goncharuk commented on IGNITE-12412:
---
[~akalashnikov] [~irakov] folks, I appreciate if you review this change as it
introduces some refactoring to {{PageMemoryImpl}}
> Incomplete check for ABA problem in PageMemoryImpl#PagePool
> ---
>
> Key: IGNITE-12412
> URL: https://issues.apache.org/jira/browse/IGNITE-12412
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In current implementation, {{PagePool#releasePage}} clears the counter part
> of the returned page ID, which effectively disables the ABA check intended in
> the class. This issue can be rarely reproduced on zOS during checkpoints
> (when pages are being taken and returned to the checkpoint pages pool).
> I managed to write a unit-test to reproduce this issue on x86.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986884#comment-16986884 ] swy commented on IGNITE-12374: -- Thanks [~zstan]. The performance is better now with 1220 TPS. === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 82 seconds TPS: 1220 === However by adding "long" into play it will drop again. 1220 TPS is still far from expectation. Do you think we have chance to boost further, as nearer as possible to 10k? > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986884#comment-16986884 ] swy edited comment on IGNITE-12374 at 12/3/19 1:13 PM: --- Thanks [~zstan]. The performance is better now with 1220 TPS. === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 82 seconds TPS: 1220 === However by adding "long" into play it will drop again. 1220 TPS is still far from expectation. Do you think we have chance to boost further, as nearer as possible to 10k? Also do you spot any problem in source code? was (Author: yow): Thanks [~zstan]. The performance is better now with 1220 TPS. === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 82 seconds TPS: 1220 === However by adding "long" into play it will drop again. 1220 TPS is still far from expectation. Do you think we have chance to boost further, as nearer as possible to 10k? > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12183) Rebalancing process metrics for cache groups
[
https://issues.apache.org/jira/browse/IGNITE-12183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-12183:
-
Labels: IEP-35 (was: IEP-35 await)
> Rebalancing process metrics for cache groups
>
>
> Key: IGNITE-12183
> URL: https://issues.apache.org/jira/browse/IGNITE-12183
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksandr Brazhnikov
>Priority: Major
> Labels: IEP-35
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h3. Intro
> Currently, some of the Apache Ignite rebalance process metrics are not
> working well enough. For instance, `EstimatedRebalancingKeys` keys time to
> time returns `-1` value due to a bug, or `rebalanceKeysReceived` metric
> treated as CacheMetric in fact calculated for the whole cache group (e.g.
> historical rebalance, see IGNITE-11330 and code block comment below).
> All the rebalance process metrics must be re-worked.
> {code:java}
> /**
> * Update rebalancing metrics.
> */
> private void updateGroupMetrics() {
> // TODO: IGNITE-11330: Update metrics for touched cache only.
> // Due to historical rebalancing "EstimatedRebalancingKeys" metric is
> currently calculated for the whole cache
> // group (by partition counters), so "RebalancedKeys" and
> "RebalancingKeysRate" is calculated in the same way.
> for (GridCacheContext cctx0 : grp.caches()) {
> if (cctx0.statisticsEnabled())
> cctx0.cache().metrics0().onRebalanceKeyReceived();
> }
> }
> {code}
> h3. What we have
> _CacheMetrics_ - statistics must be enabled to see these metrics.
> * getRebalancedKeys
> * getKeysToRebalanceLeft
> * getEstimatedRebalancingKeys
> * getEstimatedRebalancingFinishTime
> * getRebalancingStartTime
> * getRebalanceClearingPartitionsLeft
> * getRebalancingKeysRate
> * getRebalancingBytesRate
> h3. What to do
> All such metrics (or their analogue) must be available for the
> _CacheGroupMetrics_. I'd suggest to do the following:
> # Phase-1
> #* rebalancingPartitionsLeft long metric
> #* rebalancingReceivedKeys long metric
> #* rebalancingReceivedBytes long metric
> #* rebalancingStartTime long metric
> #* rebalancingFinishTime long metric
> # Phase-2
> #* rebalancingExpectedKeys long metric
> #* rebalancingExpectedBytes long metric
> #* rebalancingEvictedPartitionsLeft long metric
> # Phase-3 (statistics must be enabled)
> #* rebalancingKeysRate HitRate metric
> #* rebalancingBytesRate HitRate metric
> # Phase-4
> #* Mark rebalancing _CacheMetrics_ deprecated and remove from metrics
> framework IGNITE-11961.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12183) Rebalancing process metrics for cache groups
[
https://issues.apache.org/jira/browse/IGNITE-12183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-12183:
-
Fix Version/s: (was: 2.8)
> Rebalancing process metrics for cache groups
>
>
> Key: IGNITE-12183
> URL: https://issues.apache.org/jira/browse/IGNITE-12183
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksandr Brazhnikov
>Priority: Major
> Labels: IEP-35, await
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h3. Intro
> Currently, some of the Apache Ignite rebalance process metrics are not
> working well enough. For instance, `EstimatedRebalancingKeys` keys time to
> time returns `-1` value due to a bug, or `rebalanceKeysReceived` metric
> treated as CacheMetric in fact calculated for the whole cache group (e.g.
> historical rebalance, see IGNITE-11330 and code block comment below).
> All the rebalance process metrics must be re-worked.
> {code:java}
> /**
> * Update rebalancing metrics.
> */
> private void updateGroupMetrics() {
> // TODO: IGNITE-11330: Update metrics for touched cache only.
> // Due to historical rebalancing "EstimatedRebalancingKeys" metric is
> currently calculated for the whole cache
> // group (by partition counters), so "RebalancedKeys" and
> "RebalancingKeysRate" is calculated in the same way.
> for (GridCacheContext cctx0 : grp.caches()) {
> if (cctx0.statisticsEnabled())
> cctx0.cache().metrics0().onRebalanceKeyReceived();
> }
> }
> {code}
> h3. What we have
> _CacheMetrics_ - statistics must be enabled to see these metrics.
> * getRebalancedKeys
> * getKeysToRebalanceLeft
> * getEstimatedRebalancingKeys
> * getEstimatedRebalancingFinishTime
> * getRebalancingStartTime
> * getRebalanceClearingPartitionsLeft
> * getRebalancingKeysRate
> * getRebalancingBytesRate
> h3. What to do
> All such metrics (or their analogue) must be available for the
> _CacheGroupMetrics_. I'd suggest to do the following:
> # Phase-1
> #* rebalancingPartitionsLeft long metric
> #* rebalancingReceivedKeys long metric
> #* rebalancingReceivedBytes long metric
> #* rebalancingStartTime long metric
> #* rebalancingFinishTime long metric
> # Phase-2
> #* rebalancingExpectedKeys long metric
> #* rebalancingExpectedBytes long metric
> #* rebalancingEvictedPartitionsLeft long metric
> # Phase-3 (statistics must be enabled)
> #* rebalancingKeysRate HitRate metric
> #* rebalancingBytesRate HitRate metric
> # Phase-4
> #* Mark rebalancing _CacheMetrics_ deprecated and remove from metrics
> framework IGNITE-11961.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12414) .NET: Performance: review CopyOnWriteConcurrentDictionary.GetOrAdd usage and locking
Pavel Tupitsyn created IGNITE-12414: --- Summary: .NET: Performance: review CopyOnWriteConcurrentDictionary.GetOrAdd usage and locking Key: IGNITE-12414 URL: https://issues.apache.org/jira/browse/IGNITE-12414 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0, 2.9 CopyOnWriteConcurrentDictionary.GetOrAdd uses lock right away, while the class assumes frequent reads and infrequent writes. It can be beneficial to check for the key outside of the lock. In particular, this often causes contention because of BinarySystemHandlers.GetWriteHandler call. Review other usages of this method. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (IGNITE-12410) Too long muted tests should be ignored with annotation
[ https://issues.apache.org/jira/browse/IGNITE-12410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita closed IGNITE-12410. > Too long muted tests should be ignored with annotation > -- > > Key: IGNITE-12410 > URL: https://issues.apache.org/jira/browse/IGNITE-12410 > Project: Ignite > Issue Type: Task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > There are some tests that muted and having long execution time: > Test: CacheGroupsMetricsRebalanceTest.testRebalanceEstimateFinishTime > Duration: 2m:37s,377ms (x2, 2 suite) > Muted by: https://issues.apache.org/jira/browse/IGNITE-9391 > Fail rate: 97% > Test: IgniteOsgiServiceTest.testServiceExposedAndCallbacksInvoked > Duration: 3m:04s,307ms > Muted by: https://issues.apache.org/jira/browse/IGNITE-8300 > Fail rate: 93% > Test: > IgniteKarafFeaturesInstallationTest.testAllBundlesActiveAndFeaturesInstalled > Duration: 3m:01s,369ms > Muted by: https://issues.apache.org/jira/browse/IGNITE-8254 > Fail rate: 100% > Test: > JoinInActiveNodeToActiveClusterWithPersistence.testJoinClientStaticCacheConfigurationInCluster > Duration: 9s,194ms > Muted by: https://issues.apache.org/jira/browse/IGNITE-5518 > Fail rate: 100% > Total time: 11+ minutes > They should be ignored with the ignored annotation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12413) .NET: XMLDoc does not work when using Ignite NuGet from .NET Core
[ https://issues.apache.org/jira/browse/IGNITE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986697#comment-16986697 ] Pavel Tupitsyn commented on IGNITE-12413: - Looks like this happens only on Linux, because xml file has uppercase extension: Apache.Ignite.Core.XML > .NET: XMLDoc does not work when using Ignite NuGet from .NET Core > - > > Key: IGNITE-12413 > URL: https://issues.apache.org/jira/browse/IGNITE-12413 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 2.8 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Critical > Labels: .NET > Fix For: 2.8 > > > * Create new .NET Core project (2.x or 3.x): dotnet new console > * Install nightly build of 2.8.0: dotnet add package Apache.Ignite -v > 2.8.0-alpha20191118 > * Open the project in any IDE (VS, VSCode, Rider) and start using Ignite APIs > There is no documentation in IDE tooltips. NuGet package is malformed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
