[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989701#comment-16989701
]
Stanilovsky Evgeny edited comment on IGNITE-12374 at 12/6/19 12:29 PM:
---
[~yow] look, you can got some speed boost if :
1. replicated cache appropriate for your case :WITH
\"template=REPLICATED,CACHE_NAME=cdrtest_cacheR,WRITE_SYNCHRONIZATION_MODE=PRIMARY_SYNC\"
2. call run.sh with default 50 2000 with clear persistent base can give flaky
results cause some time need for inner structures initialization, partitions
etc, run first warmup run and second one will give you clear results.
3. can you change your key for example for Long ?
4. looks like 10k per one thread is impossible now.
looks like that`s all i can give you.
was (Author: zstan):
[~yow] look, you can got some speed boost if :
1. replicated cache appropriate for your case :WITH
\"template=REPLICATED,CACHE_NAME=cdrtest_cacheR,WRITE_SYNCHRONIZATION_MODE=PRIMARY_SYNC\"
2. call run.sh with default 50 2000 witch clear persistent base can give flaky
results cause some time need for inner structures initialization, partitions
etc, run first warmup run and second one will give you clear results.
3. can you change your key for example for Long ?
4. looks like 10k per one thread is impossible now.
looks like that`s all i can give you.
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample,
> odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png,
> profiling03.png, profling02.png, screenshot-1.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
> select+update operation.
> Please refer to attach sample program. It is just a simple test case.
> Based on the profiling most of the time consumed by sql execution. Please
> advice if the application did not do the right thing.
> Thank you.
> local ignite server, but the result same to remote container Ignite
> deployment too.
> {{cat /etc/apache-ignite/my-config.xml}}
> {noformat}
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd;>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration"/>
>
>
>
>
>
> class="org.apache.ignite.binary.BinaryBasicIdMapper">
>
>
>
>
>
>
>
> {noformat}
> {{g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include
> -I./ignite/common/os/linux/include -I./ignite/common/os/win/include
> -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample
> -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986884#comment-16986884 ] swy edited comment on IGNITE-12374 at 12/3/19 1:13 PM: --- Thanks [~zstan]. The performance is better now with 1220 TPS. === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 82 seconds TPS: 1220 === However by adding "long" into play it will drop again. 1220 TPS is still far from expectation. Do you think we have chance to boost further, as nearer as possible to 10k? Also do you spot any problem in source code? was (Author: yow): Thanks [~zstan]. The performance is better now with 1220 TPS. === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 82 seconds TPS: 1220 === However by adding "long" into play it will drop again. 1220 TPS is still far from expectation. Do you think we have chance to boost further, as nearer as possible to 10k? > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986677#comment-16986677
]
Stanilovsky Evgeny edited comment on IGNITE-12374 at 12/3/19 7:27 AM:
--
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it (my-config-zstan.xml)
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
further steps : remove bigdecimal, use index only with long or some simple
types, tune your network (my servers are all bare with real 10G network), ones
more read perf tuning documentation.
is it ok ?
was (Author: zstan):
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it (my-config-zstan.xml)
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample,
> odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png,
> profiling03.png, profling02.png, screenshot-1.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
> select+update operation.
> Please refer to attach sample program. It is just a simple test case.
> Based on the profiling most of the time consumed by sql execution. Please
> advice if the application did not do the right thing.
> Thank you.
> local ignite server, but the result same to remote container Ignite
> deployment too.
> cat /etc/apache-ignite/my-config.xml
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd;>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration"/>
>
>
>
>
>
> class="org.apache.ignite.binary.BinaryBasicIdMapper">
>
>
>
>
>
>
>
> g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include
> -I./ignite/common/os/linux/include -I./ignite/common/os/win/include
> -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample
> -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986677#comment-16986677
]
Stanilovsky Evgeny edited comment on IGNITE-12374 at 12/3/19 7:25 AM:
--
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it (my-config-zstan.xml)
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
was (Author: zstan):
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample,
> odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png,
> profiling03.png, profling02.png, screenshot-1.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
> select+update operation.
> Please refer to attach sample program. It is just a simple test case.
> Based on the profiling most of the time consumed by sql execution. Please
> advice if the application did not do the right thing.
> Thank you.
> local ignite server, but the result same to remote container Ignite
> deployment too.
> cat /etc/apache-ignite/my-config.xml
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd;>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration"/>
>
>
>
>
>
> class="org.apache.ignite.binary.BinaryBasicIdMapper">
>
>
>
>
>
>
>
> g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include
> -I./ignite/common/os/linux/include -I./ignite/common/os/win/include
> -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample
> -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984837#comment-16984837
]
swy edited comment on IGNITE-12374 at 11/29/19 9:19 AM:
hi [~zstan], please check attachment ignite-logs-config-source-20191129.zip. It
contain all information you need including sample app binary and ./run.sh to
execute the test.
1. copy binary "odbcsample" to test dir.
2. copy run.sh to test dir.
3. open run.sh, modity the following for ignite server addr and port.
ADDRESS="*localhost*"
PORT="*10800*"
4. also point the LD_LIBRARY_PATH to your ignite libraries in run.sh
export LD_LIBRARY_PATH=*${PWD}/ignite-libs*:$LD_LIBRARY_PATH
for example in our lab ignite-libs dir contain,
#./ignite-libs
#total 47184
#drwxrwxr-x. 2 user1 user14096 Oct 28 11:11 .
#drwxrwxr-x. 4 user1 user1 92 Nov 29 03:39 ..
#-rw-rw-r--. 1 user1 user1 3093840 Aug 29 11:38 libignite-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user1 3093840 Aug 29 11:38 libignite-2.7.5.38010.so.0.0.0
#-rw-rw-r--. 1 user1 user1 8833950 Aug 29 11:38 libignite.a
#-rw-rw-r--. 1 user1 user1 1697232 Oct 28 11:09
libignite-binary-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user1 678128 Oct 28 11:09
libignite-common-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user1 448896 Oct 28 11:08 libignite-jni-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user11255 Aug 29 11:38 libignite.lai
#-rw-rw-r--. 1 user1 user1 9115128 Aug 21 18:35 libignite-odbc-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user1 9115128 Aug 21 18:35
libignite-odbc-2.7.5.38010.so.0.0.0
#-rw-rw-r--. 1 user1 user1 9115128 Aug 21 18:35 libignite-odbc.so
#-rw-rw-r--. 1 user1 user1 3093840 Aug 29 11:38 libignite.so
5. you can now just run ./run.sh, it will generate data and test before output
performance TPS.
If you are interested in how the source code being compile, it is in the run.sh
fyr. Let me know if you have issue. Thanks.
PS.
threaddump-1575017063829.tdump and snapshot generated from window at 16:42 on
29th
ignite-9d152e2f.0.log, safepoint.log generated from server at 03:42 on 29th
tdump and snapshot is ahead 13 hours
was (Author: yow):
hi [~zstan], please check attachment ignite-logs-config-source-20191129.zip. It
contain all information you need including sample app binary and ./run.sh to
execute the test.
1. copy binary "odbcsample" to test dir.
2. copy run.sh to test dir.
3. open run.sh, modity the following for ignite server addr and port.
ADDRESS="*localhost*"
PORT="*10800*"
4. also point the LD_LIBRARY_PATH to your ignite libraries in run.sh
export LD_LIBRARY_PATH=*${PWD}/ignite-libs*:$LD_LIBRARY_PATH
for example in our lab ignite-libs dir contain,
#./ignite-libs
#total 47184
#drwxrwxr-x. 2 user1 user14096 Oct 28 11:11 .
#drwxrwxr-x. 4 user1 user1 92 Nov 29 03:39 ..
#-rw-rw-r--. 1 user1 user1 3093840 Aug 29 11:38 libignite-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user1 3093840 Aug 29 11:38 libignite-2.7.5.38010.so.0.0.0
#-rw-rw-r--. 1 user1 user1 8833950 Aug 29 11:38 libignite.a
#-rw-rw-r--. 1 user1 user1 1697232 Oct 28 11:09
libignite-binary-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user1 678128 Oct 28 11:09
libignite-common-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user1 448896 Oct 28 11:08 libignite-jni-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user11255 Aug 29 11:38 libignite.lai
#-rw-rw-r--. 1 user1 user1 9115128 Aug 21 18:35 libignite-odbc-2.7.5.38010.so.0
#-rw-rw-r--. 1 user1 user1 9115128 Aug 21 18:35
libignite-odbc-2.7.5.38010.so.0.0.0
#-rw-rw-r--. 1 user1 user1 9115128 Aug 21 18:35 libignite-odbc.so
#-rw-rw-r--. 1 user1 user1 3093840 Aug 29 11:38 libignite.so
5. you can now just run ./run.sh, it will generate data and test before output
performance TPS.
If you are interested in how the source code being compile, it is in the run.sh
fyr. Let me know if you have issue. Thanks.
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, odbcsample.allchar.rebind.cc,
> odbcsample.cc, profiling01.png, profiling03.png, profling02.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
>
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984121#comment-16984121 ] swy edited comment on IGNITE-12374 at 11/28/19 3:31 AM: [~zstan] please find ignite-logs-config-source-20191128.zip attached. It include, 1. source code of sample app 2. my-config.xml for ignite configuration 3. tdump 4. java visualvm snapshot 5. heapdump hdump, threaddump and snapshot generated from window at 11:10am on 28th ignite-ee834330.0.log generated from server at 22:10pm on 27th so tdump, heapdump and snapshot is ahead 13 hours from ignite log. Can you help also to run the same app in your environment? It would be easier to isolate environment and configuration issue. Thanks. was (Author: yow): [~zstan] please find ignite-logs-config-source-20191128.zip attached. It include, 1. source code of sample app 2. my-config.xml for ignite configuration 3. tdump 4. java visualvm snapshot 5. heapdump hdump, threaddump and snapshot generated from window at 11:10am ignite-ee834330.0.log generated from server at 22:10pm so tdump, heapdump and snapshot is ahead 13 hours from ignite log. Can you help also to run the same app in your environment? It would be easier to isolate environment and configuration issue. Thanks. > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, odbcsample.allchar.rebind.cc, > odbcsample.cc, profiling01.png, profiling03.png, profling02.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984121#comment-16984121 ] swy edited comment on IGNITE-12374 at 11/28/19 3:28 AM: [~zstan] please find ignite-logs-config-source-20191128.zip attached. It include, 1. source code of sample app 2. my-config.xml for ignite configuration 3. tdump 4. java visualvm snapshot 5. heapdump hdump, threaddump and snapshot generated from window at 11:10am ignite-ee834330.0.log generated from server at 22:10pm so tdump, heapdump and snapshot is ahead 13 hours from ignite log. Can you help also to run the same app in your environment? It would be easier to isolate environment and configuration issue. Thanks. was (Author: yow): [~zstan] please find ignite-logs-config-source-20191128.zip attached. It include, 1. source code of sample app 2. my-config.xml for ignite configuration 3. tdump 4. java visualvm snapshot 5. heapdump hdump, threaddump and snapshot generated from window at 11:10am ignite-ee834330.0.log generated from server at 22:10pm so tdump, heapdump and snapshot is ahead 13 hours from ignite log. > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, odbcsample.allchar.rebind.cc, > odbcsample.cc, profiling01.png, profiling03.png, profling02.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984077#comment-16984077 ] swy edited comment on IGNITE-12374 at 11/28/19 1:58 AM: I found the timezone diff between ignite host and window that generate threadump. Let me cleanup and regenerate. -DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is something recommended to improve performance but I did not see too much of difference. No cache config in xml. Only in source code via ODBC cache creation, and nothing special too if you refer to attach source code. Can you help to run the same source code in your environment to isolate those configuration and environmental issue? Or do you have problem to compile the sample application with ignite library? I can share the binary if that is the case. Thanks. was (Author: yow): the time is matched, or you mean others? In log: *[05:01:31,220]*[INFO][grid-timeout-worker-#71][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=9bb53f34, uptime=01:05:00.331] ^-- H/N/C [hosts=1, nodes=1, CPUs=32] ^-- CPU [cur=0.1%, avg=0.56%, GC=0%] ^-- PageMemory [pages=8540] ^-- Heap [used=486MB, free=98.42%, comm=1024MB] ^-- Off-heap [used=33MB, free=99.83%, comm=336MB] ^-- sysMemPlc region [used=0MB, free=99.21%, comm=40MB] ^-- default region [used=32MB, free=99.84%, comm=256MB] ^-- TxLog region [used=0MB, free=100%, comm=40MB] ^-- Outbound messages queue [size=0] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=0, idle=6, qSize=0] In dump: *2019-11-27 17:01:48* Full thread dump OpenJDK 64-Bit Server VM (25.232-b09 mixed mode): "sys-#194" - Thread t@240 java.lang.Thread.State: TIMED_WAITING at sun.misc.Unsafe.park(Native Method) - parking to wait for <4daa692c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None -DIGNITE_MAX_INDEX_PAYLOAD_SIZE=64 is something recommended to improve performance but I did not see too much of difference. No cache config in xml. Only in source code via ODBC cache creation, and nothing special too if you refer to attach source code. Can you help to run the same source code in your environment to isolate those configuration and environmental issue? Or do you have problem to compile the sample application with ignite library? I can share the binary if that is the case. Thanks. > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, odbcsample.allchar.rebind.cc, > odbcsample.cc, profiling01.png, profiling03.png, profling02.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > >
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983356#comment-16983356 ] Stanilovsky Evgeny edited comment on IGNITE-12374 at 11/27/19 10:20 AM: ok , but i request : "attach logs" , ignite*.log on your working directory. Can you attach it ? the same in latest dump: "RMI TCP Connection(10)-135.253.176.192" - Thread t@153 java.lang.Thread.State: RUNNABLE at sun.management.ThreadImpl.dumpThreads0(Native Method) was (Author: zstan): ok , but i request : "attach logs" , ignite*.log on your working directory. Can you attach it ? > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: odbcsample.allchar.rebind.cc, odbcsample.cc, > profiling01.png, profiling03.png, profling02.png, snapshot-1574845275597.nps, > threaddump-1573207804944.tdump, threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982051#comment-16982051 ] swy edited comment on IGNITE-12374 at 11/26/19 2:19 AM: [~isapego] any clue to share? Or can I conclude this as the limit of apache ignite in ODBC case? Or any other channel we can bring this forward? was (Author: yow): [~isapego] any clue to share? Or can I conclude this is the limit of apache ignite? > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: odbcsample.allchar.rebind.cc, odbcsample.cc, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
