Hi Vladimir,
thank you for your help.
Yes because our application is stressed every 0.25ms.
I have a public REST API that encapsulate IgniteCallable and inside we do a
query on cache. Then every request yes I create a Thread
iClusterService
.submit(new RestClusterTask(
suggestion how to optimize “we cache connections on per-thread” in my scenario
will be fully appreciate.
Best Regards
Devis
Devis Balsemin, FlexValley SRL (Italy) ,
Via Dante Alighieri 4, I-36075 Montecchio Maggiore, Italy | t: +39.0444.699622
| f: +39.0444.1830543
Please consider your environmental responsibility before printing this e-mail
This communication (including any attachments) may contain privileged or
confidential information of FlexValley SRL and is intended for a specific
individual. If you are not the intended recipient, you should delete this
communication, including any attachments without reading or saving them in any
manner, and you are hereby notified that any disclosure, copying, or
distribution of this communication, or the taking of any action based on it, is
strictly prohibited.
Da: Vladimir Ozerov [via Apache Ignite Users]
[mailto:[email protected]]
Inviato: martedì 6 giugno 2017 16:47
A: devis76 <[email protected]>
Oggetto: Re: Memory Leak or my Error?
Hi Devis,
Is it correct that your application create lots of threads? Currently we cache
connections on per-thread level for performance reasons. If there are many
threads created over and over again, it could consume more and more memory.
Probably we should improve this piece of code.
On Tue, Jun 6, 2017 at 4:38 PM, Devis Balsemin <[hidden email]> wrote:
Hi Vladimir,
this is my cache configuration
CacheConfiguration<K, V> cacheCfg = new CacheConfiguration<>();
cacheCfg.setTypes(keyType, valueType);
cacheCfg.setBackups(0);
cacheCfg.setName(ctx.name <http://ctx.name> ());
cacheCfg.setCacheMode(PARTITIONED);
cacheCfg.setEagerTtl(false);
cacheCfg.setCopyOnRead(true);
cacheCfg.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(new
Duration(TimeUnit.HOURS, 24)));
cacheCfg.setIndexedTypes(keyType, valueType);
ignite.configuration().setCacheConfiguration(cacheCfg);
cache = ignite.getOrCreateCache(cacheCfg);
<http://apache-ignite-users.70518.x6.nabble.com/attachment/13425/0/image001.png>
Devis Balsemin, FlexValley SRL (Italy) ,
Via Dante Alighieri 4, I-36075 Montecchio Maggiore, Italy | t: +39.0444.699622
| f: +39.0444.1830543
Please consider your environmental responsibility before printing this e-mail
This communication (including any attachments) may contain privileged or
confidential information of FlexValley SRL and is intended for a specific
individual. If you are not the intended recipient, you should delete this
communication, including any attachments without reading or saving them in any
manner, and you are hereby notified that any disclosure, copying, or
distribution of this communication, or the taking of any action based on it, is
strictly prohibited.
Da: Vladimir Ozerov [mailto:[hidden email]]
Inviato: martedì 6 giugno 2017 15:02
A: [hidden email]
Oggetto: Re: Memory Leak or my Error?
Hi Devis,
Can you show GC roots of these Session objects?
On Tue, Jun 6, 2017 at 3:52 PM, Devis Balsemin <[hidden email]> wrote:
Hi,
I’m using Ignite 1.7.
I have a function nativeSQL that is called in many point of my programs.
But after 2/3days I receive my OOM (this function is called every 0.25ms X 10
concurrent users).
My dubmp show me
Class Name |
Objects | Shallow Heap | Retained Heap
----------------------------------------------------------------------------------------------------------------
org.h2.engine.Session |
201.406 | 51.559.936 | >= 95.063.648
char[] |
501.160 | 40.283.480 | >= 40.283.480
java.lang.Object[] |
625.053 | 21.469.296 | >= 34.984.640
org.h2.jdbc.JdbcConnection |
201.404 | 17.723.552 | >= 43.503.144
java.util.ArrayList |
616.718 | 14.801.232 | >= 38.616.296
java.util.HashMap$Node |
470.953 | 11.302.872 | >= 16.857.176
org.h2.engine.UndoLog |
201.406 | 9.667.488 | >= 32.224.960
org.apache.ignite.internal.util.GridCircularBuffer$Item |
343.040 | 8.232.960 | >= 8.232.968
java.lang.String |
500.219 | 8.003.504 | >= 47.308.152
org.h2.util.CloseWatcher |
201.404 | 6.444.928 | >= 13.375.872
java.util.HashMap$Node[] |
9.546 | 5.074.192 | >= 19.757.480
org.h2.message.Trace |
201.411 | 4.833.864 | >= 16.033.024
byte[] |
5.104 | 1.698.496 | >= 1.698.496
org.jsr166.ConcurrentHashMap8$Node[] |
85 | 1.531.792 | >= 2.455.608
org.apache.ignite.internal.util.GridCircularBuffer$Item[] |
4.216 | 1.439.616 | >= 9.672.576
org.apache.ignite.internal.processors.jobmetrics.GridJobMetricsSnapshot|
16.384 | 1.310.720 | >= 1.310.720
org.jsr166.ConcurrentLinkedHashMap$HashEntry[] |
1.308 | 1.225.408 | >= 1.225.472
Class Name | Objects | Shallow Heap
--------------------------------------------------------------
org.h2.engine.Session | 201.406 | 51.559.936
|- org.h2.jdbc.JdbcConnection | 201.404 | 17.723.552
|- org.h2.util.CloseWatcher | 201.404 | 6.444.928
|- java.util.HashMap$Node | 201.404 | 4.833.696
|- org.h2.command.dml.Select | 65 | 11.440
|- org.h2.table.TableFilter | 65 | 6.760
|- org.h2.jdbc.JdbcPreparedStatement | 65 | 5.720
|- org.h2.index.IndexCursor | 64 | 4.608
|- org.h2.command.CommandContainer | 65 | 2.600
|- org.h2.engine.Database | 1 | 368
|- java.lang.Thread | 1 | 120
|- org.jsr166.ConcurrentHashMap8$Node| 4 | 96
|- org.h2.result.LocalResult | 1 | 72
'- Total: 12 entries | |
Class Name
| Objects | Shallow Heap
-------------------------------------------------------------------------------------------------------------------
org.h2.jdbc.JdbcConnection
| 201.404 | 17.723.552
|- org.h2.util.CloseWatcher
| 201.404 | 6.444.928
|- java.lang.Object[]
| 1 | 960.400
|- org.h2.jdbc.JdbcPreparedStatement
| 65 | 5.720
|-
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$ConnectionWrapper
| 14 | 224
|- java.lang.Thread
| 1 | 120
|-
org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor$QueryRun|
1 | 32
Class Name | Objects |
Shallow Heap
--------------------------------------------------------------------------------------------
java.util.ArrayList | 616.718 |
14.801.232
|- org.h2.engine.Session | 201.406 |
51.559.936
|- org.h2.engine.UndoLog | 201.406 |
9.667.488
|- org.apache.felix.framework.capabilityset.SimpleFilter | 5.798 |
139.152
|- org.apache.felix.framework.wiring.BundleCapabilityImpl | 1.777 |
71.080
|- java.lang.Object[] | 46 |
19.560
|- org.apache.felix.framework.BundleRevisionImpl | 221 |
17.680
|- org.apache.felix.framework.BundleImpl | 221 |
14.144
|- java.util.HashMap$Node | 514 |
12.336
|- org.h2.command.dml.Select | 65 |
11.440
|- com.flexvalley.drivers.zway.internal.config.ZWaveDbProduct | 340 |
8.160
|- java.security.Provider$Service | 126 |
7.056
|- java.util.concurrent.ConcurrentHashMap$Node | 292 |
7.008
|- org.h2.table.TableFilter | 65 |
6.760
|- org.apache.cxf.bus.osgi.CXFExtensionBundleListener$OSGiExtension| 106 |
5.936
|- org.apache.xerces.dom.AttributeMap | 167 |
4.008
--------------------------------------------------------------------------------------------
H2 is used only by Ignite.
This is only one function that runs QueryCursor.
@Override
public List<V> nativeSQL(final Class<?> cls, final String whereCondition, final
Object... whereParams) {
StringBuilder sqlText=new StringBuilder("select * from
").append(cls.getSimpleName()).append(" where ").append(whereCondition);
try(QueryCursor<List<?>> qryx = cache
.query(new SqlFieldsQuery(sqlText.toString())
.setArgs(whereParams))) {
List<V> list = new CopyOnWriteArrayList<>();
iteratorToStream(qryx.iterator(),true).parallel().forEach(objects -> {
list.add((V) objects.get(1));
});
//qryx.close();
return list;
} catch (Exception ex) {
logger.error("Query Error {}", ex);
}
finally {
sqlText.setLength(0);
sqlText=null;
}
return null;
}
Can you help me understand if it’s my mistake please?
Best Regards
<http://apache-ignite-users.70518.x6.nabble.com/attachment/13425/0/image001.png>
Devis Balsemin, FlexValley SRL (Italy) ,
Via Dante Alighieri 4, I-36075 Montecchio Maggiore, Italy | t: +39.0444.699622
| f: +39.0444.1830543
Please consider your environmental responsibility before printing this e-mail
This communication (including any attachments) may contain privileged or
confidential information of FlexValley SRL and is intended for a specific
individual. If you are not the intended recipient, you should delete this
communication, including any attachments without reading or saving them in any
manner, and you are hereby notified that any disclosure, copying, or
distribution of this communication, or the taking of any action based on it, is
strictly prohibited.
_____
If you reply to this email, your message will be added to the discussion below:
http://apache-ignite-users.70518.x6.nabble.com/Memory-Leak-or-my-Error-tp13411p13425.html
To start a new topic under Apache Ignite Users, email
[email protected] <mailto:[email protected]>
To unsubscribe from Apache Ignite Users, click here
<http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=ZGV2aXMuYmFsc2VtaW5AZmxleHZhbGxleS5jb218MXw4MjYyOTQ0MTk=>
.
<http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
NAML
image001.png (18K)
<http://apache-ignite-users.70518.x6.nabble.com/attachment/13427/0/image001.png>
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/R-Memory-Leak-or-my-Error-tp13427.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
