Hi Devis, Behavior you saw doesn't depend on how many rows are returned, but rather on how many threads are created. There should not be any problems when switching from 100 to 2000 rows.
On Wed, Jun 7, 2017 at 9:20 PM, Devis Balsemin < devis.balse...@flexvalley.com> wrote: > Hi Vladimir, > > for now i have change my code in this way: > > > > From > > queryCursor = *cache*.query(*new *SqlQuery(cls, whereCondition).setArgs( > whereParams)); > > > > With > > cache.iterator() and Java 8 Stream/Filter/…. > > > > My memory now works very fine over 37.4% less of resources and CMS objects > in garbage > > > > > > What do you think about this? > > > > For your what will happen with over 2000 Rows (now I’m testing only with 100 > rows)? > > Have I to wait for a decrease in performance with this approach? > > > > Thank for your help > > Devis > > > > [image: cid:image001.png@01D12220.7E4C4300] > > > > > > Devis Balsemin, FlexValley SRL (Italy) , > Via Dante Alighieri 4, I-36075 Montecchio Maggiore, Italy | t: +39.0444.699622 > | f: +39.0444.1830543 > > > > *Please consider your environmental responsibility before printing this > e-mail* > > This communication (including any attachments) may contain privileged or > confidential information of FlexValley SRL and is intended for a specific > individual. If you are not the intended recipient, you should delete this > communication, including any attachments without reading or saving them in > any manner, and you are hereby notified that any disclosure, copying, or > distribution of this communication, or the taking of any action based on > it, is strictly prohibited. > > > > *Da:* devis76 [mailto:devis.balse...@flexvalley.com] > *Inviato:* martedì 6 giugno 2017 17:12 > *A:* user@ignite.apache.org > *Oggetto:* R: Memory Leak or my Error? > > > > Hi Vladimir, > > sorry… > > when i call “submit”…. > > I’m using a IgniteThreadPoolExecutor in this way…. > > > > *threadPoolExecutor *= *new *IgniteThreadPoolExecutor(coreSize, maxSize, > keepAlive, *arrayBlockingQueue*, *threadFactory*, *new * > ThreadPoolExecutor.CallerRunsPolicy()); > *threadPoolExecutor*.allowCoreThreadTimeOut(*true*); > > > > > > [image: cid:image001.png@01D12220.7E4C4300] > > > > > > Devis Balsemin, FlexValley SRL (Italy) , > Via Dante Alighieri 4, I-36075 Montecchio Maggiore, Italy | t: +39.0444.699622 > | f: +39.0444.1830543 > > > > *Please consider your environmental responsibility before printing this > e-mail* > > This communication (including any attachments) may contain privileged or > confidential information of FlexValley SRL and is intended for a specific > individual. If you are not the intended recipient, you should delete this > communication, including any attachments without reading or saving them in > any manner, and you are hereby notified that any disclosure, copying, or > distribution of this communication, or the taking of any action based on > it, is strictly prohibited. > > > > *Da:* Vladimir Ozerov [via Apache Ignite Users] [mailto:[hidden email] > <http:///user/SendEmail.jtp?type=node&node=13429&i=0>] > *Inviato:* martedì 6 giugno 2017 16:47 > *A:* devis76 <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=13429&i=1>> > *Oggetto:* Re: Memory Leak or my Error? > > > > Hi Devis, > > > > Is it correct that your application create lots of threads? Currently we > cache connections on per-thread level for performance reasons. If there are > many threads created over and over again, it could consume more and more > memory. Probably we should improve this piece of code. > > > > On Tue, Jun 6, 2017 at 4:38 PM, Devis Balsemin <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=13425&i=0>> wrote: > > Hi Vladimir, > > > > this is my cache configuration > > > > > CacheConfiguration<K, V> cacheCfg = *new *CacheConfiguration<>(); > cacheCfg.setTypes(*keyType*, *valueType*); > > cacheCfg.setBackups(0); > cacheCfg.setName(ctx.name()); > cacheCfg.setCacheMode(*PARTITIONED*); > cacheCfg.setEagerTtl(*false*); > cacheCfg.setCopyOnRead(*true*); > cacheCfg.setExpiryPolicyFactory(CreatedExpiryPolicy.*factoryOf*(*new * > Duration(TimeUnit.*HOURS*, 24))); > cacheCfg.setIndexedTypes(*keyType*, *valueType*); > *ignite*.configuration().setCacheConfiguration(cacheCfg); > *cache *= *ignite*.getOrCreateCache(cacheCfg); > > > > > > [image: cid:image001.png@01D12220.7E4C4300] > > > > > > Devis Balsemin, FlexValley SRL (Italy) , > Via Dante Alighieri 4, I-36075 Montecchio Maggiore, Italy | t: +39.0444.699622 > | f: +39.0444.1830543 > > > > *Please consider your environmental responsibility before printing this > e-mail* > > This communication (including any attachments) may contain privileged or > confidential information of FlexValley SRL and is intended for a specific > individual. If you are not the intended recipient, you should delete this > communication, including any attachments without reading or saving them in > any manner, and you are hereby notified that any disclosure, copying, or > distribution of this communication, or the taking of any action based on > it, is strictly prohibited. > > > > *Da:* Vladimir Ozerov [mailto:[hidden email] > <http:///user/SendEmail.jtp?type=node&node=13425&i=1>] > *Inviato:* martedì 6 giugno 2017 15:02 > *A:* [hidden email] <http:///user/SendEmail.jtp?type=node&node=13425&i=2> > *Oggetto:* Re: Memory Leak or my Error? > > > > Hi Devis, > > > > Can you show GC roots of these Session objects? > > > > On Tue, Jun 6, 2017 at 3:52 PM, Devis Balsemin <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=13425&i=3>> wrote: > > Hi, > > I’m using Ignite 1.7. > > I have a function nativeSQL that is called in many point of my programs. > > But after 2/3days I receive my OOM (this function is called every 0.25ms > X 10 concurrent users). > > > > My dubmp show me > > > > Class Name | > Objects | Shallow Heap | Retained Heap > > ------------------------------------------------------------ > ---------------------------------------------------- > > org.h2.engine.Session | > 201.406 | 51.559.936 | >= 95.063.648 > > char[] | > 501.160 | 40.283.480 | >= 40.283.480 > > java.lang.Object[] | > 625.053 | 21.469.296 | >= 34.984.640 > > org.h2.jdbc.JdbcConnection | > 201.404 | 17.723.552 | >= 43.503.144 > > java.util.ArrayList | > 616.718 | 14.801.232 | >= 38.616.296 > > java.util.HashMap$Node | > 470.953 | 11.302.872 | >= 16.857.176 > > org.h2.engine.UndoLog | > 201.406 | 9.667.488 | >= 32.224.960 > > org.apache.ignite.internal.util.GridCircularBuffer$Item | > 343.040 | 8.232.960 | >= 8.232.968 > > java.lang.String | > 500.219 | 8.003.504 | >= 47.308.152 > > org.h2.util.CloseWatcher | > 201.404 | 6.444.928 | >= 13.375.872 > > java.util.HashMap$Node[] > | 9.546 | 5.074.192 | >= 19.757.480 > > org.h2.message.Trace | > 201.411 | 4.833.864 | >= 16.033.024 > > byte[] > | 5.104 | 1.698.496 | >= 1.698.496 > > org.jsr166.ConcurrentHashMap8$Node[] > | 85 | 1.531.792 | >= 2.455.608 > > org.apache.ignite.internal.util.GridCircularBuffer$Item[] > | 4.216 | 1.439.616 | >= 9.672.576 > > org.apache.ignite.internal.processors.jobmetrics.GridJobMetricsSnapshot| > 16.384 | 1.310.720 | >= 1.310.720 > > org.jsr166.ConcurrentLinkedHashMap$HashEntry[] > | 1.308 | 1.225.408 | >= 1.225.472 > > > > > > > > Class Name | Objects | Shallow Heap > > -------------------------------------------------------------- > > org.h2.engine.Session | 201.406 | 51.559.936 > > |- org.h2.jdbc.JdbcConnection | 201.404 | 17.723.552 > > |- org.h2.util.CloseWatcher | 201.404 | 6.444.928 > > |- java.util.HashMap$Node | 201.404 | 4.833.696 > > |- org.h2.command.dml.Select | 65 | 11.440 > > |- org.h2.table.TableFilter | 65 | 6.760 > > |- org.h2.jdbc.JdbcPreparedStatement | 65 | 5.720 > > |- org.h2.index.IndexCursor | 64 | 4.608 > > |- org.h2.command.CommandContainer | 65 | 2.600 > > |- org.h2.engine.Database | 1 | 368 > > |- java.lang.Thread | 1 | 120 > > |- org.jsr166.ConcurrentHashMap8$Node| 4 | 96 > > |- org.h2.result.LocalResult | 1 | 72 > > '- Total: 12 entries | | > > > > Class Name > | Objects | Shallow Heap > > ------------------------------------------------------------ > ------------------------------------------------------- > > org.h2.jdbc.JdbcConnection > | 201.404 | 17.723.552 > > |- org.h2.util.CloseWatcher > | 201.404 | 6.444.928 > > |- java.lang.Object[] > | 1 | 960.400 > > |- org.h2.jdbc.JdbcPreparedStatement > | 65 | 5.720 > > |- > org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$ConnectionWrapper > | 14 | 224 > > |- java.lang.Thread > | 1 | 120 > > |- org.apache.ignite.internal.processors.query.h2.twostep. > GridReduceQueryExecutor$QueryRun| 1 | 32 > > > > > > > > > > Class Name | > Objects | Shallow Heap > > ------------------------------------------------------------ > -------------------------------- > > java.util.ArrayList | > 616.718 | 14.801.232 > > |- org.h2.engine.Session | > 201.406 | 51.559.936 > > |- org.h2.engine.UndoLog | > 201.406 | 9.667.488 > > |- org.apache.felix.framework.capabilityset.SimpleFilter | > 5.798 | 139.152 > > |- org.apache.felix.framework.wiring.BundleCapabilityImpl | > 1.777 | 71.080 > > |- java.lang.Object[] | > 46 | 19.560 > > |- org.apache.felix.framework.BundleRevisionImpl | > 221 | 17.680 > > |- org.apache.felix.framework.BundleImpl | > 221 | 14.144 > > |- java.util.HashMap$Node | > 514 | 12.336 > > |- org.h2.command.dml.Select | > 65 | 11.440 > > |- com.flexvalley.drivers.zway.internal.config.ZWaveDbProduct | > 340 | 8.160 > > |- java.security.Provider$Service | > 126 | 7.056 > > |- java.util.concurrent.ConcurrentHashMap$Node | > 292 | 7.008 > > |- org.h2.table.TableFilter | > 65 | 6.760 > > |- org.apache.cxf.bus.osgi.CXFExtensionBundleListener$OSGiExtension| > 106 | 5.936 > > |- org.apache.xerces.dom.AttributeMap | > 167 | 4.008 > > ------------------------------------------------------------ > -------------------------------- > > > > > > > > H2 is used only by Ignite. > > > > This is only one function that runs QueryCursor. > > > > > > @Override > *public *List<V> nativeSQL(*final *Class<?> cls, *final *String > whereCondition, *final *Object... whereParams) { > StringBuilder sqlText=*new *StringBuilder(*"select * from "* > ).append(cls.getSimpleName()).append(*" where "*).append(whereCondition); > *try*(QueryCursor<List<?>> qryx = > *cache *.query(*new *SqlFieldsQuery(sqlText.toString()) > .setArgs(whereParams))) { > List<V> list = *new *CopyOnWriteArrayList<>(); > *iteratorToStream*(qryx.iterator(),*true*).parallel().forEach(objects > -> { > list.add((V) objects.get(1)); > > }); > > *//qryx.close(); **return *list; > } *catch *(Exception ex) { > *logger*.error(*"Query Error {}"*, ex); > } > *finally *{ > sqlText.setLength(0); > sqlText=*null*; > } > *return null*; > } > > > > Can you help me understand if it’s my mistake please? > > Best Regards > > > > [image: cid:image001.png@01D12220.7E4C4300] > > > > > > Devis Balsemin, FlexValley SRL (Italy) , > Via Dante Alighieri 4, I-36075 Montecchio Maggiore, Italy | t: +39.0444.699622 > | f: +39.0444.1830543 > > > > *Please consider your environmental responsibility before printing this > e-mail* > > This communication (including any attachments) may contain privileged or > confidential information of FlexValley SRL and is intended for a specific > individual. If you are not the intended recipient, you should delete this > communication, including any attachments without reading or saving them in > any manner, and you are hereby notified that any disclosure, copying, or > distribution of this communication, or the taking of any action based on > it, is strictly prohibited. > > > > > > > > > ------------------------------ > > *If you reply to this email, your message will be added to the discussion > below:* > > http://apache-ignite-users.70518.x6.nabble.com/Memory-Leak-or-my-Error- > tp13411p13425.html > > To start a new topic under Apache Ignite Users, email [hidden email] > <http:///user/SendEmail.jtp?type=node&node=13429&i=2> > To unsubscribe from Apache Ignite Users, click here. > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > ------------------------------ > > View this message in context: R: Memory Leak or my Error? > <http://apache-ignite-users.70518.x6.nabble.com/R-Memory-Leak-or-my-Error-tp13429.html> > Sent from the Apache Ignite Users mailing list archive > <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com. >
