RE: CacheAbstractJdbcStore batch deleteAll bug.

[Gordon Reid (Nine Mile)]

Sorry, I don’t have time to produce a sample right now, but I can see that this 
bug still exists in ignite 2.0

Just look in CacheAbstractJdbcStore.deleteAll

You can see that delStmt never gets updated after it is first initialized

Thanks,
Gordon.

From: Alexey Kuznetsov [mailto:akuznet...@apache.org]
Sent: Thursday, 27 April 2017 6:50 PM
To: user@ignite.apache.org
Subject: Re: CacheAbstractJdbcStore batch deleteAll bug.

Hi, Gordon.

Do you have a reproducer we could debug?

Thanks.

On Wed, Apr 26, 2017 at 9:50 AM, Gordon Reid (Nine Mile) 
> 
wrote:
Actually the fix will be a little more complicated, because the variable “em” 
has already been updated to the new type before the last batch has been 
executed.

From: Gordon Reid (Nine Mile) 
[mailto:gordon.r...@ninemilefinancial.com]
Sent: Wednesday, 26 April 2017 12:05 PM
To: user@ignite.apache.org
Subject: CacheAbstractJdbcStore batch deleteAll bug.

Hi Guys,

I think there is a problem in CacheAbstractJdbcStore.deleteAll

I find that sometimes, I see messages like this

[DEBUG] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore - 
Delete entries from db [cache=D9Cache, keyType=com.nmf.model.trading.TradeKey, 
cnt=2]
[WARN ] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore - 
Batch deleteAll returned unexpected updated row count [table=public.nmfctrade, 
entry=RiskRuleKey [id=1], expected=1, actual=0]
[WARN ] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore - 
Batch deleteAll returned unexpected updated row count [table=public.nmfctrade, 
entry=RiskRuleKey [id=2], expected=1, actual=0]

Note the entity types are different!

Also look at the attached image. You can see the statement is still targeting 
Trade, but our entity is now Position.

You can see in the code, that the delStmt never gets refreshed, when changing 
over to a different entity type.


for (Object key : keys) {
Object keyTypeId = typeIdForObject(key);

em = entryMapping(cacheName, keyTypeId);

if (delStmt == null) {
delStmt = conn.prepareStatement(em.remQry);

currKeyTypeId = keyTypeId;
}

if (!currKeyTypeId.equals(keyTypeId)) {
if (log.isDebugEnabled())
log.debug("Delete entries from db [cache=" + U.maskName(cacheName) +
", keyType=" + em.keyType() + ", cnt=" + prepared + "]");

executeBatch(em, delStmt, "deleteAll", fromIdx, prepared, lazyKeys);

fromIdx += prepared;

prepared = 0;

currKeyTypeId = keyTypeId;

}


I believe should be something like this


for (Object key : keys) {
Object keyTypeId = typeIdForObject(key);

em = entryMapping(cacheName, keyTypeId);

if (delStmt == null) {
delStmt = conn.prepareStatement(em.remQry);

currKeyTypeId = keyTypeId;
}

if (!currKeyTypeId.equals(keyTypeId)) {
if (log.isDebugEnabled())
log.debug("Delete entries from db [cache=" + U.maskName(cacheName) +
", keyType=" + em.keyType() + ", cnt=" + prepared + "]");

executeBatch(em, delStmt, "deleteAll", fromIdx, prepared, lazyKeys);

fromIdx += prepared;

prepared = 0;

currKeyTypeId = keyTypeId;

delStmt = conn.prepareStatement(em.remQry);
}

Thanks,
Gordon.


This email and any attachments are proprietary & confidential and are intended 
solely for the use of the individuals to whom it is addressed. Any views or 
opinions expressed are solely for those of the author and do not necessarily 
reflect those of Nine Mile Financial Pty. Limited. If you have received this 
email in error, please let us know immediately by reply email and delete from 
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252


This email and any attachments are proprietary & confidential and are intended 
solely for the use of the individuals to whom it is addressed. Any views or 
opinions expressed are solely for those of the author and do not necessarily 
reflect those of Nine Mile Financial Pty. Limited. If you have received this 
email in error, please let us know immediately by reply email and delete from 
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252


This email and any attachments are proprietary & confidential and are intended 
solely for the use of the individuals to whom it is addressed. Any views or 
opinions expressed are solely for those of the author and do not necessarily 
reflect those of Nine Mile Financial Pty. Limited. If you have received this 
email in error, please let us know immediately by reply email and delete from 
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252



--
Alexey Kuznetsov


This email and any attachments are proprietary & confidential and are intended 
solely for the use of the individuals to whom it is addressed. 

Re: vertx-ignite

[Anil]

HI Andrey,

i am using vertx-ignite 3.4.1 and ignite 1.9 version. i will check the
default-ignite.xml.

Thanks

On 10 May 2017 at 21:31, Andrey Gura  wrote:

> Anil,
>
> What version of vertx-ignite or Ignite itself do you use?
>
> In provided ignite.xml there is no minimal configuration that is
> mandatory for Ignite cluster manager for vert.x (see
> default-ignite.xml for example).
>
>
> On Tue, May 2, 2017 at 9:18 AM, Anil  wrote:
> >
> > Hi Andrey,
> >
> > Apologies for late reply. I don't have any exact reproduce. I can see
> this
> > log frequently in our logs.
> >
> > attached the ignite.xml.
> >
> > Thanks.
> >
> >
> >
> > On 26 April 2017 at 18:32, Andrey Gura  wrote:
> >>
> >> Anil,
> >>
> >> what kind of lock do you mean? What are steps for reproduce? What
> >> version if vert-ignite do use and what is your configuration?
> >>
> >> On Wed, Apr 26, 2017 at 2:16 PM, Anil  wrote:
> >> > HI,
> >> >
> >> > I am using vertx-ignite and when node is left the topology, lock is
> not
> >> > getting released and whole server is not responding.
> >> >
> >> > 2017-04-26 04:09:15 WARN  vertx-blocked-thread-checker
> >> > BlockedThreadChecker:57 - Thread
> >> > Thread[vert.x-worker-thread-82,5,ignite]
> >> > has been blocked for 2329981 ms, time limit is 6
> >> > io.vertx.core.VertxException: Thread blocked
> >> > at sun.misc.Unsafe.park(Native Method)
> >> > at
> >> > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> >> > at
> >> >
> >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.
> parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> >> > at
> >> >
> >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.
> doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> >> > at
> >> >
> >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.
> acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> >> > at
> >> >
> >> > org.apache.ignite.internal.util.future.GridFutureAdapter.
> get0(GridFutureAdapter.java:161)
> >> > at
> >> >
> >> > org.apache.ignite.internal.util.future.GridFutureAdapter.
> get(GridFutureAdapter.java:119)
> >> > at
> >> >
> >> > org.apache.ignite.internal.processors.cache.distributed.
> dht.atomic.GridDhtAtomicCache.get0(GridDhtAtomicCache.java:488)
> >> > at
> >> >
> >> > org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(
> GridCacheAdapter.java:4663)
> >> > at
> >> >
> >> > org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(
> GridCacheAdapter.java:1388)
> >> > at
> >> >
> >> > org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(
> IgniteCacheProxy.java:1117)
> >> > at io.vertx.spi.cluster.ignite.impl.MapImpl.get(MapImpl.java:
> 81)
> >> > at
> >> > io.vertx.core.impl.HAManager.chooseHashedNode(HAManager.java:590)
> >> > at io.vertx.core.impl.HAManager.checkSubs(HAManager.java:519)
> >> > at io.vertx.core.impl.HAManager.nodeLeft(HAManager.java:305)
> >> > at io.vertx.core.impl.HAManager.
> access$100(HAManager.java:107)
> >> > at io.vertx.core.impl.HAManager$
> 1.nodeLeft(HAManager.java:157)
> >> > at
> >> >
> >> > io.vertx.spi.cluster.ignite.IgniteClusterManager.lambda$
> null$4(IgniteClusterManager.java:254)
> >> > at
> >> >
> >> > io.vertx.spi.cluster.ignite.IgniteClusterManager$$Lambda$
> 36/837728834.handle(Unknown
> >> > Source)
> >> > at
> >> >
> >> > io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.
> java:271)
> >> > at
> >> > io.vertx.core.impl.ContextImpl$$Lambda$13/116289363.run(Unknown
> >> > Source)
> >> > at io.vertx.core.impl.TaskQueue.lambda$new$0(TaskQueue.java:
> 60)
> >> > at io.vertx.core.impl.TaskQueue$$Lambda$12/443290224.run(
> Unknown
> >> > Source)
> >> > at
> >> >
> >> > java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> >> > at
> >> >
> >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> >
> >> > was it a known issue ?
> >> >
> >> > Thanks
> >
> >
>

Re: Write Behind with delete performance

[Sasha Belyak]

Yes, if key wasn't flushed (end event wasn't collected by flusher thread to
flush) - it will be just overwritten in memory with new value (doesn't
matter what operation, i.e. insert can be overwritten to delete or visa
versa).
So in your example if WB cache get
insert k1 =0, k2=2, k3 =3 and delete k1=0
and don't start flush between these operations - it eventually write to
store:
deleteAll( k1=0 ), writeAll( k2=2, k3=3 ).
Of course, WB cache must process deleteAll ( k1=0 ) because it can't know
if the k1 key already be in store before our example.
You can read about it in
https://apacheignite.readme.io/docs/persistent-store#write-behind-caching

2017-05-11 0:24 GMT+07:00 waterg :

> Thank you for the quick explaining and creating jira tickets.
>
> Just a thought:
> If insert k1 =0, k2=2, k3 =3 and delete k1=0 and k1 is not being flushed to
> store yet, would it be possible to just delete in memory, w/o flushing to
> store? and deleteAll for all k1 like records that have been flushed to
> store?
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Write-Behind-with-delete-
> performance-tp12580p12604.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: [ANNOUNCE] Apache Ignite 2.0.0 Released

[Denis Magda]

Pavel, excellent blog post! Truly profound with cool source code examples:
https://twitter.com/denismagda/status/862460325211197441

—
Denis

> On May 10, 2017, at 8:26 AM, Pavel Tupitsyn  wrote:
> 
> Ignite.NET 2.0 blog post: 
> https://ptupitsyn.github.io/Whats-New-In-Ignite-Net-2.0/ 
> 
> 
> On Wed, May 10, 2017 at 12:44 PM, Pavel Tupitsyn  > wrote:
> Great news!
> 
> Thanks Denis, I've done some minor corrections to .NET part of release notes 
> page (IGNITE-4495 does not belong there).
> 
> On Fri, May 5, 2017 at 11:13 PM, Denis Magda  > wrote:
> The Apache Ignite Community is pleased to announce the release of Apache 
> Ignite 2.0.0.
> 
> Apache Ignite In-Memory Data Fabric [1] is a high-performance, integrated and 
> distributed in-memory platform for computing and transacting on large-scale 
> data sets in real-time, orders of magnitude faster than possible with 
> traditional disk-based or flash-based technologies.
> 
> The Fabric is a collection of independent and well integrated components some 
> of which are the following:
> - Data Grid
> - SQL Grid
> - Compute Grid
> - Service Grid
> - Machine Learning Grid (NEW!)
> 
> This release incorporates tremendous changes that shifted Ignite 2.0 to the 
> next level. The whole off-heap memory architecture was redesigned from 
> scratch and, going forward, Ignite can be easily integrated with Flash and 
> SSD drives (stay tuned!). SQL Grid has been enriched with Data Definition 
> Language support. A completely new component was added to the fabric - 
> Machine Learning Grid. We provided integrations with RocketMQ and Spring 
> Data, updated Hibernate integration, introduced a plugin system for .NET and 
> found out how to execute C++ code on remote node, and many more...
> 
> Get more details from this blog post: 
> https://blogs.apache.org/ignite/entry/apache-ignite-2-0-redesigned 
> 
> 
> The full list of the changes can be found here [2].
> 
> Please visit this page if you’re ready to try the release out:
> https://ignite.apache.org/download.cgi 
> 
> 
> Please let us know [3] if you encounter any problems.
> 
> Regards,
> 
> The Apache Ignite Community
> 
> [1] https://ignite.apache.org 
> [2] https://ignite.apache.org/releases/2.0.0/release_notes.html 
> 
> [3] https://ignite.apache.org/community/resources.html#ask 
> 
> 

Re: Ignite2.0 and Memory Policies

[Denis Magda]

Hi,

> 1. non-heap memory used increased for both nodes. Does this confirm that
> data went to off-heap memory?

The data gets to off-heap via Java heap. This is why you see heap metrics 
varying over the time.

> 2. I am just doing a cacheStreamer.addData(). If data is going into off-heap
> memory, why is heap memory varying?

The same explanation as above.

> 3. MemoryMetrics is actually not showing any results -

Make sure metrics collection is enabled. 
MemoryPolicyConfiguration.metricsEnabled parameter has to be set to ‘true’
https://apacheignite.readme.io/docs/memory-and-cache-metrics#section-memory-metrics

> 4. I captured CacheMetrics for the clustergroup as well. Even they are not
> showing any data -

Please check you set CacheConfiguration.setStatisticEnabled to ‘true’
https://apacheignite.readme.io/docs/memory-and-cache-metrics#cache-metrics

—
Denis

> On May 9, 2017, at 1:14 PM, Swetad90  wrote:
> 
> Hi,
> I tried capturing metrics for memory policies as well as from cache & nodes.
> Here is what I observed.
> 
> Before Updating Cache Node statistics -
> |Heap memory initialized  | 1gb   
>  
> |
> | Heap memory used| 139mb 
>  
> |
> | Heap memory committed   | 982mb 
>  
> |
> | Heap memory maximum | 982mb 
>  
> |
> | Non-heap memory initialized | 2mb   
>  
> |
> | Non-heap memory used| 47mb  
>  
> |
> | Non-heap memory committed   | 48mb  
>  
> |
> | Non-heap memory maximum | 1gb   
>  
> |
> 
> 
> | Heap memory initialized | 1gb   
>  
> |
> | Heap memory used| 135mb 
>  
> |
> | Heap memory committed   | 982mb 
>  
> |
> | Heap memory maximum | 982mb 
>  
> |
> | Non-heap memory initialized | 2mb   
>  
> |
> | Non-heap memory used| 45mb  
>  
> |
> | Non-heap memory committed   | 46mb  
>  
> |
> | Non-heap memory maximum | 1gb   
>  
> |
> 
> After updating cache-
> | Heap memory initialized | 1gb   
>  
> |
> | Heap memory used| 58mb  
>  
> |
> | Heap memory committed   | 982mb 
>  
> |
> | Heap memory maximum | 982mb 
>  
> |
> | Non-heap memory initialized | 2mb   
>  
> |
> | Non-heap memory used| 55mb  
>  
> |
> | Non-heap memory committed   | 56mb  
>  
> |
> | Non-heap memory maximum | 1gb   
>  
> |
> 
> | Heap memory initialized | 1gb   
>  
> |
> | Heap memory used| 235mb 
>  
> |
> | Heap memory committed   | 982mb 
>  
> |
> | Heap memory maximum | 982mb 
>  
> |
> | Non-heap memory initialized | 2mb   
>  
> |
> | Non-heap memory used| 53mb  
>  
> |
> | Non-heap memory committed   | 54mb  
>  
> |
> | Non-heap memory maximum | 1gb   
>  
> |
> 
> 
> 1. non-heap memory used increased for both nodes. Does this confirm that
> data went to off-heap memory?
> 
> 2. I am just doing a cacheStreamer.addData(). If data is going into off-heap
> memory, why is heap memory varying?
> 
> 3. MemoryMetrics is actually not showing any results -
 Memory Region Name: 40MB_Region_Eviction
 Allocation Rate: 0.0
 Fill Factor: -1.0
 EvictionRate: 0.0
 TotalAllocatedPage: 0
 Memory Region Name: Default_Region
 Allocation Rate: 0.0
 Fill Factor: -1.0
 EvictionRate: 0.0
 TotalAllocatedPage: 0
> 
> 4. I captured CacheMetrics for the clustergroup as well. Even they are not
> showing any data -
> 
 Cache Metrics: CacheMetricsSnapshot [reads=0, puts=0, hits=0, misses=0,
 txCommits=0, txRollbacks=0, evicts=0, removes=0, putAvgTimeNanos=0.0,
 getAvgTimeNanos=0.0, rmvAvgTimeNanos=0.0, commitAvgTimeNanos=0.0,
 rollbackAvgTimeNanos=0.0, cacheName=firstCache40MBP, offHeapGets=0,
 offHeapPuts=0, 

Re: Sizing in Ignite

[Denis Magda]

Hi,

Honestly, it’s unclear why you use multipliers like 2.5 and 1.3 in the formula. 

Please refer to this capacity guide to make up a rough estimation:
https://apacheignite.readme.io/docs/capacity-planning 


—
Denis

> On May 10, 2017, at 8:35 AM, Guillermo Ortiz  wrote:
> 
> What's the reason because it needs so many space Ignite to store data in 
> memory?. 
> For example, if your dataset it's about 4TB and you are going to use backup 
> (1 replica) the final size it's about
> 
> 5TBx2.5x1.3(indices) x backcup = 26TB more or less... So, it could be 28TB 
> with the memory of the system to work. 
> 
> It seems insane to have that memory in many cases. How does Ignite store data 
> to spend 3 times the size on disk? is it possible to reduce this?

Re: Pessimistic TXN did not release lock on a key, all subsequent txns failed

[bintisepaha]

Hey guys, we had a key lock issue again on 1.7.0. here is a suspicious thread
dump. Is this helpful for tracking down our issue further?
we did not see any topology changes or any other exceptions.

Attaching the entire thread dump too tdump.zip
  

"pub-#7%DataGridServer-Production%" Id=47 in WAITING on
lock=org.apache.ignite.internal.util.future.GridFutureAdapter$ChainFuture@2094df59
  at sun.misc.Unsafe.park(Native Method)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
  at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
  at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
  at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:159)
  at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:117)
  at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4800)
  at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4783)
  at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1395)
  at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:956)
  at
com.tudor.datagridI.server.tradegen.OrderHolderSaveRunnable.updatePosition(OrderHolderSaveRunnable.java:790)
  at
com.tudor.datagridI.server.tradegen.OrderHolderSaveRunnable.cancelPosition(OrderHolderSaveRunnable.java:805)
  at
com.tudor.datagridI.server.tradegen.OrderHolderSaveRunnable.cancelExistingTradeOrderForPositionUpdate(OrderHolderSaveRunnable.java:756)
  at
com.tudor.datagridI.server.tradegen.OrderHolderSaveRunnable.processOrderHolders(OrderHolderSaveRunnable.java:356)
  at
com.tudor.datagridI.server.tradegen.OrderHolderSaveRunnable.run(OrderHolderSaveRunnable.java:109)
  at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4V2.execute(GridClosureProcessor.java:2184)
  at
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:509)
  at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6521)
  at
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:503)
  at
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:456)
  at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
  at
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1161)
  at
org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1766)
  at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1238)
  at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:866)
  at
org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:106)
  at
org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:829)
  at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)

  Locked synchronizers: count = 1
 <1237e0be>  - java.util.concurrent.ThreadPoolExecutor$Worker@1237e0be





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Pessimistic-TXN-did-not-release-lock-on-a-key-all-subsequent-txns-failed-tp10536p12611.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: How to monitor and alert for server counts

[ignite_user2016]

we did not made any tweak to memory configuration and our configuration is
a bare bone configuration.

Please note that our need with ignite is not big data.

On Wed, May 10, 2017 at 2:08 AM, tysli2016 [via Apache Ignite Users] <
ml+s70518n12582...@n6.nabble.com> wrote:

> we run the visor in 10 every mins, the Ignite servers would OOME in a
> couple weeks.
> Ignite server allocated 1g memory.
>
> would you mind to share you server's config.xml? any JVM parameters
> changes?
>
> thanks
> Tom
>
> ignite_user2016 wrote
> yes you are correct, we run visor in the batch mode.
>
> we have 2 host hosting 2 client and 2 server for ignite, our conf is bare
> minimal 4 cores 8 GB RAM.
>
> Now with OOME, how often you run the visor command ? we run it every 5
> mins.should you try that ?
>
> On Mon, May 8, 2017 at 9:01 PM, tysli2016 [via Apache Ignite Users] <
> [hidden email] >
> wrote:
>
> > thanks Rishi, can you share more about that?
> > what's the version of Ignite? how many Ignite servers? how many
> > CPU/memory?
> > are you using the Visor in batch mode (https://apacheignite-tools.
> > readme.io/v1.9/docs/batch-mode)?
> > or Visor alert?
> >
> > I have tried Visor batch mode, but it lead to OOME eventually (
> > http://apache-ignite-users.70518.x6.nabble.com/OOME-on-2-
> > node-cluster-with-visor-running-repeatedly-Ignite-1-9-td12409.html).
> >
> > --
> > If you reply to this email, your message will be added to the discussion
> > below:
> > http://apache-ignite-users.70518.x6.nabble.com/How-to-
> > monitor-and-alert-for-server-counts-tp12533p12557.html
> > To start a new topic under Apache Ignite Users, email
> > [hidden email] 
> > To unsubscribe from Apache Ignite Users, click here
> >  NamlServlet.jtp?macro=unsubscribe_by_code=1=
> cmlzaGl5YWduaWtAZ21haWwuY29tfDF8MTMwNTI4OTg1Mw==>
> > .
> > NAML
> >  NamlServlet.jtp?macro=macro_viewer=instant_html%
> 21nabble%3Aemail.naml=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> nabble.view.web.template.NodeNamespace=
> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> 
> >
>
>
>
> --
> Rishi Yagnik
>
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/How-to-
> monitor-and-alert-for-server-counts-tp12533p12582.html
> To start a new topic under Apache Ignite Users, email
> ml+s70518n1...@n6.nabble.com
> To unsubscribe from Apache Ignite Users, click here
> 
> .
> NAML
> 
>



-- 
Rishi Yagnik




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/How-to-monitor-and-alert-for-server-counts-tp12533p12610.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Closures stuck in 2.0 when try to add an element into the queue.

[fatality]

Hi 

I have just subscribed to the user group also same question is posted at
http://stackoverflow.com/questions/43891757/closures-stuck-in-2-0-when-try-to-add-an-element-into-the-queue



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Closures-stuck-in-2-0-when-try-to-add-an-element-into-the-queue-tp12587p12609.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: GroupBy with index is really slow.

[Guillermo Ortiz]

Yes,

**Result SELECT distinct(age) FROM PERSONWITHINDEX
97,98,99,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,
20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,
40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,
60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,
80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,
Elapsed time SELECT distinct(age) FROM PERSONWITHINDEX:2880ms
[SELECT DISTINCT
__Z0.AGE AS __C0_0
FROM "PersonWithIndex".PERSONWITHINDEX __Z0
/* "PersonWithIndex".PERSONWITHINDEX_AGE_IDX */]
[SELECT DISTINCT
__C0_0 AS AGE
FROM PUBLIC.__T0
/* "PersonWithIndex"."merge_scan" */]



**Result SELECT distinct(age) FROM PERSONWITHOUTINDEX
97,98,99,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,
20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,
40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,
60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,
80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,
Elapsed time SELECT distinct(age) FROM PERSONWITHOUTINDEX:2976ms
[SELECT DISTINCT
__Z0.AGE AS __C0_0
FROM "PersonWithoutIndex".PERSONWITHOUTINDEX __Z0
/* "PersonWithoutIndex".PERSONWITHOUTINDEX.__SCAN_ */]
[SELECT DISTINCT
__C0_0 AS AGE
FROM PUBLIC.__T0
/* "PersonWithoutIndex"."merge_scan" */]

Age is a value between 0-100. I tried with a value between 0-1000 with the
same result.
It seems that it's not using any index.

I'm not sure if in memory a random access is slower than a sequencial
access. I guess, but I'm not sure.


2017-05-10 19:50 GMT+02:00 Andrey Mashenkov :

> Hi,
>
> I've just think you can achive same result with query without groupBy.
> Select distinct age from ..
>
> Does it workable for you?
>
> 10 мая 2017 г. 17:56 пользователь "Guillermo Ortiz" 
> написал:
>
> It's my laptop. 16gb, i7.. The collection size it's 5Mill objects (1,2GB)
>> and it takes about 30sec.
>> I have tried to execute the query with the same collection with and
>> without indices with the same time results.
>>
>> This is a test before to run in the real cluster with 6 nodes of 512Gb
>> and 48cores each one. We tried to execute there with a real dataset(22M
>> rows, 7gb) but it takes so long that query is aborted.
>>
>>
>> 2017-05-08 12:12 GMT+02:00 Andrey Mashenkov :
>>
>>> Hi,
>>>
>>> Why do you think query is slow?
>>> What its execution time? What is expected time?
>>> How many nodes do you use? How much data does cache contains?
>>>
>>> On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz 
>>> wrote:
>>>
 I have a simple cache where I have Person object (id, age and so on..)

 I have created indeces by id and age to try some examples but queries
 with "group by" go really slow.

 I'm trying this query:
 SELECT age
 FROM
 PersonWithindex
 group by age


 SELECT
 AGE AS __C0
 FROM "personCacheWithIndex".PERSONWITHINDEX
 /* "personCacheWithIndex"."age_idx" */
 GROUP BY AGE
 /* group sorted */

 SELECT
 __C0 AS AGE
 FROM PUBLIC.__T0
 /* "personCacheWithIndex"."merge_scan" */
 GROUP BY __C0

 Although it seems that it uses index, why is it going so slow? I think
 that it should be pretty fast with an index.

>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>

Re: GroupBy with index is really slow.

[Andrey Mashenkov]

Hi,

I've just think you can achive same result with query without groupBy.
Select distinct age from ..

Does it workable for you?

10 мая 2017 г. 17:56 пользователь "Guillermo Ortiz" 
написал:

> It's my laptop. 16gb, i7.. The collection size it's 5Mill objects (1,2GB)
> and it takes about 30sec.
> I have tried to execute the query with the same collection with and
> without indices with the same time results.
>
> This is a test before to run in the real cluster with 6 nodes of 512Gb and
> 48cores each one. We tried to execute there with a real dataset(22M rows,
> 7gb) but it takes so long that query is aborted.
>
>
> 2017-05-08 12:12 GMT+02:00 Andrey Mashenkov :
>
>> Hi,
>>
>> Why do you think query is slow?
>> What its execution time? What is expected time?
>> How many nodes do you use? How much data does cache contains?
>>
>> On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz 
>> wrote:
>>
>>> I have a simple cache where I have Person object (id, age and so on..)
>>>
>>> I have created indeces by id and age to try some examples but queries
>>> with "group by" go really slow.
>>>
>>> I'm trying this query:
>>> SELECT age
>>> FROM
>>> PersonWithindex
>>> group by age
>>>
>>>
>>> SELECT
>>> AGE AS __C0
>>> FROM "personCacheWithIndex".PERSONWITHINDEX
>>> /* "personCacheWithIndex"."age_idx" */
>>> GROUP BY AGE
>>> /* group sorted */
>>>
>>> SELECT
>>> __C0 AS AGE
>>> FROM PUBLIC.__T0
>>> /* "personCacheWithIndex"."merge_scan" */
>>> GROUP BY __C0
>>>
>>> Although it seems that it uses index, why is it going so slow? I think
>>> that it should be pretty fast with an index.
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>

Re: Docker cluster set up

[afedotov]

Also, setting --net=host when running both of containers may probably help
too.

Kind regards,
Alex.

On Wed, May 10, 2017 at 8:28 PM, waterg [via Apache Ignite Users] <
ml+s70518n12605...@n6.nabble.com> wrote:

> Thank you Alex!
>
> On Wed, May 10, 2017 at 1:54 AM, afedotov <[hidden email]
> > wrote:
>
>> Hi.
>> Did I get it right that containers reside on different hosts? If it's so
>> then containers won't see each other out of box. You probably need to start
>> them via docker overlay network or use Swarm.
>>
>> Kind regards,
>> Alex
>>
>> 9 мая 2017 г. 12:29 AM пользователь "waterg [via Apache Ignite Users]" 
>> <[hidden
>> email] > написал:
>>
>> All,
>>
>> I'm trying to set up a cluster using docker. The network setting seems to
>> have some problem.
>> Could you please take a look and advise?
>> I have two hosts and each will run a container. Static ip address and
>> basic address resolver are used.
>> This is part of configuration on one container.
>>
>> 
>> 
>>
>> 
>>
>>
>> 
>> 
>> 
>>
>>
>> host1 public ip
>>
>> host2 public ip
>> 
>> 
>> 
>> 
>> 
>> 
>>
>>  
>>  
>>  
>>  
>>  
>>  > value="container 1 private ip"/>
>>  
>>  
>>  
>>  
>>
>> The other container has the same configuration under
>> TcpDiscoveryVmIpFinder, but the BasicAddressResolver has ip address for
>> host2/container2.
>>
>> when I run the container, I use command "sudo docker run -p 47100:47100
>> -p 47500:47500 imagename"
>>
>> Here's the result of docker ps on two hosts.
>>
>> $ sudo docker ps
>> CONTAINER IDIMAGE   COMMAND
>>  CREATED STATUS
>> PORTS
>>  NAMES
>> 3e1e86d18b43imagename   "/bin/sh -c $IGNITE_H"   27 seconds ago
>>Up 26seconds   
>> 0.0.0.0:47100->47100/tcp,
>> 11211/tcp, 49112/tcp, 0.0.0.0:47500->47500/tcp   nostalgic_
>>   noyce
>>
>>
>> $ sudo docker ps
>> CONTAINER IDIMAGE   COMMAND
>>  CREATED STATUS  PORTS
>>  NAMES
>> 5bbd182dca5cimagename   "/bin/sh -c $IGNITE_H"   19 minutes ago
>>Up 19 minutes   0.0.0.0:47100->47100/tcp, 11211/tcp, 49112/tcp,
>> 0.0.0.0:47500->47500/tcp   romantic_galileo
>>
>>
>> Does the settings look right? Thank you!
>>
>>
>> --
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-ignite-users.70518.x6.nabble.com/Docker-cluste
>> r-set-up-tp12549.html
>> To start a new topic under Apache Ignite Users, email [hidden email]
>> 
>> To unsubscribe from Apache Ignite Users, click here.
>> NAML
>> 
>>
>>
>>
>> --
>> View this message in context: Re: Docker cluster set up
>> 
>>
>> Sent from the Apache Ignite Users mailing list archive
>>  at Nabble.com.
>>
>
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Docker-
> cluster-set-up-tp12549p12605.html
> To start a new topic under Apache Ignite Users, email
> ml+s70518n1...@n6.nabble.com
> To unsubscribe from Apache Ignite Users, click here
> 
> .
> NAML
> 
>




--
View this message in context: 

Re: Docker cluster set up

[Jessie Lin]

Thank you Alex!

On Wed, May 10, 2017 at 1:54 AM, afedotov 
wrote:

> Hi.
> Did I get it right that containers reside on different hosts? If it's so
> then containers won't see each other out of box. You probably need to start
> them via docker overlay network or use Swarm.
>
> Kind regards,
> Alex
>
> 9 мая 2017 г. 12:29 AM пользователь "waterg [via Apache Ignite Users]" 
> <[hidden
> email] > написал:
>
> All,
>
> I'm trying to set up a cluster using docker. The network setting seems to
> have some problem.
> Could you please take a look and advise?
> I have two hosts and each will run a container. Static ip address and
> basic address resolver are used.
> This is part of configuration on one container.
>
> 
> 
>
> 
>
>
> 
> 
> 
>
>
> host1 public ip
>
> host2 public ip
> 
> 
> 
> 
> 
> 
>
>  
>  
>  
>  
>  
>   value="container 1 private ip"/>
>  
>  
>  
>  
>
> The other container has the same configuration under
> TcpDiscoveryVmIpFinder, but the BasicAddressResolver has ip address for
> host2/container2.
>
> when I run the container, I use command "sudo docker run -p 47100:47100 -p
> 47500:47500 imagename"
>
> Here's the result of docker ps on two hosts.
>
> $ sudo docker ps
> CONTAINER IDIMAGE   COMMAND
>  CREATED STATUS
> PORTS
>  NAMES
> 3e1e86d18b43imagename   "/bin/sh -c $IGNITE_H"   27 seconds ago
>Up 26seconds   
> 0.0.0.0:47100->47100/tcp,
> 11211/tcp, 49112/tcp, 0.0.0.0:47500->47500/tcp   nostalgic_
> noyce
>
>
> $ sudo docker ps
> CONTAINER IDIMAGE   COMMAND
>  CREATED STATUS  PORTS
>  NAMES
> 5bbd182dca5cimagename   "/bin/sh -c $IGNITE_H"   19 minutes ago
>Up 19 minutes   0.0.0.0:47100->47100/tcp, 11211/tcp, 49112/tcp,
> 0.0.0.0:47500->47500/tcp   romantic_galileo
>
>
> Does the settings look right? Thank you!
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Docker-cluste
> r-set-up-tp12549.html
> To start a new topic under Apache Ignite Users, email [hidden email]
> 
> To unsubscribe from Apache Ignite Users, click here.
> NAML
> 
>
>
>
> --
> View this message in context: Re: Docker cluster set up
> 
>
> Sent from the Apache Ignite Users mailing list archive
>  at Nabble.com.
>

Re: Write Behind with delete performance

[waterg]

Thank you for the quick explaining and creating jira tickets. 

Just a thought:
If insert k1 =0, k2=2, k3 =3 and delete k1=0 and k1 is not being flushed to
store yet, would it be possible to just delete in memory, w/o flushing to
store? and deleteAll for all k1 like records that have been flushed to
store?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Write-Behind-with-delete-performance-tp12580p12604.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Closures stuck in 2.0 when try to add an element into the queue.

[afedotov]

Hi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.

Kind regards,
Alex.

On Wed, May 10, 2017 at 2:30 PM, fatality [via Apache Ignite Users] <
ml+s70518n12587...@n6.nabble.com> wrote:

> Hi
>
> We have a use case as below
>
>
> 1-) Start 2 ignite instance as data nodes and insert data into the cache.
> 2-)Create a queue and register a remote listener by using remoteListen as
> below
>
> //Queue creation
> CollectionConfiguration colCfg = new CollectionConfiguration();
> colCfg.setCacheMode(PARTITIONED);
> IgniteQueue queue = Ignition.ignite().queue(queueName, 0,
> colCfg);
>
> //Remote Listener Closure
> IgnitePredicate rmtLsnr = new IgnitePredicate() {
> @Override public boolean apply(CacheEvent evt) {
> System.out.println("Cache event [name=" +
> evt.name() + ", key=" + evt.key() + ']');
> Ignite ignite = Ignition.ignite();
> IgniteQueue queue = ignite.queue(queueName, 0,
> null);
> String key = evt.key();
> BinaryObject profile = (BinaryObject) evt.newValue();
> System.out.println("Received event [evt=" + evt.name() +
> ", key=" + evt.key() +
> ", oldVal=" + evt.oldValue().toString() + ",
> newVal=" + evt.newValue().toString());
>
> if (profile.field("usage") > start &&
> profile.field("usage") < end
> && ignite.affinity("profileCache"
> ).isPrimary(ignite.cluster().localNode(), key)){
> queue.add(profile.field("number"));
> }
> return false;
> }
> };
>
>
> Ignition.ignite().events(ignite.cluster().forCacheNodes("profileCache")).remoteListen(1,1l,false,null,
> rmtLsnr,
> EVT_CACHE_OBJECT_PUT,
> EVT_CACHE_OBJECT_REMOVED);
>
>
> 3-)Do some updates in the cache instances as below to get updates into the
> remotelistener.
>
> void updateAnyProfile(Double newDataUsage){
> SqlQuery qry = new SqlQuery(VeonProfile.class,"select * from
> VeonProfile where dataUsage < 30 limit 10");
> List> res =
> veonProfileCache.query(qry).getAll();
> VeonProfile profile = res.iterator().next().getValue();
> profile.setDataUsage(newDataUsage);
> veonProfileCache.put(profile.getCtn(), profile);
> profile.setDataUsage(newDataUsage+1);
> veonProfileCache.put(profile.getCtn(), profile);
>
> }
>
>
> 4-) Take the elements from the queue.
>
> public void readFromQueue (String queueName) {
> // Initialize new FIFO queue.
> IgniteQueue queue = Ignition.ignite().queue(queueName, 0,
> null);
> while (true) {
> String profile = queue.take();
> System.out.println("Profile from queue: " +
> profile.toString());
> }
> }
>
> The problem is application hangs to do any operation after doing above
> scenario. Could you please help us? We would really appriciate if you could
> tell us what we are doing wrong?
>
> Below is the thread dump of the hanging datanode and same datanode hangs
> at  below code   IgniteQueue queue =
> ignite.queue(queueName, 0, null);
>
>
> "sys-stripe-5-#6%null%" #25 prio=5 os_prio=31 tid=0x7fd88d031800
> nid=0x14c07 waiting on condition [0x736e7000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
>
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> get0(GridFutureAdapter.java:176)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> get(GridFutureAdapter.java:139)
> at org.apache.ignite.internal.processors.cache.
> GridCacheAdapter.get0(GridCacheAdapter.java:4482)
> at org.apache.ignite.internal.processors.cache.
> GridCacheAdapter.get(GridCacheAdapter.java:4463)
> at org.apache.ignite.internal.processors.cache.
> GridCacheAdapter.get(GridCacheAdapter.java:1405)
> at org.apache.ignite.internal.processors.cache.datastructures.
> CacheDataStructuresManager.queue0(CacheDataStructuresManager.java:270)
> at org.apache.ignite.internal.processors.cache.datastructures.
> CacheDataStructuresManager.queue(CacheDataStructuresManager.java:231)
> at org.apache.ignite.internal.processors.datastructures.
> DataStructuresProcessor$12.applyx(DataStructuresProcessor.java:952)
> at org.apache.ignite.internal.processors.datastructures.
> DataStructuresProcessor$12.applyx(DataStructuresProcessor.java:950)
> at org.apache.ignite.internal.processors.datastructures.
> 

Re: GroupBy with index is really slow.

[afedotov]

Hi.

I think the problem is that age and month have bad selectivity. For
example, in the case of month, each month index will select in average
200 / 12 ~ 167 rows.
In this case, it's probably faster scan all the rows in storage/disk order
then using the index order.

Kind regards,
Alex.

On Wed, May 10, 2017 at 6:29 PM, Guillermo Ortiz [via Apache Ignite Users] <
ml+s70518n1259...@n6.nabble.com> wrote:

> Are indices used always that it'spossible or it depends how many records
> data you have to read as Oracle? So, if You use and index but the result
> it's to read 25% of the data, usually indices are not used because
> sequencial read are faster. As this is in memory I understand that it
> doesn't apply but I'm not sure and I didn't read anything about it in the
> documentation.
>
> 2017-05-10 17:27 GMT+02:00 Guillermo Ortiz <[hidden email]
> >:
>
>> If I execute many times, time is similar in both.
>>
>> 2017-05-10 17:25 GMT+02:00 Guillermo Ortiz <[hidden email]
>> >:
>>
>>> I'm using 2.0.0.
>>> Right now in my laptop with 2Mill objects.
>>>
>>>
>>> *WITH INDICES*
>>> **Result SELECT distinct(MONTH) FROM PERSONWITHINDEX
>>> September,February,March,August,June,April,July,January,May,
>>> November,October,
>>> Elapsed time SELECT distinct(MONTH) FROM PERSONWITHINDEX:1990ms
>>> [SELECT DISTINCT
>>> __Z0.MONTH AS __C0_0
>>> FROM "PersonWithIndex".PERSONWITHINDEX __Z0
>>> /* "PersonWithIndex".PERSONWITHINDEX_MONTH_IDX */]
>>> [SELECT DISTINCT
>>> __C0_0 AS MONTH
>>> FROM PUBLIC.__T0
>>> /* "PersonWithIndex"."merge_scan" */]
>>>
>>>
>>> *WITHOUT INDICES*
>>> **Result SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX
>>> September,November,February,March,May,April,July,August,Janu
>>> ary,June,October,
>>> Elapsed time SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX:1690ms
>>> [SELECT DISTINCT
>>> __Z0.MONTH AS __C0_0
>>> FROM "PersonWithoutIndex".PERSONWITHOUTINDEX __Z0
>>> /* "PersonWithoutIndex".PERSONWITHOUTINDEX.__SCAN_ */]
>>> [SELECT DISTINCT
>>> __C0_0 AS MONTH
>>> FROM PUBLIC.__T0
>>> /* "PersonWithoutIndex"."merge_scan" */]
>>> ***
>>>
>>> public class PersonWithIndex implements Serializable {
>>> @QuerySqlField(index = true)
>>> public int id;
>>> @QuerySqlField(index = true, orderedGroups = 
>>> {@QuerySqlField.Group(name="index_group_month_age", order=1)})
>>> public int age;
>>> @QuerySqlField
>>> public String dni;
>>> @QuerySqlField(index = true, orderedGroups = 
>>> {@QuerySqlField.Group(name="index_group_month_age", order=0)})
>>> public String month;
>>>
>>>
>>> 2017-05-10 17:06 GMT+02:00 afedotov <[hidden email]
>>> >:
>>>
 Hi,

 Which Ignite version do you use?
 How many Ignite nodes do you run?
 What is your cache configuration?

 As well, could you please try the following query and see the
 performance?
 SELECT DISTINCT age FROM PersonWithindex;


 Kind regards,
 Alex.

 On Wed, May 10, 2017 at 5:57 PM, Guillermo Ortiz [via Apache Ignite
 Users] <[hidden email]
 > wrote:

> It's my laptop. 16gb, i7.. The collection size it's 5Mill objects
> (1,2GB) and it takes about 30sec.
> I have tried to execute the query with the same collection with and
> without indices with the same time results.
>
> This is a test before to run in the real cluster with 6 nodes of 512Gb
> and 48cores each one. We tried to execute there with a real dataset(22M
> rows, 7gb) but it takes so long that query is aborted.
>
>
> 2017-05-08 12:12 GMT+02:00 Andrey Mashenkov <[hidden email]
> >:
>
>> Hi,
>>
>> Why do you think query is slow?
>> What its execution time? What is expected time?
>> How many nodes do you use? How much data does cache contains?
>>
>> On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz <[hidden email]
>> > wrote:
>>
>>> I have a simple cache where I have Person object (id, age and so
>>> on..)
>>>
>>> I have created indeces by id and age to try some examples but
>>> queries with "group by" go really slow.
>>>
>>> I'm trying this query:
>>> SELECT age
>>> FROM
>>> PersonWithindex
>>> group by age
>>>
>>>
>>> SELECT
>>> AGE AS __C0
>>> FROM "personCacheWithIndex".PERSONWITHINDEX
>>> /* "personCacheWithIndex"."age_idx" */
>>> GROUP BY AGE
>>> /* group sorted */
>>>
>>> SELECT
>>> __C0 AS AGE
>>> FROM PUBLIC.__T0
>>> /* "personCacheWithIndex"."merge_scan" */
>>> GROUP BY __C0
>>>
>>> Although it seems that it uses index, why is it going so slow? I
>>> 

Re: vertx-ignite

[Andrey Gura]

Anil,

What version of vertx-ignite or Ignite itself do you use?

In provided ignite.xml there is no minimal configuration that is
mandatory for Ignite cluster manager for vert.x (see
default-ignite.xml for example).


On Tue, May 2, 2017 at 9:18 AM, Anil  wrote:
>
> Hi Andrey,
>
> Apologies for late reply. I don't have any exact reproduce. I can see this
> log frequently in our logs.
>
> attached the ignite.xml.
>
> Thanks.
>
>
>
> On 26 April 2017 at 18:32, Andrey Gura  wrote:
>>
>> Anil,
>>
>> what kind of lock do you mean? What are steps for reproduce? What
>> version if vert-ignite do use and what is your configuration?
>>
>> On Wed, Apr 26, 2017 at 2:16 PM, Anil  wrote:
>> > HI,
>> >
>> > I am using vertx-ignite and when node is left the topology, lock is not
>> > getting released and whole server is not responding.
>> >
>> > 2017-04-26 04:09:15 WARN  vertx-blocked-thread-checker
>> > BlockedThreadChecker:57 - Thread
>> > Thread[vert.x-worker-thread-82,5,ignite]
>> > has been blocked for 2329981 ms, time limit is 6
>> > io.vertx.core.VertxException: Thread blocked
>> > at sun.misc.Unsafe.park(Native Method)
>> > at
>> > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>> > at
>> >
>> > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>> > at
>> >
>> > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
>> > at
>> >
>> > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
>> > at
>> >
>> > org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:161)
>> > at
>> >
>> > org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:119)
>> > at
>> >
>> > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.get0(GridDhtAtomicCache.java:488)
>> > at
>> >
>> > org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4663)
>> > at
>> >
>> > org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1388)
>> > at
>> >
>> > org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:1117)
>> > at io.vertx.spi.cluster.ignite.impl.MapImpl.get(MapImpl.java:81)
>> > at
>> > io.vertx.core.impl.HAManager.chooseHashedNode(HAManager.java:590)
>> > at io.vertx.core.impl.HAManager.checkSubs(HAManager.java:519)
>> > at io.vertx.core.impl.HAManager.nodeLeft(HAManager.java:305)
>> > at io.vertx.core.impl.HAManager.access$100(HAManager.java:107)
>> > at io.vertx.core.impl.HAManager$1.nodeLeft(HAManager.java:157)
>> > at
>> >
>> > io.vertx.spi.cluster.ignite.IgniteClusterManager.lambda$null$4(IgniteClusterManager.java:254)
>> > at
>> >
>> > io.vertx.spi.cluster.ignite.IgniteClusterManager$$Lambda$36/837728834.handle(Unknown
>> > Source)
>> > at
>> >
>> > io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:271)
>> > at
>> > io.vertx.core.impl.ContextImpl$$Lambda$13/116289363.run(Unknown
>> > Source)
>> > at io.vertx.core.impl.TaskQueue.lambda$new$0(TaskQueue.java:60)
>> > at io.vertx.core.impl.TaskQueue$$Lambda$12/443290224.run(Unknown
>> > Source)
>> > at
>> >
>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
>> >
>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> >
>> > was it a known issue ?
>> >
>> > Thanks
>
>

Re: GroupBy with index is really slow.

[Guillermo Ortiz]

Are indices used always that it'spossible or it depends how many records
data you have to read as Oracle? So, if You use and index but the result
it's to read 25% of the data, usually indices are not used because
sequencial read are faster. As this is in memory I understand that it
doesn't apply but I'm not sure and I didn't read anything about it in the
documentation.

2017-05-10 17:27 GMT+02:00 Guillermo Ortiz :

> If I execute many times, time is similar in both.
>
> 2017-05-10 17:25 GMT+02:00 Guillermo Ortiz :
>
>> I'm using 2.0.0.
>> Right now in my laptop with 2Mill objects.
>>
>>
>> *WITH INDICES*
>> **Result SELECT distinct(MONTH) FROM PERSONWITHINDEX
>> September,February,March,August,June,April,July,January,May,
>> November,October,
>> Elapsed time SELECT distinct(MONTH) FROM PERSONWITHINDEX:1990ms
>> [SELECT DISTINCT
>> __Z0.MONTH AS __C0_0
>> FROM "PersonWithIndex".PERSONWITHINDEX __Z0
>> /* "PersonWithIndex".PERSONWITHINDEX_MONTH_IDX */]
>> [SELECT DISTINCT
>> __C0_0 AS MONTH
>> FROM PUBLIC.__T0
>> /* "PersonWithIndex"."merge_scan" */]
>>
>>
>> *WITHOUT INDICES*
>> **Result SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX
>> September,November,February,March,May,April,July,August,Janu
>> ary,June,October,
>> Elapsed time SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX:1690ms
>> [SELECT DISTINCT
>> __Z0.MONTH AS __C0_0
>> FROM "PersonWithoutIndex".PERSONWITHOUTINDEX __Z0
>> /* "PersonWithoutIndex".PERSONWITHOUTINDEX.__SCAN_ */]
>> [SELECT DISTINCT
>> __C0_0 AS MONTH
>> FROM PUBLIC.__T0
>> /* "PersonWithoutIndex"."merge_scan" */]
>> ***
>>
>> public class PersonWithIndex implements Serializable {
>> @QuerySqlField(index = true)
>> public int id;
>> @QuerySqlField(index = true, orderedGroups = 
>> {@QuerySqlField.Group(name="index_group_month_age", order=1)})
>> public int age;
>> @QuerySqlField
>> public String dni;
>> @QuerySqlField(index = true, orderedGroups = 
>> {@QuerySqlField.Group(name="index_group_month_age", order=0)})
>> public String month;
>>
>>
>> 2017-05-10 17:06 GMT+02:00 afedotov :
>>
>>> Hi,
>>>
>>> Which Ignite version do you use?
>>> How many Ignite nodes do you run?
>>> What is your cache configuration?
>>>
>>> As well, could you please try the following query and see the
>>> performance?
>>> SELECT DISTINCT age FROM PersonWithindex;
>>>
>>>
>>> Kind regards,
>>> Alex.
>>>
>>> On Wed, May 10, 2017 at 5:57 PM, Guillermo Ortiz [via Apache Ignite
>>> Users] <[hidden email]
>>> > wrote:
>>>
 It's my laptop. 16gb, i7.. The collection size it's 5Mill objects
 (1,2GB) and it takes about 30sec.
 I have tried to execute the query with the same collection with and
 without indices with the same time results.

 This is a test before to run in the real cluster with 6 nodes of 512Gb
 and 48cores each one. We tried to execute there with a real dataset(22M
 rows, 7gb) but it takes so long that query is aborted.


 2017-05-08 12:12 GMT+02:00 Andrey Mashenkov <[hidden email]
 >:

> Hi,
>
> Why do you think query is slow?
> What its execution time? What is expected time?
> How many nodes do you use? How much data does cache contains?
>
> On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz <[hidden email]
> > wrote:
>
>> I have a simple cache where I have Person object (id, age and so on..)
>>
>> I have created indeces by id and age to try some examples but queries
>> with "group by" go really slow.
>>
>> I'm trying this query:
>> SELECT age
>> FROM
>> PersonWithindex
>> group by age
>>
>>
>> SELECT
>> AGE AS __C0
>> FROM "personCacheWithIndex".PERSONWITHINDEX
>> /* "personCacheWithIndex"."age_idx" */
>> GROUP BY AGE
>> /* group sorted */
>>
>> SELECT
>> __C0 AS AGE
>> FROM PUBLIC.__T0
>> /* "personCacheWithIndex"."merge_scan" */
>> GROUP BY __C0
>>
>> Although it seems that it uses index, why is it going so slow? I
>> think that it should be pretty fast with an index.
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>



 --
 If you reply to this email, your message will be added to the
 discussion below:
 http://apache-ignite-users.70518.x6.nabble.com/GroupBy-with-
 index-is-really-slow-tp12526p12592.html
 To start a new topic under Apache Ignite Users, email [hidden email]
 
 To unsubscribe from Apache Ignite Users, click here.
 NAML
 

Re: [ANNOUNCE] Apache Ignite 2.0.0 Released

[Pavel Tupitsyn]

Ignite.NET 2.0 blog post:
https://ptupitsyn.github.io/Whats-New-In-Ignite-Net-2.0/

On Wed, May 10, 2017 at 12:44 PM, Pavel Tupitsyn 
wrote:

> Great news!
>
> Thanks Denis, I've done some minor corrections to .NET part of release
> notes page (IGNITE-4495 does not belong there).
>
> On Fri, May 5, 2017 at 11:13 PM, Denis Magda  wrote:
>
>> The Apache Ignite Community is pleased to announce the release of Apache
>> Ignite 2.0.0.
>>
>> Apache Ignite In-Memory Data Fabric [1] is a high-performance, integrated
>> and distributed in-memory platform for computing and transacting on
>> large-scale data sets in real-time, orders of magnitude faster than
>> possible with traditional disk-based or flash-based technologies.
>>
>> The Fabric is a collection of independent and well integrated components
>> some of which are the following:
>> - Data Grid
>> - SQL Grid
>> - Compute Grid
>> - Service Grid
>> - Machine Learning Grid (NEW!)
>>
>> This release incorporates tremendous changes that shifted Ignite 2.0 to
>> the next level. The whole off-heap memory architecture was redesigned from
>> scratch and, going forward, Ignite can be easily integrated with Flash and
>> SSD drives (stay tuned!). SQL Grid has been enriched with Data Definition
>> Language support. A completely new component was added to the fabric -
>> Machine Learning Grid. We provided integrations with RocketMQ and Spring
>> Data, updated Hibernate integration, introduced a plugin system for .NET
>> and found out how to execute C++ code on remote node, and many more...
>>
>> Get more details from this blog post: https://blogs.apache.org/ignit
>> e/entry/apache-ignite-2-0-redesigned
>>
>> The full list of the changes can be found here [2].
>>
>> Please visit this page if you’re ready to try the release out:
>> https://ignite.apache.org/download.cgi
>>
>> Please let us know [3] if you encounter any problems.
>>
>> Regards,
>>
>> The Apache Ignite Community
>>
>> [1] https://ignite.apache.org
>> [2] https://ignite.apache.org/releases/2.0.0/release_notes.html
>> [3] https://ignite.apache.org/community/resources.html#ask
>
>
>

Re: GroupBy with index is really slow.

[Guillermo Ortiz]

If I execute many times, time is similar in both.

2017-05-10 17:25 GMT+02:00 Guillermo Ortiz :

> I'm using 2.0.0.
> Right now in my laptop with 2Mill objects.
>
>
> *WITH INDICES*
> **Result SELECT distinct(MONTH) FROM PERSONWITHINDEX
> September,February,March,August,June,April,July,
> January,May,November,October,
> Elapsed time SELECT distinct(MONTH) FROM PERSONWITHINDEX:1990ms
> [SELECT DISTINCT
> __Z0.MONTH AS __C0_0
> FROM "PersonWithIndex".PERSONWITHINDEX __Z0
> /* "PersonWithIndex".PERSONWITHINDEX_MONTH_IDX */]
> [SELECT DISTINCT
> __C0_0 AS MONTH
> FROM PUBLIC.__T0
> /* "PersonWithIndex"."merge_scan" */]
>
>
> *WITHOUT INDICES*
> **Result SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX
> September,November,February,March,May,April,July,August,
> January,June,October,
> Elapsed time SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX:1690ms
> [SELECT DISTINCT
> __Z0.MONTH AS __C0_0
> FROM "PersonWithoutIndex".PERSONWITHOUTINDEX __Z0
> /* "PersonWithoutIndex".PERSONWITHOUTINDEX.__SCAN_ */]
> [SELECT DISTINCT
> __C0_0 AS MONTH
> FROM PUBLIC.__T0
> /* "PersonWithoutIndex"."merge_scan" */]
> ***
>
> public class PersonWithIndex implements Serializable {
> @QuerySqlField(index = true)
> public int id;
> @QuerySqlField(index = true, orderedGroups = 
> {@QuerySqlField.Group(name="index_group_month_age", order=1)})
> public int age;
> @QuerySqlField
> public String dni;
> @QuerySqlField(index = true, orderedGroups = 
> {@QuerySqlField.Group(name="index_group_month_age", order=0)})
> public String month;
>
>
> 2017-05-10 17:06 GMT+02:00 afedotov :
>
>> Hi,
>>
>> Which Ignite version do you use?
>> How many Ignite nodes do you run?
>> What is your cache configuration?
>>
>> As well, could you please try the following query and see the performance?
>> SELECT DISTINCT age FROM PersonWithindex;
>>
>>
>> Kind regards,
>> Alex.
>>
>> On Wed, May 10, 2017 at 5:57 PM, Guillermo Ortiz [via Apache Ignite
>> Users] <[hidden email]
>> > wrote:
>>
>>> It's my laptop. 16gb, i7.. The collection size it's 5Mill objects
>>> (1,2GB) and it takes about 30sec.
>>> I have tried to execute the query with the same collection with and
>>> without indices with the same time results.
>>>
>>> This is a test before to run in the real cluster with 6 nodes of 512Gb
>>> and 48cores each one. We tried to execute there with a real dataset(22M
>>> rows, 7gb) but it takes so long that query is aborted.
>>>
>>>
>>> 2017-05-08 12:12 GMT+02:00 Andrey Mashenkov <[hidden email]
>>> >:
>>>
 Hi,

 Why do you think query is slow?
 What its execution time? What is expected time?
 How many nodes do you use? How much data does cache contains?

 On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz <[hidden email]
 > wrote:

> I have a simple cache where I have Person object (id, age and so on..)
>
> I have created indeces by id and age to try some examples but queries
> with "group by" go really slow.
>
> I'm trying this query:
> SELECT age
> FROM
> PersonWithindex
> group by age
>
>
> SELECT
> AGE AS __C0
> FROM "personCacheWithIndex".PERSONWITHINDEX
> /* "personCacheWithIndex"."age_idx" */
> GROUP BY AGE
> /* group sorted */
>
> SELECT
> __C0 AS AGE
> FROM PUBLIC.__T0
> /* "personCacheWithIndex"."merge_scan" */
> GROUP BY __C0
>
> Although it seems that it uses index, why is it going so slow? I think
> that it should be pretty fast with an index.
>



 --
 Best regards,
 Andrey V. Mashenkov

>>>
>>>
>>>
>>> --
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>> http://apache-ignite-users.70518.x6.nabble.com/GroupBy-with-
>>> index-is-really-slow-tp12526p12592.html
>>> To start a new topic under Apache Ignite Users, email [hidden email]
>>> 
>>> To unsubscribe from Apache Ignite Users, click here.
>>> NAML
>>> 
>>>
>>
>>
>> --
>> View this message in context: Re: GroupBy with index is really slow.
>> 
>> Sent from the Apache Ignite Users mailing list archive
>> 

Re: GroupBy with index is really slow.

[Guillermo Ortiz]

I'm using 2.0.0.
Right now in my laptop with 2Mill objects.


*WITH INDICES*
**Result SELECT distinct(MONTH) FROM PERSONWITHINDEX
September,February,March,August,June,April,July,January,May,November,October,
Elapsed time SELECT distinct(MONTH) FROM PERSONWITHINDEX:1990ms
[SELECT DISTINCT
__Z0.MONTH AS __C0_0
FROM "PersonWithIndex".PERSONWITHINDEX __Z0
/* "PersonWithIndex".PERSONWITHINDEX_MONTH_IDX */]
[SELECT DISTINCT
__C0_0 AS MONTH
FROM PUBLIC.__T0
/* "PersonWithIndex"."merge_scan" */]


*WITHOUT INDICES*
**Result SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX
September,November,February,March,May,April,July,August,January,June,October,
Elapsed time SELECT distinct(MONTH) FROM PERSONWITHOUTINDEX:1690ms
[SELECT DISTINCT
__Z0.MONTH AS __C0_0
FROM "PersonWithoutIndex".PERSONWITHOUTINDEX __Z0
/* "PersonWithoutIndex".PERSONWITHOUTINDEX.__SCAN_ */]
[SELECT DISTINCT
__C0_0 AS MONTH
FROM PUBLIC.__T0
/* "PersonWithoutIndex"."merge_scan" */]
***

public class PersonWithIndex implements Serializable {
@QuerySqlField(index = true)
public int id;
@QuerySqlField(index = true, orderedGroups =
{@QuerySqlField.Group(name="index_group_month_age", order=1)})
public int age;
@QuerySqlField
public String dni;
@QuerySqlField(index = true, orderedGroups =
{@QuerySqlField.Group(name="index_group_month_age", order=0)})
public String month;


2017-05-10 17:06 GMT+02:00 afedotov :

> Hi,
>
> Which Ignite version do you use?
> How many Ignite nodes do you run?
> What is your cache configuration?
>
> As well, could you please try the following query and see the performance?
> SELECT DISTINCT age FROM PersonWithindex;
>
>
> Kind regards,
> Alex.
>
> On Wed, May 10, 2017 at 5:57 PM, Guillermo Ortiz [via Apache Ignite Users]
> <[hidden email] >
> wrote:
>
>> It's my laptop. 16gb, i7.. The collection size it's 5Mill objects (1,2GB)
>> and it takes about 30sec.
>> I have tried to execute the query with the same collection with and
>> without indices with the same time results.
>>
>> This is a test before to run in the real cluster with 6 nodes of 512Gb
>> and 48cores each one. We tried to execute there with a real dataset(22M
>> rows, 7gb) but it takes so long that query is aborted.
>>
>>
>> 2017-05-08 12:12 GMT+02:00 Andrey Mashenkov <[hidden email]
>> >:
>>
>>> Hi,
>>>
>>> Why do you think query is slow?
>>> What its execution time? What is expected time?
>>> How many nodes do you use? How much data does cache contains?
>>>
>>> On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz <[hidden email]
>>> > wrote:
>>>
 I have a simple cache where I have Person object (id, age and so on..)

 I have created indeces by id and age to try some examples but queries
 with "group by" go really slow.

 I'm trying this query:
 SELECT age
 FROM
 PersonWithindex
 group by age


 SELECT
 AGE AS __C0
 FROM "personCacheWithIndex".PERSONWITHINDEX
 /* "personCacheWithIndex"."age_idx" */
 GROUP BY AGE
 /* group sorted */

 SELECT
 __C0 AS AGE
 FROM PUBLIC.__T0
 /* "personCacheWithIndex"."merge_scan" */
 GROUP BY __C0

 Although it seems that it uses index, why is it going so slow? I think
 that it should be pretty fast with an index.

>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>>
>> --
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-ignite-users.70518.x6.nabble.com/GroupBy-with-
>> index-is-really-slow-tp12526p12592.html
>> To start a new topic under Apache Ignite Users, email [hidden email]
>> 
>> To unsubscribe from Apache Ignite Users, click here.
>> NAML
>> 
>>
>
>
> --
> View this message in context: Re: GroupBy with index is really slow.
> 
> Sent from the Apache Ignite Users mailing list archive
>  at Nabble.com.
>

Re: Group indices and group by query

[Guillermo Ortiz]

Is there any limitation like databases which if it has to read more than x%
of data it do a full scan?

I tried this:

Result *SELECT MONTH, MAX(ID) FROM PERSONWITHINDEX GROUP BY MONTH *
September,1999868,February,1999869,March,1999871,August,1999844,June,1999867,April,1999854,July,1999866,January,1999857,May,1999862,November,1999830,October,1999864,
Elapsed time SELECT MONTH, MAX(ID) FROM PERSONWITHINDEX GROUP BY MONTH:
*2743ms*
[SELECT
__Z0.MONTH AS __C0_0,
MAX(__Z0.ID) AS __C0_1
FROM "PersonWithIndex".PERSONWITHINDEX __Z0
/* "PersonWithIndex".PERSONWITHINDEX_MONTH_IDX */
GROUP BY __Z0.MONTH
/* group sorted */]
[SELECT
__C0_0 AS MONTH,
MAX(__C0_1) AS __C0_1
FROM PUBLIC.__T0
/* "PersonWithIndex"."merge_scan" */
GROUP BY __C0_0]


**Result *SELECT MONTH, MAX(ID) FROM PERSONWITHOUTINDEX GROUP BY MONTH*
September,1999860,November,1999870,February,1999869,March,1999866,May,1999864,April,1999865,July,1999859,August,1999871,January,1999867,June,1999868,October,1999854,
Elapsed time SELECT MONTH, MAX(ID) FROM PERSONWITHOUTINDEX GROUP BY MONTH:
*2418ms*
[SELECT
__Z0.MONTH AS __C0_0,
MAX(__Z0.ID) AS __C0_1
FROM "PersonWithoutIndex".PERSONWITHOUTINDEX __Z0
/* "PersonWithoutIndex".PERSONWITHOUTINDEX.__SCAN_ */
GROUP BY __Z0.MONTH]
[SELECT
__C0_0 AS MONTH,
MAX(__C0_1) AS __C0_1
FROM PUBLIC.__T0
/* "PersonWithoutIndex"."merge_scan" */
GROUP BY __C0_0]


I tried to execute many times same query and times are similar all the
times, I guess that indices aren't being useful. Am I doing something bad?

public class PersonWithIndex implements Serializable {
@QuerySqlField(index = true)
public int id;
@QuerySqlField(index = true, orderedGroups =
{@QuerySqlField.Group(name="index_group_month_age", order=1)})
public int age;
@QuerySqlField
public String dni;
@QuerySqlField(index = true, orderedGroups =
{@QuerySqlField.Group(name="index_group_month_age", order=0)})
public String month;





public class PersonWithoutIndex implements Serializable {
@QuerySqlField
public int id;
@QuerySqlField
public int age;
@QuerySqlField
public String dni;
@QuerySqlField
public String month;


2017-05-08 12:32 GMT+02:00 Andrey Mashenkov :

> Group indices description can be found here [1]. Please, check if you read
> docs for your ignite version, as configuration for ignite 1.x and 2.0 can
> differs.
> Group indices in Ignite a similar to composite or multi-column indixes in
> databases.
>
> Yes, it is work that way. Group index for (groupFiled, timestampField)
> should be helpful.
> You can try to use EXPLAIN comand to check if correct index is in use.
>
>
> [1] https://apacheignite.readme.io/docs/indexes#section-group-indexes
>
> On Fri, May 5, 2017 at 8:17 PM, Guillermo Ortiz 
> wrote:
>
>>
>>
>> If I have two single indices to query:
>>
>> select * table a,b where a=1 and b=2
>>
>> it doesn't work pretty good and I have to create a group index, how is
>> that possible? how does group indices work?
>>
>> Similar to this:
>>
>> select max(timestampField)
>> from myTable
>> group by groupField
>>
>> I have a index by groupField what it means that it should be really fast
>> to make the group and later make a partial map to find in each node the max
>> of the subset of data to merge the final submax and get the final result in
>> a "reduce" task after sending data through network. Is it how does it work?
>> This query it's really slow, I don't know if I created an index by
>> timestampField I would get better performance, but it doesn't too much
>> sense to create a index with a timestampField where there aren't too row
>> with the same value in a table with 250M rows
>> Any advice about this query?
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>

Re: HDP, Hive + Ignite

[Ivan Veselovsky]

Alena, regarding NPEs in Ignite node logs, this seems to be
https://issues.apache.org/jira/browse/IGNITE-4862 , fixed, but not yet
merged.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12594.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: GroupBy with index is really slow.

[afedotov]

Hi,

Which Ignite version do you use?
How many Ignite nodes do you run?
What is your cache configuration?

As well, could you please try the following query and see the performance?
SELECT DISTINCT age FROM PersonWithindex;


Kind regards,
Alex.

On Wed, May 10, 2017 at 5:57 PM, Guillermo Ortiz [via Apache Ignite Users] <
ml+s70518n12592...@n6.nabble.com> wrote:

> It's my laptop. 16gb, i7.. The collection size it's 5Mill objects (1,2GB)
> and it takes about 30sec.
> I have tried to execute the query with the same collection with and
> without indices with the same time results.
>
> This is a test before to run in the real cluster with 6 nodes of 512Gb and
> 48cores each one. We tried to execute there with a real dataset(22M rows,
> 7gb) but it takes so long that query is aborted.
>
>
> 2017-05-08 12:12 GMT+02:00 Andrey Mashenkov <[hidden email]
> >:
>
>> Hi,
>>
>> Why do you think query is slow?
>> What its execution time? What is expected time?
>> How many nodes do you use? How much data does cache contains?
>>
>> On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz <[hidden email]
>> > wrote:
>>
>>> I have a simple cache where I have Person object (id, age and so on..)
>>>
>>> I have created indeces by id and age to try some examples but queries
>>> with "group by" go really slow.
>>>
>>> I'm trying this query:
>>> SELECT age
>>> FROM
>>> PersonWithindex
>>> group by age
>>>
>>>
>>> SELECT
>>> AGE AS __C0
>>> FROM "personCacheWithIndex".PERSONWITHINDEX
>>> /* "personCacheWithIndex"."age_idx" */
>>> GROUP BY AGE
>>> /* group sorted */
>>>
>>> SELECT
>>> __C0 AS AGE
>>> FROM PUBLIC.__T0
>>> /* "personCacheWithIndex"."merge_scan" */
>>> GROUP BY __C0
>>>
>>> Although it seems that it uses index, why is it going so slow? I think
>>> that it should be pretty fast with an index.
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/GroupBy-
> with-index-is-really-slow-tp12526p12592.html
> To start a new topic under Apache Ignite Users, email
> ml+s70518n1...@n6.nabble.com
> To unsubscribe from Apache Ignite Users, click here
> 
> .
> NAML
> 
>




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/GroupBy-with-index-is-really-slow-tp12526p12593.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: GroupBy with index is really slow.

[Guillermo Ortiz]

It's my laptop. 16gb, i7.. The collection size it's 5Mill objects (1,2GB)
and it takes about 30sec.
I have tried to execute the query with the same collection with and without
indices with the same time results.

This is a test before to run in the real cluster with 6 nodes of 512Gb and
48cores each one. We tried to execute there with a real dataset(22M rows,
7gb) but it takes so long that query is aborted.


2017-05-08 12:12 GMT+02:00 Andrey Mashenkov :

> Hi,
>
> Why do you think query is slow?
> What its execution time? What is expected time?
> How many nodes do you use? How much data does cache contains?
>
> On Mon, May 8, 2017 at 1:47 AM, Guillermo Ortiz 
> wrote:
>
>> I have a simple cache where I have Person object (id, age and so on..)
>>
>> I have created indeces by id and age to try some examples but queries
>> with "group by" go really slow.
>>
>> I'm trying this query:
>> SELECT age
>> FROM
>> PersonWithindex
>> group by age
>>
>>
>> SELECT
>> AGE AS __C0
>> FROM "personCacheWithIndex".PERSONWITHINDEX
>> /* "personCacheWithIndex"."age_idx" */
>> GROUP BY AGE
>> /* group sorted */
>>
>> SELECT
>> __C0 AS AGE
>> FROM PUBLIC.__T0
>> /* "personCacheWithIndex"."merge_scan" */
>> GROUP BY __C0
>>
>> Although it seems that it uses index, why is it going so slow? I think
>> that it should be pretty fast with an index.
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>

Using Native Infiniband for node interconnect?

[connie.woodward]

I see posts about people using IP over IB for inter-node communication, but
is it possible to use native IB protocols?  (IB RDMA, etc.).  I googled
around and saw that this was possible with Apache Spark, but what about for
Ignite? instructions?

thanks



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Using-Native-Infiniband-for-node-interconnect-tp12591.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

RE: How to correctly shut down Ignite Application

[Juan Barani]

Thanks Andrey,

For the moment it works correctly with preferIPv4Stack. The issue first 
appeared on 2.0, we run the same test suite since 1.6, and we have never seen 
it.

Thanks again,

Juan


From: Andrey Mashenkov [mailto:andrey.mashen...@gmail.com]
Sent: Wednesday, May 10, 2017 12:32 PM
To: user@ignite.apache.org
Subject: Re: How to correctly shut down Ignite Application

Hi Juan,

Looks like there is a bug, sometimes ignite choose ipv6 protocol to communicate 
with other nodes that causes strange communication errors.


On Wed, May 10, 2017 at 1:01 PM, Juan Barani 
> wrote:
Hi,

Not necessarily is the case that the client cannot connect to the server. I 
experienced yesterday with ignite 2.0, that when a client is forced with:

l_discoverySpi.setForceServerMode(true);

it comes as isClient=false in the exception message, even if setClientMode=true.

Since updating to 2.0, we experienced random NPEs when publishing a message in 
a topic during the interval a server went down, but failed to notify other 
nodes due to a hard stop =>

ERROR 2017-05-10T09:57:58,282 - 
de.uplanet.test.integration.RemoteTestServiceBean[pool-4-thread-1]
Failed to send message to remote node: TcpDiscoveryNode 
[id=ef626cb1-3880-418e-a9d1-68fd692771fd, addrs=[0:0:0:0:0:0:0:1%lo, 10.0.2.15, 
127.0.0.1, 172.17.0.1], sockAddrs=[/172.17.0.1:0, 
0:0:0:0:0:0:0:1%lo:0, /127.0.0.1:0, 
/10.0.2.15:0], discPort=0, order=3, intOrder=3, 
lastExchangeTime=1494410235152, loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, 
isClient=true]
org.apache.ignite.spi.IgniteSpiException: Failed to send message to remote 
node: TcpDiscoveryNode [id=ef626cb1-3880-418e-a9d1-68fd692771fd, 
addrs=[0:0:0:0:0:0:0:1%lo, 10.0.2.15, 127.0.0.1, 172.17.0.1], 
sockAddrs=[/172.17.0.1:0, 0:0:0:0:0:0:0:1%lo:0, 
/127.0.0.1:0, /10.0.2.15:0], 
discPort=0, order=3, intOrder=3, lastExchangeTime=1494410235152, loc=false, 
ver=2.0.0#20170430-sha1:d4eef3c6, isClient=true]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2483)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2419)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1329)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1698)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessageToGridTopic(GridIoManager.java:1473)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.sendUserMessage(GridIoManager.java:1588)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.IgniteMessagingImpl.sendOrdered(IgniteMessagingImpl.java:165)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
de.uplanet.lucy.server.distributed.cloud.datagrid.ignite.IgniteGridTopic.publish(IgniteGridTopic.java:58)
 ~[update/:?]
at 
de.uplanet.test.integration.RemoteTestServiceBean.lambda$3(RemoteTestServiceBean.java:123)
 ~[update/:?]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[?:1.8.0_92]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[?:1.8.0_92]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[?:1.8.0_92]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
Caused by: org.apache.ignite.IgniteCheckedException: 
java.lang.NullPointerException
at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7242) 
~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:258)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:170)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:139)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2630)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2455)
 ~[ignite-core-2.0.0.jar:2.0.0]
... 13 more
Caused by: java.util.concurrent.ExecutionException: 

Re: HDP, Hive + Ignite

[Alena Melnikova]

Hi Ivan,

1. I tried to run analytical query on table that created in IGFS. Here
couple of examples of errors.
beeline_output_1.log

  
ignite-node-dn1_1.log

  
beeline_output_2.log

  
ignite-node-dn1_2.log

  

4. We are looking forward results of your experiments.

p.s. I sent email again on 5th May.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12588.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: How to correctly shut down Ignite Application

[Andrey Mashenkov]

Hi Juan,

Looks like there is a bug, sometimes ignite choose ipv6 protocol to
communicate with other nodes that causes strange communication errors.


On Wed, May 10, 2017 at 1:01 PM, Juan Barani 
wrote:

> Hi,
>
>
>
> Not necessarily is the case that the client cannot connect to the server.
> I experienced yesterday with ignite 2.0, that when a client is forced with:
>
>
>
> l_discoverySpi.setForceServerMode(true);
>
>
>
> it comes as isClient=false in the exception message, even if
> setClientMode=true.
>
>
>
> Since updating to 2.0, we experienced random NPEs when publishing a
> message in a topic during the interval a server went down, but failed to
> notify other nodes due to a hard stop =>
>
>
>
> ERROR 2017-05-10T09:57:58,282 - de.uplanet.test.integration.
> RemoteTestServiceBean[pool-4-thread-1]
>
> Failed to send message to remote node: TcpDiscoveryNode
> [id=ef626cb1-3880-418e-a9d1-68fd692771fd, addrs=[0:0:0:0:0:0:0:1%lo,
> 10.0.2.15, 127.0.0.1, 172.17.0.1], sockAddrs=[/172.17.0.1:0,
> 0:0:0:0:0:0:0:1%lo:0, /127.0.0.1:0, /10.0.2.15:0], discPort=0, order=3,
> intOrder=3, lastExchangeTime=1494410235152, loc=false,
> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=true]
>
> org.apache.ignite.spi.IgniteSpiException: Failed to send message to
> remote node: TcpDiscoveryNode [id=ef626cb1-3880-418e-a9d1-68fd692771fd,
> addrs=[0:0:0:0:0:0:0:1%lo, 10.0.2.15, 127.0.0.1, 172.17.0.1], sockAddrs=[/
> 172.17.0.1:0, 0:0:0:0:0:0:0:1%lo:0, /127.0.0.1:0, /10.0.2.15:0],
> discPort=0, order=3, intOrder=3, lastExchangeTime=1494410235152,
> loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, isClient=true]
>
> at org.apache.ignite.spi.communication.tcp.
> TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2483)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.spi.communication.tcp.
> TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2419)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.internal.managers.communication.
> GridIoManager.send(GridIoManager.java:1329) ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.internal.managers.communication.
> GridIoManager.send(GridIoManager.java:1698) ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.internal.managers.communication.
> GridIoManager.sendOrderedMessageToGridTopic(GridIoManager.java:1473)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.internal.managers.communication.
> GridIoManager.sendUserMessage(GridIoManager.java:1588)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.internal.IgniteMessagingImpl.
> sendOrdered(IgniteMessagingImpl.java:165) ~[ignite-core-2.0.0.jar:2.0.0]
>
> at de.uplanet.lucy.server.distributed.cloud.datagrid.
> ignite.IgniteGridTopic.publish(IgniteGridTopic.java:58) ~[update/:?]
>
> at de.uplanet.test.integration.
> RemoteTestServiceBean.lambda$3(RemoteTestServiceBean.java:123)
> ~[update/:?]
>
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_92]
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_92]
>
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [?:1.8.0_92]
>
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [?:1.8.0_92]
>
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
>
> Caused by: org.apache.ignite.IgniteCheckedException:
> java.lang.NullPointerException
>
> at 
> org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7242)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.internal.
> util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:258)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.internal.
> util.future.GridFutureAdapter.get0(GridFutureAdapter.java:170)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.internal.
> util.future.GridFutureAdapter.get(GridFutureAdapter.java:139)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.spi.communication.tcp.
> TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2630)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> at org.apache.ignite.spi.communication.tcp.
> TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2455)
> ~[ignite-core-2.0.0.jar:2.0.0]
>
> ... 13 more
>
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.NullPointerException
>
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> ~[?:1.8.0_92]
>
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> ~[?:1.8.0_92]
>
> at org.apache.ignite.internal.util.IgniteUtils.
> filterReachable(IgniteUtils.java:1895) 

RE: How to correctly shut down Ignite Application

[Juan Barani]

Hi,

Not necessarily is the case that the client cannot connect to the server. I 
experienced yesterday with ignite 2.0, that when a client is forced with:

l_discoverySpi.setForceServerMode(true);

it comes as isClient=false in the exception message, even if setClientMode=true.

Since updating to 2.0, we experienced random NPEs when publishing a message in 
a topic during the interval a server went down, but failed to notify other 
nodes due to a hard stop =>

ERROR 2017-05-10T09:57:58,282 - 
de.uplanet.test.integration.RemoteTestServiceBean[pool-4-thread-1]
Failed to send message to remote node: TcpDiscoveryNode 
[id=ef626cb1-3880-418e-a9d1-68fd692771fd, addrs=[0:0:0:0:0:0:0:1%lo, 10.0.2.15, 
127.0.0.1, 172.17.0.1], sockAddrs=[/172.17.0.1:0, 0:0:0:0:0:0:0:1%lo:0, 
/127.0.0.1:0, /10.0.2.15:0], discPort=0, order=3, intOrder=3, 
lastExchangeTime=1494410235152, loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, 
isClient=true]
org.apache.ignite.spi.IgniteSpiException: Failed to send message to remote 
node: TcpDiscoveryNode [id=ef626cb1-3880-418e-a9d1-68fd692771fd, 
addrs=[0:0:0:0:0:0:0:1%lo, 10.0.2.15, 127.0.0.1, 172.17.0.1], 
sockAddrs=[/172.17.0.1:0, 0:0:0:0:0:0:0:1%lo:0, /127.0.0.1:0, /10.0.2.15:0], 
discPort=0, order=3, intOrder=3, lastExchangeTime=1494410235152, loc=false, 
ver=2.0.0#20170430-sha1:d4eef3c6, isClient=true]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2483)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2419)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1329)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1698)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessageToGridTopic(GridIoManager.java:1473)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.sendUserMessage(GridIoManager.java:1588)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.IgniteMessagingImpl.sendOrdered(IgniteMessagingImpl.java:165)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
de.uplanet.lucy.server.distributed.cloud.datagrid.ignite.IgniteGridTopic.publish(IgniteGridTopic.java:58)
 ~[update/:?]
at 
de.uplanet.test.integration.RemoteTestServiceBean.lambda$3(RemoteTestServiceBean.java:123)
 ~[update/:?]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[?:1.8.0_92]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[?:1.8.0_92]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[?:1.8.0_92]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[?:1.8.0_92]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
Caused by: org.apache.ignite.IgniteCheckedException: 
java.lang.NullPointerException
at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7242) 
~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:258)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:170)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:139)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2630)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2455)
 ~[ignite-core-2.0.0.jar:2.0.0]
... 13 more
Caused by: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
~[?:1.8.0_92]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
~[?:1.8.0_92]
at 
org.apache.ignite.internal.util.IgniteUtils.filterReachable(IgniteUtils.java:1895)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2891)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2702)
 ~[ignite-core-2.0.0.jar:2.0.0]
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2594)
 

Re: [ANNOUNCE] Apache Ignite 2.0.0 Released

[Pavel Tupitsyn]

Great news!

Thanks Denis, I've done some minor corrections to .NET part of release
notes page (IGNITE-4495 does not belong there).

On Fri, May 5, 2017 at 11:13 PM, Denis Magda  wrote:

> The Apache Ignite Community is pleased to announce the release of Apache
> Ignite 2.0.0.
>
> Apache Ignite In-Memory Data Fabric [1] is a high-performance, integrated
> and distributed in-memory platform for computing and transacting on
> large-scale data sets in real-time, orders of magnitude faster than
> possible with traditional disk-based or flash-based technologies.
>
> The Fabric is a collection of independent and well integrated components
> some of which are the following:
> - Data Grid
> - SQL Grid
> - Compute Grid
> - Service Grid
> - Machine Learning Grid (NEW!)
>
> This release incorporates tremendous changes that shifted Ignite 2.0 to
> the next level. The whole off-heap memory architecture was redesigned from
> scratch and, going forward, Ignite can be easily integrated with Flash and
> SSD drives (stay tuned!). SQL Grid has been enriched with Data Definition
> Language support. A completely new component was added to the fabric -
> Machine Learning Grid. We provided integrations with RocketMQ and Spring
> Data, updated Hibernate integration, introduced a plugin system for .NET
> and found out how to execute C++ code on remote node, and many more...
>
> Get more details from this blog post: https://blogs.apache.org/
> ignite/entry/apache-ignite-2-0-redesigned
>
> The full list of the changes can be found here [2].
>
> Please visit this page if you’re ready to try the release out:
> https://ignite.apache.org/download.cgi
>
> Please let us know [3] if you encounter any problems.
>
> Regards,
>
> The Apache Ignite Community
>
> [1] https://ignite.apache.org
> [2] https://ignite.apache.org/releases/2.0.0/release_notes.html
> [3] https://ignite.apache.org/community/resources.html#ask

Re: Docker cluster set up

[afedotov]

Hi.
Did I get it right that containers reside on different hosts? If it's so
then containers won't see each other out of box. You probably need to start
them via docker overlay network or use Swarm.

Kind regards,
Alex

9 мая 2017 г. 12:29 AM пользователь "waterg [via Apache Ignite Users]" <
ml+s70518n12549...@n6.nabble.com> написал:

All,

I'm trying to set up a cluster using docker. The network setting seems to
have some problem.
Could you please take a look and advise?
I have two hosts and each will run a container. Static ip address and basic
address resolver are used.
This is part of configuration on one container.












host1 public ip

host2 public ip







 
 
 
 
 
 
 
 
 
 

The other container has the same configuration under
TcpDiscoveryVmIpFinder, but the BasicAddressResolver has ip address for
host2/container2.

when I run the container, I use command "sudo docker run -p 47100:47100 -p
47500:47500 imagename"

Here's the result of docker ps on two hosts.

$ sudo docker ps
CONTAINER IDIMAGE   COMMAND
 CREATED STATUS
PORTS
 NAMES
3e1e86d18b43imagename   "/bin/sh -c $IGNITE_H"   27 seconds ago
 Up 26seconds
0.0.0.0:47100->47100/tcp,
11211/tcp, 49112/tcp, 0.0.0.0:47500->47500/tcp   nostalgic_
  noyce


$ sudo docker ps
CONTAINER IDIMAGE   COMMAND
 CREATED STATUS  PORTS
 NAMES
5bbd182dca5cimagename   "/bin/sh -c $IGNITE_H"   19 minutes ago
 Up 19 minutes   0.0.0.0:47100->47100/tcp, 11211/tcp, 49112/tcp,
0.0.0.0:47500->47500/tcp   romantic_galileo


Does the settings look right? Thank you!


--
If you reply to this email, your message will be added to the discussion
below:
http://apache-ignite-users.70518.x6.nabble.com/Docker-
cluster-set-up-tp12549.html
To start a new topic under Apache Ignite Users, email
ml+s70518n1...@n6.nabble.com
To unsubscribe from Apache Ignite Users, click here

.
NAML





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Docker-cluster-set-up-tp12549p12583.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: How to monitor and alert for server counts

[tysli2016]

we run the visor in 10 every mins, the Ignite servers would OOME in a couple
weeks.
Ignite server allocated 1g memory.

would you mind to share you server's config.xml? any JVM parameters changes?

thanks 
Tom


ignite_user2016 wrote
> yes you are correct, we run visor in the batch mode.
> 
> we have 2 host hosting 2 client and 2 server for ignite, our conf is bare
> minimal 4 cores 8 GB RAM.
> 
> Now with OOME, how often you run the visor command ? we run it every 5
> mins.should you try that ?
> 
> On Mon, May 8, 2017 at 9:01 PM, tysli2016 [via Apache Ignite Users] <

> ml+s70518n12557h21@.nabble

>> wrote:
> 
>> thanks Rishi, can you share more about that?
>> what's the version of Ignite? how many Ignite servers? how many
>> CPU/memory?
>> are you using the Visor in batch mode (https://apacheignite-tools.
>> readme.io/v1.9/docs/batch-mode)?
>> or Visor alert?
>>
>> I have tried Visor batch mode, but it lead to OOME eventually (
>> http://apache-ignite-users.70518.x6.nabble.com/OOME-on-2-
>> node-cluster-with-visor-running-repeatedly-Ignite-1-9-td12409.html).
>>
>> --
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-ignite-users.70518.x6.nabble.com/How-to-
>> monitor-and-alert-for-server-counts-tp12533p12557.html
>> To start a new topic under Apache Ignite Users, email
>> 

> ml+s70518n1h85@.nabble

>> To unsubscribe from Apache Ignite Users, click here
>> http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=1code=cmlzaGl5YWduaWtAZ21haWwuY29tfDF8MTMwNTI4OTg1Mw==;
>> .
>> NAML
>> http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml;
>>
> 
> 
> 
> -- 
> Rishi Yagnik





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/How-to-monitor-and-alert-for-server-counts-tp12533p12582.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.