Re: Another replicated cache oddity

2023-11-22 Thread Raymond Wilson
matter for similar reasons to your experience. > > Are there any signs of strain in any of the Ignite servers' monitoring > metrics when these events happen? What kind of accumulated workload is on > the cluster at those times, and how often does this happen? > > > On

Re: Another replicated cache oddity

2023-11-22 Thread Raymond Wilson
ter and cache configuration in play? What does the code look like for > the updates to the replicated cache, and what does the code look like for > the distributed compute operation? > > On Tue, Nov 21, 2023 at 5:21 PM Raymond Wilson > wrote: > >> Hi, >> >&

Another replicated cache oddity

2023-11-21 Thread Raymond Wilson
e back' after a cluster restart? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trim

Graceful shutdown of DotNet Ignite nodes running in Kubernetes pods

2023-09-06 Thread Raymond Wilson
and deployment context, how are you ensuring the Ignite node implementation gets the SIGTERM and shuts down gracefully? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New

Re: Cache write synchonization with replicated caches

2023-09-04 Thread Raymond Wilson
> > Does this imply risk if run against a production environment grid? > > It depends. > As for read repair, CHECK_ONLY is read only mode and can’t harm your data. > Other modes that fix data inconsistency was used on our production and > there are no known issues. >

Re: Possible bug failing to create a cache on a running grid causing grid failure

2023-08-28 Thread Raymond Wilson
FYI, I raised Jira ticket https://issues.apache.org/jira/browse/IGNITE-20299 for this. On Mon, Aug 28, 2023 at 3:42 PM Raymond Wilson wrote: > We have tried the same renaming in the dev environment which has multiple > server nodes impacted and contains some data (unlike the local

Re: Possible bug failing to create a cache on a running grid causing grid failure

2023-08-27 Thread Raymond Wilson
23 PM Raymond Wilson wrote: > Looking at the cache-SiteModelMetaData folder in the persistence folder > for a server node shows a "cache_data" file 6kb in size. No other cache > folders contain this file. > > As an experiment I renamed this file to "cache_dataxxx". Th

Re: Possible bug failing to create a cache on a running grid causing grid failure

2023-08-27 Thread Raymond Wilson
the grid to restart. Similarly renaming the cache folder to "xxxcache-SiteModelMetaData" also permitted the grid to restart; we will be testing this further to verify. Raymond. On Sun, Aug 27, 2023 at 5:20 PM Raymond Wilson wrote: > I have reproduced the possible bug I reported in my e

Re: Possible bug failing to create a cache on a running grid causing grid failure

2023-08-26 Thread Raymond Wilson
point the grid is now unusable. To summarise: Attempted creation of a cache with an unknown DataRegionName causes immediate and unrecovered failure in the entire grid. Raymond. On Fri, Aug 25, 2023 at 7:47 PM Raymond Wilson wrote: > We believe we had some code on a dev environment attempt to cre

Possible bug failing to create a cache on a running grid causing grid failure

2023-08-25 Thread Raymond Wilson
We believe we had some code on a dev environment attempt to create a cache that was intended for another Ignite. The creation of this cache would have failed (at least) because the data region referenced in the cache configuration does not exist on that environment. A subsequent restart of the en

Re: Cache write synchonization with replicated caches

2023-08-21 Thread Raymond Wilson
> [2] > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/cache/ReadRepairStrategy.java > > > > 21 авг. 2023 г., в 07:46, Raymond Wilson > написал(а): > > [Replying onto correct thread] > > As a follow up to this email, we are start

Re: Cache write synchonization with replicated caches

2023-08-20 Thread Raymond Wilson
, Raymond. On Thu, Jul 27, 2023 at 12:38 PM Raymond Wilson wrote: > Hi, > > I have a query regarding data safety of replicated caches in the case of > hard failure of the compute resource but where the storage resource is > available when the node returns. > > We are using Ignite 2.

Re: Cache write synchronization mode

2023-08-20 Thread Raymond Wilson
:37 AM Raymond Wilson wrote: > I have a query regarding the CacheWriteSynchronizationMode in > CacheConfiguration. > > This enum is defined like this in the .Net client: > > public enum CacheWriteSynchronizationMode > { > /// > /// Mode indicating that Igni

Re: Simple question relating to multiple data regions with persistence

2023-08-13 Thread Raymond Wilson
Actually, ignore that question. I was conflating storage and data region configurations so my question is invalid! On Mon, Aug 14, 2023 at 11:40 AM Raymond Wilson wrote: > I am looking at defining additional data regions with persistence to > support cache with some different settings (eg

Simple question relating to multiple data regions with persistence

2023-08-13 Thread Raymond Wilson
different storage folders for each data region for data and WAL files, or whether Ignite is happy to have all data regions using the same base storage path. Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham

Re: Cache write synchronization mode

2023-07-26 Thread Raymond Wilson
x27;m not 100% sure - could you ask about this > specific case in a separate thread? > > On Tue, Jul 25, 2023 at 8:50 AM Raymond Wilson > wrote: > >> >> However, if a primary node fails before at least 1 backup >> node receives an update, then the update will be l

Cache write synchonization with replicated caches

2023-07-26 Thread Raymond Wilson
is an improvement in write safety in this instance, does this imply the Fsync WalMode write pathway has opportunities for data loss in these failure situations? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software

Re: Cache write synchronization mode

2023-07-24 Thread Raymond Wilson
lost, and all nodes will have the old value. > > Related: CacheConfiguration.ReadFromBackup property is true by default, > meaning that with PrimarySync it is possible to get old value from a backup > node after an update, before backups receive new data. > > On Mon, Jul 24, 2023 at

Re: Ignite data region off-heap allocation

2023-07-24 Thread Raymond Wilson
llocated > only when the first cache is created in that region (when true), or > immediately (when false) > > > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/DataRegionConfiguration.html#isLazyMemoryAllocation-- > > > On Wed, Jul 19, 2023 at

Re: Cache write synchronization mode

2023-07-24 Thread Raymond Wilson
Hi Pavel, I understand the differences between the sync modes in terms of when the write returns. What I want to understand is if there are consistency risks with the PrimarySync versus FullSync modes. For example, if I have 4 nodes participating in the replicated cache (and am using the default

Re: Possible WAL corruption on running system during K8s update

2023-07-19 Thread Raymond Wilson
Jul 2023 at 10:02 AM, Raymond Wilson wrote: > Hi Alex, > > Here is the log from the Ignite startup. It's fairly short but shows > everything I think: > > 2023-07-17 22:38:55,061 [1] DBG [ImmutableCacheComputeServer] Starting > Ignite.NET 2.15.0.23172 > 20

Re: Ignite data region off-heap allocation

2023-07-19 Thread Raymond Wilson
Just FYI, we have held off any memory pressure changes in the meantime while we continue to investigate the memory issues we have. On Tue, 18 Jul 2023 at 9:07 AM, Raymond Wilson wrote: > Hi Pavel, > > This area is confusing. There is no indication that the memory pressure > ap

Cache write synchronization mode

2023-07-18 Thread Raymond Wilson
uration and ran into this item. Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble

Re: Possible WAL corruption on running system during K8s update

2023-07-18 Thread Raymond Wilson
--- End of inner exception stack trace --- at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg) Thanks, Raymond. On Wed, Jul 19, 2023 at 5:43 AM Raymond Wilson wrote: > Hi Alex, > > We are using Ignite v2.15. > > I will track down the additional log information a

Re: Possible WAL corruption on running system during K8s update

2023-07-18 Thread Raymond Wilson
tion during start processors, > node will be stopped and close connections" (there should be a reason in > the log, why the page delta can't be applied). > > вт, 18 июл. 2023 г. в 05:05, Raymond Wilson : > >> Hi, >> >> We run a dev/alpha stack of our application in A

Possible WAL corruption on running system during K8s update

2023-07-17 Thread Raymond Wilson
and checkpoints are every 30-45 seconds or so, so this won't be significant). Is this an error anyone else has noticed? Has anyone else had similar issues with Azure Files when using strict update/sync semantics? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimbl

Re: Ignite data region off-heap allocation

2023-07-17 Thread Raymond Wilson
the > sum of those is not bigger than available RAM. > > [1] > https://stackoverflow.com/questions/1149181/what-is-the-point-of-using-gc-addmemorypressure-with-an-unmanaged-resource > [2] > https://ignite.apache.org/docs/latest/memory-configuration/data-regions#configuring-de

Re: Ignite data region off-heap allocation

2023-07-11 Thread Raymond Wilson
How do Ignite .Net server nodes manage this memory issue in other projects? On Tue, Jul 11, 2023 at 5:32 PM Raymond Wilson wrote: > Oops, commutes => committed > > On Tue, 11 Jul 2023 at 4:34 PM, Raymond Wilson > wrote: > >> I can’t see another way of letting . Net

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
Oops, commutes => committed On Tue, 11 Jul 2023 at 4:34 PM, Raymond Wilson wrote: > I can’t see another way of letting . Net know that it can’t have access to > all the ‘free’ memory in the process when a large slab of that is spoken > for in terms of memory commutes to Ignite

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
yPressure documentation [1] states that this will "improve > performance only for types that exclusively depend on finalizers". > > [1] > https://learn.microsoft.com/en-us/dotnet/api/system.gc.addmemorypressure?view=net-7.0 > > On Tue, Jul 11, 2023 at 1:02 AM Raymond Wilson &g

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
u aware of any? Raymond. On Mon, Jul 10, 2023 at 9:41 PM Raymond Wilson wrote: > Thanks Pavel, this makes sense. > > Querying the .Net Process instance shows this as the difference between > PagesMemorySize (includes committed) versus WorkingSet (includes > uses/written to) size.

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
Thanks Pavel, this makes sense. Querying the .Net Process instance shows this as the difference between PagesMemorySize (includes committed) versus WorkingSet (includes uses/written to) size. Raymond.

Re: Ignite data region off-heap allocation

2023-07-09 Thread Raymond Wilson
committed-allocated-memory-in-linux-is-less-than-used-memory-how-is-that-possib > - > https://unix.stackexchange.com/questions/137773/is-inactive-memory-related-to-the-commited-but-unused > > On Sat, Jul 8, 2023 at 11:44 AM Raymond Wilson > wrote: > >> Hi, >> >>

Ignite data region off-heap allocation

2023-07-08 Thread Raymond Wilson
actually allocated. I feel I am missing something here! Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.tr

Pending critical volnerabilities for OpenSSL and Apache Commons Text

2022-10-30 Thread Raymond Wilson
s made available. Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>

Re: Ignite Async API call continutation behaviour

2022-10-02 Thread Raymond Wilson
Thanks Pavel! :) On Mon, Oct 3, 2022 at 6:50 PM Pavel Tupitsyn wrote: > Hi Raymond, > > Yes, you can remove those ContinueWith() calls. Default behavior is now > safe. > > On Mon, Oct 3, 2022 at 2:25 AM Raymond Wilson > wrote: > >> Jay, >> >> Th

Re: Ignite Async API call continutation behaviour

2022-10-02 Thread Raymond Wilson
t; thread. > > When doing `await`, we most likely end up on another thread and lose > transaction context. > > > > There were plans to make transactions work across multiple threads, but > the ticket looks abandoned: > > https://issues.apache.org/jira/browse/IGNIT

Ignite Async API call continutation behaviour

2022-10-02 Thread Raymond Wilson
ing on a .Net threadpool thread by default. Is that correct? This will allow us to remove the (quite a few) .ContinueWith() statements which do add a layer of overhead on async calls. Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Con

Re: Re[2]: Checkpointing threads

2022-09-07 Thread Raymond Wilson
ing threads appropriate, or are we likely to see > better performance if we increased it to 8 (or more)? > > If there is a discussion related to this a pointer to it would be good > (it's not really covered in the performance tuning section). > > Thanks, > Raymond. >

Re: Checkpointing threads

2022-09-02 Thread Raymond Wilson
nce if we increased it to 8 (or more)? > > If there is a discussion related to this a pointer to it would be good > (it's not really covered in the performance tuning section). > > Thanks, > Raymond. > > -- > <http://www.trimble.com/> > Raymond Wilson > Tri

Checkpointing threads

2022-09-01 Thread Raymond Wilson
performance if we increased it to 8 (or more)? If there is a discussion related to this a pointer to it would be good (it's not really covered in the performance tuning section). Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Constructi

Re: Checkpointing is taking long time

2022-04-18 Thread Raymond Wilson
gt; > > ^-- default region [type=default, persistence=true, lazyAlloc=true, > > > > ... initCfg=24576MB, maxCfg=24576MB, usedRam=1018MB, freeRam=95.86%, > > > > allocRam=24576MB, allocTotal=3820MB] > > > > ^-- metastoreMemPlc region [type=internal, persistenc

Re: Running Apache Ignite on .Net 6

2022-01-20 Thread Raymond Wilson
/core/compatibility/6.0 > > On Wed, Jan 19, 2022 at 11:56 PM Raymond Wilson < > raymond_wil...@trimble.com> wrote: > >> Has anyone tried running Apache Ignite on .Net 6? >> >> I understand it runs on .Net 5 from Pavel's blog ( >> https://ptupitsyn.github.io

Running Apache Ignite on .Net 6

2022-01-19 Thread Raymond Wilson
om IA 2.8 and make use of the latest .Net platform as well. Given .Net 6 is LTS, I'd prefer going there directly... Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchu

Re: Native Persistence Tuning - Percentage kept in RAM

2021-06-02 Thread Raymond Wilson
t; quickly ran in to an IgniteOutOfMemoryException, despite the fact there was > plenty of space on disk. Why would it run out of RAM if it can persist data > above the MaxSize the excess to disk? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > >

Issue building Ignite 2.10 branch

2021-05-09 Thread Raymond Wilson
f :ignite-jta Is there an easy work around for this? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?

Re: Poor Performance with Increasing Public Thread Pool Size

2021-04-26 Thread Raymond Wilson
gt; } > > > > > > public class LongRunning_DotNet_Task : IComputeFunc > > { > > public bool Invoke(string[] arg) > > { > > // do work that takes up to 5 minutes with 12 threads, but > takes 50 minutes with 48 threads! > > return true; > >} > > } > > > -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>

Re: Async operations in IA C# client appear dangeroud

2021-03-11 Thread Raymond Wilson
Just as a followup query: Are async Compute methods susceptible to a similar issue, and should have the same treatment using a ContinueWith()? Thanks, Raymond. On Thu, Mar 11, 2021 at 8:45 PM Raymond Wilson wrote: > Hi Pavel, > > Thanks for the quick response. > > We ha

Re: Async operations in IA C# client appear dangeroud

2021-03-10 Thread Raymond Wilson
nd > verbose. > > > There should be a global Ignite setting to move all async continuations > away from the striped pool. > I'm taking the ticket [1] ASAP, it is a shame that we let it sit for so > long. > > [1] https://issues.apache.org/jira/browse/IGNITE-12033 > &

Re: Async operations in IA C# client appear dangeroud

2021-03-09 Thread Raymond Wilson
Title correction: Async operations in IA C# client appear dangerous On Wed, Mar 10, 2021 at 6:05 PM Raymond Wilson wrote: > We are using IA 2.8.1 with the C# client. > > After triaging intermittent critical thread blockages we determined we > have run into this problem reported

Async operations in IA C# client appear dangeroud

2021-03-09 Thread Raymond Wilson
deprecated in the C# client until the underlying issues are resolved? It is hard to see how any non-trivial C# client based Ignite application can safely use them. Regards, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CC

IA 2.8.1 C# Client ICacheAffinity question

2021-03-03 Thread Raymond Wilson
xposable through the ICacheAffinity interface. The MapPartitionToPrimaryAndBackups(int part) looks like it would provide the required information, but would need to be called for every partition in the cache, which feels like it might be slow. Thoughts? Thanks, Raymond. -- <http://www.trimb

Re: Graceful shutdown and request draining of Ignite servers

2021-02-18 Thread Raymond Wilson
Ilya Kasnacheev wrote: > Hello! > > This sounds like a too detailed and peculiar scenario that should be taken > care of on the application level, as you already do. > > Regards, > -- > Ilya Kasnacheev > > > ср, 17 февр. 2021 г. в 23:50, Raymond Wilson : > >>

Re: Graceful shutdown and request draining of Ignite servers

2021-02-17 Thread Raymond Wilson
uling can gracefully exit the grid after ensuring the active requests it is involved in can complete normally. In the case above, node B would be a client node providing services through a web api gateway (A) and requesting results from co-located processing on node C. Thanks, Raymond. On Thu, Feb 1

Re: Graceful shutdown and request draining of Ignite servers

2021-02-17 Thread Raymond Wilson
wrote: > Hello! > > Why can't you just use Ignite.stop(instanceName, false)? > > Just make sure your projections are not singleton and the tasks will be > rolled over. > > Regards, > -- > Ilya Kasnacheev > > > вт, 9 февр. 2021 г. в 06:41, Raymond Wil

Graceful shutdown and request draining of Ignite servers

2021-02-08 Thread Raymond Wilson
) then we would proceed with actual shut down of that node. I have not seen any capability in Ignite today where a node can be 'un-blessed'; does one exist? Or should we construct this facility within our application logic layer? Thanks, Raymond. -- <http://www.trimble.com/> Ra

Re: Detecting checkpoints programmatically

2021-01-28 Thread Raymond Wilson
OINT_SAVED, > EVT_CHECKPOINT_LOADED, > EVT_CHECKPOINT_REMOVED > }; > > Are these the appropriate events to listen to? > > Thanks, > Raymond. > > -- > <http://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Constructio

Detecting checkpoints programmatically

2021-01-28 Thread Raymond Wilson
? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>

Re: Ever increasing startup times as data grow in persistent storage

2021-01-28 Thread Raymond Wilson
> can create a thread in the Ignite developer list: > > http://apache-ignite-developers.2346864.n4.nabble.com/ > > BR, > Andrei > > > 1/20/2021 9:16 PM, Raymond Wilson пишет: > > Hi Andre, > > I would like to see Ignite support a graceful shutdown scenario you

Re: Ever increasing startup times as data grow in persistent storage

2021-01-20 Thread Raymond Wilson
drei > 1/20/2021 5:50 AM, Raymond Wilson пишет: > > We have been experimenting with using deactivation to shutdown the grid to > reduce the time for the grid to start up again. > > It appears there is a downside to this: once deactivated the grid does not > appear to auto-activ

Re: Ever increasing startup times as data grow in persistent storage

2021-01-19 Thread Raymond Wilson
Cluster.ICluster.html?#Apache_Ignite_Core_Cluster_ICluster_SetActive_System_Boolean_ > > On Wed, Jan 13, 2021 at 11:54 AM Raymond Wilson < > raymond_wil...@trimble.com> wrote: > >> Of course. Obvious! :) >> >> Sent from my iPhone >> >> On 13/01/2021, at 9:15 PM, Zhenya Stanilovsky wrote: >>

Re: Ignite best practice for restarting k8s pod

2021-01-15 Thread Raymond Wilson
tps://issues.apache.org/jira/browse/IGNITE-13974 >> >> >> >> >> >> Currently I do not see any cleanup functions getting called when we do a >> >> 'kubectl delete pod'. >> >> >> >> May I know, what is the best practice for r

Re: Ever increasing startup times as data grow in persistent storage

2021-01-13 Thread Raymond Wilson
partition read while applying WAL changes? don`t think so, may be someone else suggest here? Does anyone else have this issue? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New

Re: Re[2]: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
Architect, Civil Construction Software Systems (CCSS) > 11 Birmingham Drive | Christchurch, New Zealand > raymond_wil...@trimble.com > <http://e.mail.ru/compose/?mailto=mailto%3araymond_wil...@trimble.com> > > > > <https://worksos.trimble.com/?utm_source=Trimb

Re: Re[2]: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
hecking pointing intervals help? > > > yes, it will help. Check > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood > > Is the entire content of a partition read while applying WAL changes? > > > don`t think so, may be someone els

Re: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
eck > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood > > Is the entire content of a partition read while applying WAL changes? > > > don`t think so, may be someone else suggest here? > > Does anyone else have this issue? > > T

Re: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
We are currently using AI 2.8.1 with the c# client. On Wed, Jan 13, 2021 at 8:12 PM Kirill Tkalenko wrote: > Hello, Raymond! What version are you using? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- <http://www.trimble.com/> Raymon

Re: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
er active > Regarding the no of partitions also, 128 partitions should do and its doing > good for us > > Thanks > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- <http://www.trimble.com/> Raymond Wilson Solution Architect

Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
while applying WAL changes? Does anyone else have this issue? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.c

Re: Difference between Stopping and Closing Ignite on C#

2021-01-12 Thread Raymond Wilson
Apologies - I misread the description of Stop(). Please ignore this thread. On Wed, Jan 13, 2021 at 3:50 PM Raymond Wilson wrote: > I've just been reading this article ( > https://stackoverflow.com/questions/52780726/how-to-trigger-ignite-checkpoint-before-shutdown) > which

Difference between Stopping and Closing Ignite on C#

2021-01-12 Thread Raymond Wilson
terminated during shutdown. Thanks Raymond -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble&utm_medium

Re: Questions related to check pointing

2021-01-12 Thread Raymond Wilson
rowth of the data > region? Is it better to set the initial and maximum sizes of data regions > to be the same number? > > 2. We have two data regions, one supporting inbound arrival of data (with > low numbers of writes), and one supporting storage of processed results > from the arr

Re: Re[4]: Questions related to check pointing

2021-01-06 Thread Raymond Wilson
t 11:39 PM Ilya Kasnacheev wrote: > Hello! > > I guess it's pool.pages() * 3L / 4 > Since, counter intuitively, the default ThrottlingPolicy is not > ThrottlingPolicy.DISABLED. It's CHECKPOINT_BUFFER_ONLY. > > Regards, > > -- > Ilya Kasnacheev > > >

Re: Moving caches between data regions

2021-01-01 Thread Raymond Wilson
Once the cache has been moved to another region, can the region it was held in be removed (it is the only cache in that region)? On Thu, Dec 31, 2020 at 11:57 PM Raymond Wilson wrote: > Hi Ilya, > > Thanks for confirming. > > Raymond. > > Sent from my iPhone > >

Re: Moving caches between data regions

2020-12-31 Thread Raymond Wilson
these data regions is this > possible by changing the configuration of the caches to be moved so they > all point to the same data region? > > Thanks, > Raymond. > > -- > <http://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Construction Softwa

Moving caches between data regions

2020-12-31 Thread Raymond Wilson
by changing the configuration of the caches to be moved so they all point to the same data region? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimb

Re: Re[4]: Questions related to check pointing

2020-12-30 Thread Raymond Wilson
better to set the initial and maximum sizes of data regions > to be the same number? > > 2. We have two data regions, one supporting inbound arrival of data (with > low numbers of writes), and one supporting storage of processed results > from the arriving data (with many more w

Re: Re[4]: Questions related to check pointing

2020-12-30 Thread Raymond Wilson
e growth of the data > region? Is it better to set the initial and maximum sizes of data regions > to be the same number? > > 2. We have two data regions, one supporting inbound arrival of data (with > low numbers of writes), and one supporting storage of processed results > from th

Re: Re[2]: Questions related to check pointing

2020-12-30 Thread Raymond Wilson
? If so, is this something that can be improved? Thanks, Raymond. On Wed, Dec 30, 2020 at 9:17 PM Raymond Wilson wrote: > I'm working on getting automatic JVM thread stack dumping occurring if we > detect long delays in put (PutIfAbsent) operations. Hopefully this will > provide mo

Re: Re[2]: Questions related to check pointing

2020-12-30 Thread Raymond Wilson
ra ticket for this in the 2.9.1 > Jira area at > https://issues.apache.org/jira/browse/IGNITE-13876?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.9.1%20and%20status%20%3D%20Resolved > > Perhaps this change may improve the checkpointing issue we are seeing? > > Raymo

Re: Questions related to check pointing

2020-12-29 Thread Raymond Wilson
%20and%20status%20%3D%20Resolved Perhaps this change may improve the checkpointing issue we are seeing? Raymond. On Tue, Dec 29, 2020 at 8:35 PM Raymond Wilson wrote: > Hi Zhenya, > > 1. We currently use AWS EFS for primary storage, with provisioned IOPS to > provide sufficient IO

Re: Ignite thread pools

2020-12-29 Thread Raymond Wilson
20 г. в 08:33, Raymond Wilson : > Hi, > > I have been drilling into some of the Ignite thread pool support and have > found references in the code to three thread pools that don't appear to be > referenced in the Ignite thread pool documentation at > https://apacheignite.readm

Re: Feature request: On demand thread dumps from Ignite

2020-12-28 Thread Raymond Wilson
) Otherwise, is > this something worth adding to the backlog? > > Thanks, > Raymond. > > -- > <http://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Construction Software Systems (CCSS) > 11 Birmingham Drive | Christchurch, New Zealand > +64-21-2013

Re: Questions related to check pointing

2020-12-28 Thread Raymond Wilson
int. > > It seems like the checkpoint is affecting the put operations, but I don't > understand why that may be given the documented checkpointing process, and > the checkpoint itself (at least via Informational logging) is not > advertising any restrictions. > > Thanks, >

Feature request: On demand thread dumps from Ignite

2020-12-28 Thread Raymond Wilson
actually the case then please point me at it:) Otherwise, is this something worth adding to the backlog? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand +64-21-2013317

Re: Questions related to check pointing

2020-12-28 Thread Raymond Wilson
As another detail, we have the WriteThrottlingEnabled property left at its default value of 'false', so I would not ordinarily expect throttling, correct? On Tue, Dec 29, 2020 at 10:04 AM Raymond Wilson wrote: > Hi Ilya, > > Regarding the throttling question, I have not y

Re: Questions related to check pointing

2020-12-28 Thread Raymond Wilson
> between them. > > 3. I don't think that default throttling mechanism will emit any warnings. > What do you see in thread dumps? > > Regards, > -- > Ilya Kasnacheev > > > ср, 23 дек. 2020 г. в 12:48, Raymond Wilson : > >> Hi, >> >> We have been inve

Ignite thread pools

2020-12-27 Thread Raymond Wilson
f the thread pools themselves. Can the thread pool documentation be extended to include these additional pools, particularly with respect to their purpose and roles, and the circumstances when you would change them from the default value. Thanks, Raymond. -- <http://www.trimble.com/> Ray

Questions related to check pointing

2020-12-23 Thread Raymond Wilson
oint. It seems like the checkpoint is affecting the put operations, but I don't understand why that may be given the documented checkpointing process, and the checkpoint itself (at least via Informational logging) is not advertising any restrictions. Thanks, Raymond. -- <http://www.trimb

Re: C#/.NET: Possible to assign different nodes different roles?

2020-12-10 Thread Raymond Wilson
Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand +64-21-2013317 Mobile raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>

Re: Binary memory restoration

2020-10-08 Thread Raymond Wilson
gt; вт, 6 окт. 2020 г. в 02:10, Raymond Wilson : > >> Thanks for the thoughts Ilya and Vladimir. >> >> We'll do a comparison with 2.9 when it releases to see if that makes any >> difference. >> >> One of the advantages with persistent storage is that it i

Re: Continuous Query

2020-10-07 Thread Raymond Wilson
gt;>>>>>>> >>>>>>>>>> - >>>>>>>>>> Denis >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Oct 2, 2020 at 11:22 AM narges saleh < >>>>>>>>>> snarges...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> The detection should happen at most a couple of minutes after a >>>>>>>>>>> record is inserted in the cache but all the detections are local to >>>>>>>>>>> the >>>>>>>>>>> node. But some records with the current timestamp might show up in >>>>>>>>>>> the >>>>>>>>>>> system with big delays. >>>>>>>>>>> >>>>>>>>>>> On Fri, Oct 2, 2020 at 12:23 PM Denis Magda >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> What are your requirements? Do you need to process the records >>>>>>>>>>>> as soon as they are put into the cluster? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Friday, October 2, 2020, narges saleh >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Thank you Dennis for the reply. >>>>>>>>>>>>> From the perspective of performance/resource overhead and >>>>>>>>>>>>> reliability, which approach is preferable? Does a continuous >>>>>>>>>>>>> query based >>>>>>>>>>>>> approach impose a lot more overhead? >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Oct 2, 2020 at 9:52 AM Denis Magda >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Narges, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Use continuous queries if you need to be notified in >>>>>>>>>>>>>> real-time, i.e. 1) a record is inserted, 2) the continuous >>>>>>>>>>>>>> filter confirms >>>>>>>>>>>>>> the record's time satisfies your condition, 3) the continuous >>>>>>>>>>>>>> queries >>>>>>>>>>>>>> notifies your application that does require processing. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The jobs are better for a batching use case when it's ok to >>>>>>>>>>>>>> process records together with some delay. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> - >>>>>>>>>>>>>> Denis >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Oct 2, 2020 at 3:50 AM narges saleh < >>>>>>>>>>>>>> snarges...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>> If I want to watch for a rolling timestamp pattern in all >>>>>>>>>>>>>>> the records that get inserted to all my caches, is it more >>>>>>>>>>>>>>> efficient to use >>>>>>>>>>>>>>> timer based jobs (that checks all the records in some interval) >>>>>>>>>>>>>>> or >>>>>>>>>>>>>>> continuous queries that locally filter on the pattern? These >>>>>>>>>>>>>>> records can >>>>>>>>>>>>>>> get inserted in any order and some can arrive with delays. >>>>>>>>>>>>>>> An example is to watch for all the records whose timestamp >>>>>>>>>>>>>>> ends in 50, if the timestamp is in the format -mm-dd hh:mi. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> thanks >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> - >>>>>>>>>>>> Denis >>>>>>>>>>>> >>>>>>>>>>>> -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand +64-21-2013317 Mobile raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>

Re: Binary memory restoration

2020-10-05 Thread Raymond Wilson
have > at least on query entity configured. > As far as I can see it's going to be a part of the Ignite 2.9 release. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Cons

Re: Binary memory restoration

2020-09-28 Thread Raymond Wilson
in this example. Currently we have around 10 caches with less than 20 Gb of data. Do these startup times seem reasonable for a node that is shut down gracefully? Thanks, Raymond. On Tue, Sep 29, 2020 at 11:20 AM Raymond Wilson wrote: > Just a general query on Ignite server node startup sequence. &

Binary memory restoration

2020-09-28 Thread Raymond Wilson
2.8.1 using the C# client with persistence enabled. Do these messages indicate Ignite is reloading the memory state of the caches at the point it was shutdown? If not, what is this activity? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Constr

Failing to update metadata for type

2020-08-23 Thread Raymond Wilson
ation to Ignite configuration on startup, eg: cfg.BinaryConfiguration = new BinaryConfiguration { Serializer = new BinarizableSerializer() }; Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software System

Re: Ignite C# client does not report correct node ID after reconnection

2020-08-18 Thread Raymond Wilson
bug report. > I've filed a ticket [1]. > > I'll see if that's possible to squeeze the fix into the 2.9 release, > otherwise you can expect it in 2.10. Will keep you posted. > > [1] https://issues.apache.org/jira/browse/IGNITE-13369 > > On Tue, Aug 18, 2020

Ignite C# client does not report correct node ID after reconnection

2020-08-18 Thread Raymond Wilson
l node that may be at risk of being inconsistent if the underlying grid information changes. Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand +64-21-2013317 Mobile

Re: Are CPU Metrics logged by Ignite correct?

2020-07-28 Thread Raymond Wilson
owse/IGNITE-13306 > > Evgenii > > пн, 27 июл. 2020 г. в 03:17, Mat : > >> Tested with Windows and Linux K8 containers all on Java 11. >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> > -- <http://www.trim

Re: Are CPU Metrics logged by Ignite correct?

2020-07-24 Thread Raymond Wilson
; I have tested 2.8.1, 2.8.0 and 2.7.0 as well as multiple environment and >> bottom line CPU always show negative values that cannot be correlated with >> the real cpu usage. Am I missing something? >> >> >> >> -- >> Sent from: http://apache-ignite-users.705

  1   2   3   >