Re: Another replicated cache oddity

2023-11-22 Thread Raymond Wilson
in in any of the Ignite servers' monitoring > metrics when these events happen? What kind of accumulated workload is on > the cluster at those times, and how often does this happen? > > > On Wed, Nov 22, 2023 at 1:50 PM Raymond Wilson > wrote: > >> Hi Jeremy, &g

Re: Another replicated cache oddity

2023-11-22 Thread Raymond Wilson
ation in play? What does the code look like for > the updates to the replicated cache, and what does the code look like for > the distributed compute operation? > > On Tue, Nov 21, 2023 at 5:21 PM Raymond Wilson > wrote: > >> Hi, >> >> We have been triagi

Another replicated cache oddity

2023-11-21 Thread Raymond Wilson
cluster restart? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>

Graceful shutdown of DotNet Ignite nodes running in Kubernetes pods

2023-09-06 Thread Raymond Wilson
yment context, how are you ensuring the Ignite node implementation gets the SIGTERM and shuts down gracefully? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand ra

Re: Cache write synchonization with replicated caches

2023-09-04 Thread Raymond Wilson
> Does this imply risk if run against a production environment grid? > > It depends. > As for read repair, CHECK_ONLY is read only mode and can’t harm your data. > Other modes that fix data inconsistency was used on our production and > there are no known issues. > >

Re: Possible bug failing to create a cache on a running grid causing grid failure

2023-08-28 Thread Raymond Wilson
FYI, I raised Jira ticket https://issues.apache.org/jira/browse/IGNITE-20299 for this. On Mon, Aug 28, 2023 at 3:42 PM Raymond Wilson wrote: > We have tried the same renaming in the dev environment which has multiple > server nodes impacted and contains some data (unlike the local

Re: Possible bug failing to create a cache on a running grid causing grid failure

2023-08-27 Thread Raymond Wilson
Raymond Wilson wrote: > Looking at the cache-SiteModelMetaData folder in the persistence folder > for a server node shows a "cache_data" file 6kb in size. No other cache > folders contain this file. > > As an experiment I renamed this file to "cache_dataxxx". This ap

Re: Possible bug failing to create a cache on a running grid causing grid failure

2023-08-27 Thread Raymond Wilson
the grid to restart. Similarly renaming the cache folder to "xxxcache-SiteModelMetaData" also permitted the grid to restart; we will be testing this further to verify. Raymond. On Sun, Aug 27, 2023 at 5:20 PM Raymond Wilson wrote: > I have reproduced the possible bug I reported in my e

Re: Possible bug failing to create a cache on a running grid causing grid failure

2023-08-26 Thread Raymond Wilson
point the grid is now unusable. To summarise: Attempted creation of a cache with an unknown DataRegionName causes immediate and unrecovered failure in the entire grid. Raymond. On Fri, Aug 25, 2023 at 7:47 PM Raymond Wilson wrote: > We believe we had some code on a dev environment attem

Possible bug failing to create a cache on a running grid causing grid failure

2023-08-25 Thread Raymond Wilson
We believe we had some code on a dev environment attempt to create a cache that was intended for another Ignite. The creation of this cache would have failed (at least) because the data region referenced in the cache configuration does not exist on that environment. A subsequent restart of the

Re: Cache write synchonization with replicated caches

2023-08-21 Thread Raymond Wilson
[2] > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/cache/ReadRepairStrategy.java > > > > 21 авг. 2023 г., в 07:46, Raymond Wilson > написал(а): > > [Replying onto correct thread] > > As a follow up to this email, we are starting t

Re: Cache write synchonization with replicated caches

2023-08-20 Thread Raymond Wilson
, Raymond. On Thu, Jul 27, 2023 at 12:38 PM Raymond Wilson wrote: > Hi, > > I have a query regarding data safety of replicated caches in the case of > hard failure of the compute resource but where the storage resource is > available when the node returns. > > We are using Ignite 2.

Re: Cache write synchronization mode

2023-08-20 Thread Raymond Wilson
:37 AM Raymond Wilson wrote: > I have a query regarding the CacheWriteSynchronizationMode in > CacheConfiguration. > > This enum is defined like this in the .Net client: > > public enum CacheWriteSynchronizationMode > { > /// > /// Mode indicating that I

Re: Simple question relating to multiple data regions with persistence

2023-08-13 Thread Raymond Wilson
Actually, ignore that question. I was conflating storage and data region configurations so my question is invalid! On Mon, Aug 14, 2023 at 11:40 AM Raymond Wilson wrote: > I am looking at defining additional data regions with persistence to > support cache with some different settings (eg

Simple question relating to multiple data regions with persistence

2023-08-13 Thread Raymond Wilson
different storage folders for each data region for data and WAL files, or whether Ignite is happy to have all data regions using the same base storage path. Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham

Re: Cache write synchronization mode

2023-07-26 Thread Raymond Wilson
- could you ask about this > specific case in a separate thread? > > On Tue, Jul 25, 2023 at 8:50 AM Raymond Wilson > wrote: > >> >> However, if a primary node fails before at least 1 backup >> node receives an update, then the update will be lost, and all nodes w

Cache write synchonization with replicated caches

2023-07-26 Thread Raymond Wilson
re is an improvement in write safety in this instance, does this imply the Fsync WalMode write pathway has opportunities for data loss in these failure situations? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software

Re: Cache write synchronization mode

2023-07-24 Thread Raymond Wilson
all nodes will have the old value. > > Related: CacheConfiguration.ReadFromBackup property is true by default, > meaning that with PrimarySync it is possible to get old value from a backup > node after an update, before backups receive new data. > > On Mon, Jul 24, 2023 at 11:51 PM R

Re: Ignite data region off-heap allocation

2023-07-24 Thread Raymond Wilson
llocated > only when the first cache is created in that region (when true), or > immediately (when false) > > > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/DataRegionConfiguration.html#isLazyMemoryAllocation-- > > > On Wed, Jul 19, 2023 at

Re: Cache write synchronization mode

2023-07-24 Thread Raymond Wilson
Hi Pavel, I understand the differences between the sync modes in terms of when the write returns. What I want to understand is if there are consistency risks with the PrimarySync versus FullSync modes. For example, if I have 4 nodes participating in the replicated cache (and am using the default

Re: Possible WAL corruption on running system during K8s update

2023-07-19 Thread Raymond Wilson
Jul 2023 at 10:02 AM, Raymond Wilson wrote: > Hi Alex, > > Here is the log from the Ignite startup. It's fairly short but shows > everything I think: > > 2023-07-17 22:38:55,061 [1] DBG [ImmutableCacheComputeServer] Starting > Ignite.NET 2.15.0.23172 > 2023-07

Re: Ignite data region off-heap allocation

2023-07-19 Thread Raymond Wilson
Just FYI, we have held off any memory pressure changes in the meantime while we continue to investigate the memory issues we have. On Tue, 18 Jul 2023 at 9:07 AM, Raymond Wilson wrote: > Hi Pavel, > > This area is confusing. There is no indication that the memory pressure > ap

Cache write synchronization mode

2023-07-18 Thread Raymond Wilson
on and ran into this item. Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>

Re: Possible WAL corruption on running system during K8s update

2023-07-18 Thread Raymond Wilson
ack trace --- at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg) Thanks, Raymond. On Wed, Jul 19, 2023 at 5:43 AM Raymond Wilson wrote: > Hi Alex, > > We are using Ignite v2.15. > > I will track down the additional log information and reply on this thread. >

Re: Possible WAL corruption on running system during K8s update

2023-07-18 Thread Raymond Wilson
tion during start processors, > node will be stopped and close connections" (there should be a reason in > the log, why the page delta can't be applied). > > вт, 18 июл. 2023 г. в 05:05, Raymond Wilson : > >> Hi, >> >> We run a dev/alpha stack of our application in Azure

Possible WAL corruption on running system during K8s update

2023-07-17 Thread Raymond Wilson
are every 30-45 seconds or so, so this won't be significant). Is this an error anyone else has noticed? Has anyone else had similar issues with Azure Files when using strict update/sync semantics? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer,

Re: Ignite data region off-heap allocation

2023-07-17 Thread Raymond Wilson
vailable RAM. > > [1] > https://stackoverflow.com/questions/1149181/what-is-the-point-of-using-gc-addmemorypressure-with-an-unmanaged-resource > [2] > https://ignite.apache.org/docs/latest/memory-configuration/data-regions#configuring-default-data-region > > On Tue, Jul 11, 2

Re: Ignite data region off-heap allocation

2023-07-11 Thread Raymond Wilson
How do Ignite .Net server nodes manage this memory issue in other projects? On Tue, Jul 11, 2023 at 5:32 PM Raymond Wilson wrote: > Oops, commutes => committed > > On Tue, 11 Jul 2023 at 4:34 PM, Raymond Wilson > wrote: > >> I can’t see another way of letting . Net

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
Oops, commutes => committed On Tue, 11 Jul 2023 at 4:34 PM, Raymond Wilson wrote: > I can’t see another way of letting . Net know that it can’t have access to > all the ‘free’ memory in the process when a large slab of that is spoken > for in terms of memory commutes to Ignite

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
yPressure documentation [1] states that this will "improve > performance only for types that exclusively depend on finalizers". > > [1] > https://learn.microsoft.com/en-us/dotnet/api/system.gc.addmemorypressure?view=net-7.0 > > On Tue, Jul 11, 2023 at 1:02 AM Raymond Wilson &g

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
of any? Raymond. On Mon, Jul 10, 2023 at 9:41 PM Raymond Wilson wrote: > Thanks Pavel, this makes sense. > > Querying the .Net Process instance shows this as the difference between > PagesMemorySize (includes committed) versus WorkingSet (includes > uses/written to) size. > Raymo

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
Thanks Pavel, this makes sense. Querying the .Net Process instance shows this as the difference between PagesMemorySize (includes committed) versus WorkingSet (includes uses/written to) size. Raymond.

Re: Ignite data region off-heap allocation

2023-07-10 Thread Raymond Wilson
committed-allocated-memory-in-linux-is-less-than-used-memory-how-is-that-possib > - > https://unix.stackexchange.com/questions/137773/is-inactive-memory-related-to-the-commited-but-unused > > On Sat, Jul 8, 2023 at 11:44 AM Raymond Wilson > wrote: > >> Hi, >> >>

Ignite data region off-heap allocation

2023-07-08 Thread Raymond Wilson
it actually allocated. I feel I am missing something here! Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.tr

Pending critical volnerabilities for OpenSSL and Apache Commons Text

2022-10-30 Thread Raymond Wilson
s made available. Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>

Re: Ignite Async API call continutation behaviour

2022-10-03 Thread Raymond Wilson
Thanks Pavel! :) On Mon, Oct 3, 2022 at 6:50 PM Pavel Tupitsyn wrote: > Hi Raymond, > > Yes, you can remove those ContinueWith() calls. Default behavior is now > safe. > > On Mon, Oct 3, 2022 at 2:25 AM Raymond Wilson > wrote: > >> Jay, >> >> Th

Re: Ignite Async API call continutation behaviour

2022-10-02 Thread Raymond Wilson
t; thread. > > When doing `await`, we most likely end up on another thread and lose > transaction context. > > > > There were plans to make transactions work across multiple threads, but > the ticket looks abandoned: > > https://issues.apache.org/jira/browse/IGNIT

Ignite Async API call continutation behaviour

2022-10-02 Thread Raymond Wilson
ing on a .Net threadpool thread by default. Is that correct? This will allow us to remove the (quite a few) .ContinueWith() statements which do add a layer of overhead on async calls. Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Con

Re: Re[2]: Checkpointing threads

2022-09-07 Thread Raymond Wilson
checkpointing threads appropriate, or are we likely to see > better performance if we increased it to 8 (or more)? > > If there is a discussion related to this a pointer to it would be good > (it's not really covered in the performance tuning section). > > Thanks, > Raymond. > >

Re: Checkpointing threads

2022-09-02 Thread Raymond Wilson
rmance if we increased it to 8 (or more)? > > If there is a discussion related to this a pointer to it would be good > (it's not really covered in the performance tuning section). > > Thanks, > Raymond. > > -- > <http://www.trimble.com/> > Raymond Wilson > Trimble

Checkpointing threads

2022-09-01 Thread Raymond Wilson
performance if we increased it to 8 (or more)? If there is a discussion related to this a pointer to it would be good (it's not really covered in the performance tuning section). Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction So

Re: Checkpointing is taking long time

2022-04-18 Thread Raymond Wilson
ype=default, persistence=true, lazyAlloc=true, > > > > ... initCfg=24576MB, maxCfg=24576MB, usedRam=1018MB, freeRam=95.86%, > > > > allocRam=24576MB, allocTotal=3820MB] > > > > ^-- metastoreMemPlc region [type=internal, persistence=true, > > > > lazyAl

Re: Running Apache Ignite on .Net 6

2022-01-20 Thread Raymond Wilson
/compatibility/6.0 > > On Wed, Jan 19, 2022 at 11:56 PM Raymond Wilson < > raymond_wil...@trimble.com> wrote: > >> Has anyone tried running Apache Ignite on .Net 6? >> >> I understand it runs on .Net 5 from Pavel's blog ( >> https://ptupitsyn.github.io/Ignite

Running Apache Ignite on .Net 6

2022-01-19 Thread Raymond Wilson
2.8 and make use of the latest .Net platform as well. Given .Net 6 is LTS, I'd prefer going there directly... Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Z

Re: Native Persistence Tuning - Percentage kept in RAM

2021-06-02 Thread Raymond Wilson
an in to an IgniteOutOfMemoryException, despite the fact there was > plenty of space on disk. Why would it run out of RAM if it can persist data > above the MaxSize the excess to disk? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > > > -- <

Issue building Ignite 2.10 branch

2021-05-09 Thread Raymond Wilson
f :ignite-jta Is there an easy work around for this? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimb

Re: Poor Performance with Increasing Public Thread Pool Size

2021-04-26 Thread Raymond Wilson
gt; } > > > > > > public class LongRunning_DotNet_Task : IComputeFunc > > { > > public bool Invoke(string[] arg) > > { > > // do work that takes up to 5 minutes with 12 threads, but > takes 50 minutes with 48 threads! > > return true; > >} > > } > > > -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>

Re: Async operations in IA C# client appear dangeroud

2021-03-11 Thread Raymond Wilson
Just as a followup query: Are async Compute methods susceptible to a similar issue, and should have the same treatment using a ContinueWith()? Thanks, Raymond. On Thu, Mar 11, 2021 at 8:45 PM Raymond Wilson wrote: > Hi Pavel, > > Thanks for the quick response. > > W

Re: Async operations in IA C# client appear dangeroud

2021-03-10 Thread Raymond Wilson
nd > verbose. > > > There should be a global Ignite setting to move all async continuations > away from the striped pool. > I'm taking the ticket [1] ASAP, it is a shame that we let it sit for so > long. > > [1] https://issues.apache.org/jira/browse/IGNITE-12033 > > O

Re: Async operations in IA C# client appear dangeroud

2021-03-09 Thread Raymond Wilson
Title correction: Async operations in IA C# client appear dangerous On Wed, Mar 10, 2021 at 6:05 PM Raymond Wilson wrote: > We are using IA 2.8.1 with the C# client. > > After triaging intermittent critical thread blockages we determined we > have run into this problem reported

Async operations in IA C# client appear dangeroud

2021-03-09 Thread Raymond Wilson
this be deprecated in the C# client until the underlying issues are resolved? It is hard to see how any non-trivial C# client based Ignite application can safely use them. Regards, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CC

IA 2.8.1 C# Client ICacheAffinity question

2021-03-03 Thread Raymond Wilson
le through the ICacheAffinity interface. The MapPartitionToPrimaryAndBackups(int part) looks like it would provide the required information, but would need to be called for every partition in the cache, which feels like it might be slow. Thoughts? Thanks, Raymond. -- <http://www.trimble.com/>

Re: Graceful shutdown and request draining of Ignite servers

2021-02-18 Thread Raymond Wilson
Ilya Kasnacheev wrote: > Hello! > > This sounds like a too detailed and peculiar scenario that should be taken > care of on the application level, as you already do. > > Regards, > -- > Ilya Kasnacheev > > > ср, 17 февр. 2021 г. в 23:50, Raymond Wilson : > >>

Re: Graceful shutdown and request draining of Ignite servers

2021-02-17 Thread Raymond Wilson
the grid after ensuring the active requests it is involved in can complete normally. In the case above, node B would be a client node providing services through a web api gateway (A) and requesting results from co-located processing on node C. Thanks, Raymond. On Thu, Feb 18, 2021 at 9:15 AM Raymon

Re: Graceful shutdown and request draining of Ignite servers

2021-02-17 Thread Raymond Wilson
wrote: > Hello! > > Why can't you just use Ignite.stop(instanceName, false)? > > Just make sure your projections are not singleton and the tasks will be > rolled over. > > Regards, > -- > Ilya Kasnacheev > > > вт, 9 февр. 2021 г. в 06:41, Raymond Wilson :

Graceful shutdown and request draining of Ignite servers

2021-02-08 Thread Raymond Wilson
proceed with actual shut down of that node. I have not seen any capability in Ignite today where a node can be 'un-blessed'; does one exist? Or should we construct this facility within our application logic layer? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Arc

Re: Detecting checkpoints programmatically

2021-01-28 Thread Raymond Wilson
SAVED, > EVT_CHECKPOINT_LOADED, > EVT_CHECKPOINT_REMOVED > }; > > Are these the appropriate events to listen to? > > Thanks, > Raymond. > > -- > <http://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Construction Sof

Detecting checkpoints programmatically

2021-01-28 Thread Raymond Wilson
to? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>

Re: Ever increasing startup times as data grow in persistent storage

2021-01-28 Thread Raymond Wilson
read in the Ignite developer list: > > http://apache-ignite-developers.2346864.n4.nabble.com/ > > BR, > Andrei > > > 1/20/2021 9:16 PM, Raymond Wilson пишет: > > Hi Andre, > > I would like to see Ignite support a graceful shutdown scenario you get > with deac

Re: Ever increasing startup times as data grow in persistent storage

2021-01-20 Thread Raymond Wilson
drei > 1/20/2021 5:50 AM, Raymond Wilson пишет: > > We have been experimenting with using deactivation to shutdown the grid to > reduce the time for the grid to start up again. > > It appears there is a downside to this: once deactivated the grid does not > appear to auto-activ

Re: Ever increasing startup times as data grow in persistent storage

2021-01-19 Thread Raymond Wilson
Cluster.ICluster.html?#Apache_Ignite_Core_Cluster_ICluster_SetActive_System_Boolean_ > > On Wed, Jan 13, 2021 at 11:54 AM Raymond Wilson < > raymond_wil...@trimble.com> wrote: > >> Of course. Obvious! :) >> >> Sent from my iPhone >> >> On 13/01/2021, at 9:15 PM, Zhenya Stanilovsky wrote: >>

Re: Ignite best practice for restarting k8s pod

2021-01-15 Thread Raymond Wilson
ache.org/jira/browse/IGNITE-13974 >> >> >> >> >> >> Currently I do not see any cleanup functions getting called when we do a >> >> 'kubectl delete pod'. >> >> >> >> May I know, what is the best practice for restarting k8s ignite pod

Re: Ever increasing startup times as data grow in persistent storage

2021-01-13 Thread Raymond Wilson
tion read while applying WAL changes? don`t think so, may be someone else suggest here? Does anyone else have this issue? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zeal

Re: Re[2]: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
Solution Architect, Civil Construction Software Systems (CCSS) > 11 Birmingham Drive | Christchurch, New Zealand > raymond_wil...@trimble.com > <http://e.mail.ru/compose/?mailto=mailto%3araymond_wil...@trimble.com> > > > > <https://worksos.trimble.com/?utm_source=Trimble_m

Re: Re[2]: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
ng pointing intervals help? > > > yes, it will help. Check > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood > > Is the entire content of a partition read while applying WAL changes? > > > don`t think so, may be someone else sug

Re: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
eck > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood > > Is the entire content of a partition read while applying WAL changes? > > > don`t think so, may be someone else suggest here? > > Does anyone else have this issue? > > Thanks

Re: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
We are currently using AI 2.8.1 with the c# client. On Wed, Jan 13, 2021 at 8:12 PM Kirill Tkalenko wrote: > Hello, Raymond! What version are you using? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- <http://www.trimble.com/> Raymon

Re: Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
active > Regarding the no of partitions also, 128 partitions should do and its doing > good for us > > Thanks > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Co

Ever increasing startup times as data grow in persistent storage

2021-01-12 Thread Raymond Wilson
applying WAL changes? Does anyone else have this issue? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?u

Re: Difference between Stopping and Closing Ignite on C#

2021-01-12 Thread Raymond Wilson
Apologies - I misread the description of Stop(). Please ignore this thread. On Wed, Jan 13, 2021 at 3:50 PM Raymond Wilson wrote: > I've just been reading this article ( > https://stackoverflow.com/questions/52780726/how-to-trigger-ignite-checkpoint-before-shutdown) > which

Difference between Stopping and Closing Ignite on C#

2021-01-12 Thread Raymond Wilson
during shutdown. Thanks Raymond -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>

Re: Questions related to check pointing

2021-01-12 Thread Raymond Wilson
; to be the same number? > > 2. We have two data regions, one supporting inbound arrival of data (with > low numbers of writes), and one supporting storage of processed results > from the arriving data (with many more writes). > > The block on writes due to the number of dirty pages a

Re: Re[4]: Questions related to check pointing

2021-01-06 Thread Raymond Wilson
t 11:39 PM Ilya Kasnacheev wrote: > Hello! > > I guess it's pool.pages() * 3L / 4 > Since, counter intuitively, the default ThrottlingPolicy is not > ThrottlingPolicy.DISABLED. It's CHECKPOINT_BUFFER_ONLY. > > Regards, > > -- > Ilya Kasnacheev > > > чт, 31 де

Re: Moving caches between data regions

2021-01-01 Thread Raymond Wilson
Once the cache has been moved to another region, can the region it was held in be removed (it is the only cache in that region)? On Thu, Dec 31, 2020 at 11:57 PM Raymond Wilson wrote: > Hi Ilya, > > Thanks for confirming. > > Raymond. > > Sent from my iPhone > >

Re: Moving caches between data regions

2020-12-31 Thread Raymond Wilson
these data regions is this > possible by changing the configuration of the caches to be moved so they > all point to the same data region? > > Thanks, > Raymond. > > -- > <http://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Construction Softwa

Moving caches between data regions

2020-12-31 Thread Raymond Wilson
by changing the configuration of the caches to be moved so they all point to the same data region? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimb

Re: Re[4]: Questions related to check pointing

2020-12-30 Thread Raymond Wilson
maximum sizes of data regions > to be the same number? > > 2. We have two data regions, one supporting inbound arrival of data (with > low numbers of writes), and one supporting storage of processed results > from the arriving data (with many more writes). > > The block on

Re: Re[4]: Questions related to check pointing

2020-12-30 Thread Raymond Wilson
ximum sizes of data regions > to be the same number? > > 2. We have two data regions, one supporting inbound arrival of data (with > low numbers of writes), and one supporting storage of processed results > from the arriving data (with many more writes). > > The block on writes due

Re: Re[2]: Questions related to check pointing

2020-12-30 Thread Raymond Wilson
t can be improved? Thanks, Raymond. On Wed, Dec 30, 2020 at 9:17 PM Raymond Wilson wrote: > I'm working on getting automatic JVM thread stack dumping occurring if we > detect long delays in put (PutIfAbsent) operations. Hopefully this will > provide more information. > > On Wed, D

Re: Re[2]: Questions related to check pointing

2020-12-30 Thread Raymond Wilson
cket for this in the 2.9.1 > Jira area at > https://issues.apache.org/jira/browse/IGNITE-13876?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.9.1%20and%20status%20%3D%20Resolved > > Perhaps this change may improve the checkpointing issue we are seeing? > > Raymond. &

Re: Questions related to check pointing

2020-12-29 Thread Raymond Wilson
%20and%20status%20%3D%20Resolved Perhaps this change may improve the checkpointing issue we are seeing? Raymond. On Tue, Dec 29, 2020 at 8:35 PM Raymond Wilson wrote: > Hi Zhenya, > > 1. We currently use AWS EFS for primary storage, with provisioned IOPS to > provide sufficient IO

Re: Ignite thread pools

2020-12-29 Thread Raymond Wilson
:33, Raymond Wilson : > Hi, > > I have been drilling into some of the Ignite thread pool support and have > found references in the code to three thread pools that don't appear to be > referenced in the Ignite thread pool documentation at > https://apacheignite.readme.io

Re: Feature request: On demand thread dumps from Ignite

2020-12-28 Thread Raymond Wilson
) Otherwise, is > this something worth adding to the backlog? > > Thanks, > Raymond. > > -- > <http://www.trimble.com/> > Raymond Wilson > Solution Architect, Civil Construction Software Systems (CCSS) > 11 Birmingham Drive | Christchurch, New Zealand > +64

Re: Questions related to check pointing

2020-12-28 Thread Raymond Wilson
It seems like the checkpoint is affecting the put operations, but I don't > understand why that may be given the documented checkpointing process, and > the checkpoint itself (at least via Informational logging) is not > advertising any restrictions. > > Thanks, > Raymond. > > -

Feature request: On demand thread dumps from Ignite

2020-12-28 Thread Raymond Wilson
. If this is actually the case then please point me at it:) Otherwise, is this something worth adding to the backlog? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand +64-21-2013317

Re: Questions related to check pointing

2020-12-28 Thread Raymond Wilson
As another detail, we have the WriteThrottlingEnabled property left at its default value of 'false', so I would not ordinarily expect throttling, correct? On Tue, Dec 29, 2020 at 10:04 AM Raymond Wilson wrote: > Hi Ilya, > > Regarding the throttling question, I have not yet looked

Re: Questions related to check pointing

2020-12-28 Thread Raymond Wilson
en them. > > 3. I don't think that default throttling mechanism will emit any warnings. > What do you see in thread dumps? > > Regards, > -- > Ilya Kasnacheev > > > ср, 23 дек. 2020 г. в 12:48, Raymond Wilson : > >> Hi, >> >> We have been investigat

Ignite thread pools

2020-12-27 Thread Raymond Wilson
of the thread pools themselves. Can the thread pool documentation be extended to include these additional pools, particularly with respect to their purpose and roles, and the circumstances when you would change them from the default value. Thanks, Raymond. -- <http://www.trimble.com/> Raymond

Questions related to check pointing

2020-12-23 Thread Raymond Wilson
like the checkpoint is affecting the put operations, but I don't understand why that may be given the documented checkpointing process, and the checkpoint itself (at least via Informational logging) is not advertising any restrictions. Thanks, Raymond. -- <http://www.trimble.com/> Raymond

Re: C#/.NET: Possible to assign different nodes different roles?

2020-12-10 Thread Raymond Wilson
: http://apache-ignite-users.70518.x6.nabble.com/ > -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand +64-21-2013317 Mobile raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>

Re: Binary memory restoration

2020-10-08 Thread Raymond Wilson
gt; вт, 6 окт. 2020 г. в 02:10, Raymond Wilson : > >> Thanks for the thoughts Ilya and Vladimir. >> >> We'll do a comparison with 2.9 when it releases to see if that makes any >> difference. >> >> One of the advantages with persistent storage is that it i

Re: Continuous Query

2020-10-07 Thread Raymond Wilson
t; - >>>>>>>>>> Denis >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Oct 2, 2020 at 11:22 AM narges saleh < >>>>>>>>>> snarges...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> The detection should happen at most a couple of minutes after a >>>>>>>>>>> record is inserted in the cache but all the detections are local to >>>>>>>>>>> the >>>>>>>>>>> node. But some records with the current timestamp might show up in >>>>>>>>>>> the >>>>>>>>>>> system with big delays. >>>>>>>>>>> >>>>>>>>>>> On Fri, Oct 2, 2020 at 12:23 PM Denis Magda >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> What are your requirements? Do you need to process the records >>>>>>>>>>>> as soon as they are put into the cluster? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Friday, October 2, 2020, narges saleh >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Thank you Dennis for the reply. >>>>>>>>>>>>> From the perspective of performance/resource overhead and >>>>>>>>>>>>> reliability, which approach is preferable? Does a continuous >>>>>>>>>>>>> query based >>>>>>>>>>>>> approach impose a lot more overhead? >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Oct 2, 2020 at 9:52 AM Denis Magda >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Narges, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Use continuous queries if you need to be notified in >>>>>>>>>>>>>> real-time, i.e. 1) a record is inserted, 2) the continuous >>>>>>>>>>>>>> filter confirms >>>>>>>>>>>>>> the record's time satisfies your condition, 3) the continuous >>>>>>>>>>>>>> queries >>>>>>>>>>>>>> notifies your application that does require processing. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The jobs are better for a batching use case when it's ok to >>>>>>>>>>>>>> process records together with some delay. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> - >>>>>>>>>>>>>> Denis >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Oct 2, 2020 at 3:50 AM narges saleh < >>>>>>>>>>>>>> snarges...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>> If I want to watch for a rolling timestamp pattern in all >>>>>>>>>>>>>>> the records that get inserted to all my caches, is it more >>>>>>>>>>>>>>> efficient to use >>>>>>>>>>>>>>> timer based jobs (that checks all the records in some interval) >>>>>>>>>>>>>>> or >>>>>>>>>>>>>>> continuous queries that locally filter on the pattern? These >>>>>>>>>>>>>>> records can >>>>>>>>>>>>>>> get inserted in any order and some can arrive with delays. >>>>>>>>>>>>>>> An example is to watch for all the records whose timestamp >>>>>>>>>>>>>>> ends in 50, if the timestamp is in the format -mm-dd hh:mi. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> thanks >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> - >>>>>>>>>>>> Denis >>>>>>>>>>>> >>>>>>>>>>>> -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand +64-21-2013317 Mobile raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>

Re: Binary memory restoration

2020-10-05 Thread Raymond Wilson
on query entity configured. > As far as I can see it's going to be a part of the Ignite 2.9 release. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems

Re: Binary memory restoration

2020-09-28 Thread Raymond Wilson
rrently we have around 10 caches with less than 20 Gb of data. Do these startup times seem reasonable for a node that is shut down gracefully? Thanks, Raymond. On Tue, Sep 29, 2020 at 11:20 AM Raymond Wilson wrote: > Just a general query on Ignite server node startup sequence. > > There are a

Binary memory restoration

2020-09-28 Thread Raymond Wilson
2.8.1 using the C# client with persistence enabled. Do these messages indicate Ignite is reloading the memory state of the caches at the point it was shutdown? If not, what is this activity? Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Constr

Failing to update metadata for type

2020-08-23 Thread Raymond Wilson
to Ignite configuration on startup, eg: cfg.BinaryConfiguration = new BinaryConfiguration { Serializer = new BinarizableSerializer() }; Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CC

Re: Ignite C# client does not report correct node ID after reconnection

2020-08-18 Thread Raymond Wilson
bug report. > I've filed a ticket [1]. > > I'll see if that's possible to squeeze the fix into the 2.9 release, > otherwise you can expect it in 2.10. Will keep you posted. > > [1] https://issues.apache.org/jira/browse/IGNITE-13369 > > On Tue, Aug 18, 2020 at 10:02 AM Ray

Ignite C# client does not report correct node ID after reconnection

2020-08-18 Thread Raymond Wilson
l node that may be at risk of being inconsistent if the underlying grid information changes. Thanks, Raymond. -- <http://www.trimble.com/> Raymond Wilson Solution Architect, Civil Construction Software Systems (CCSS) 11 Birmingham Drive | Christchurch, New Zealand +64-21-2013317 Mobile

Re: Are CPU Metrics logged by Ignite correct?

2020-07-28 Thread Raymond Wilson
IGNITE-13306 > > Evgenii > > пн, 27 июл. 2020 г. в 03:17, Mat : > >> Tested with Windows and Linux K8 containers all on Java 11. >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> > -- <http://www.trimble.c

Re: Are CPU Metrics logged by Ignite correct?

2020-07-24 Thread Raymond Wilson
ave tested 2.8.1, 2.8.0 and 2.7.0 as well as multiple environment and >> bottom line CPU always show negative values that cannot be correlated with >> the real cpu usage. Am I missing something? >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabb

  1   2   3   >