Recurring index corruption

2023-01-02 Thread S S
We are experimenting with Elastic Search deployed in Azure Container Instances (Debian + OpenJDK). The ES indexes are stored into an Azure file share mounted via SMB (3.0). The Elastic Search cluster is made up of 4 nodes, each one have a separate file share to store the indices. This configur

Re: Recurring index corruption

2023-01-02 Thread S S
need to change > that setting even with larger clusters. > > Uwe > > Am 02.01.2023 um 11:18 schrieb S S: >> We are experimenting with Elastic Search deployed in Azure Container >> Instances (Debian + OpenJDK). The ES indexes are stored into an Azure file >> sha

Re: Recurring index corruption

2023-01-02 Thread S S
> documentation is not mandatory and misleading. > > If you use newest version of Elasticsearch with Java 19 and you use > `--enable-preview` in you jvm.properties file, you don't even need to change > that setting even with larger clusters. > > Uwe > > Am 02.01.

Re: Recurring index corruption

2023-01-02 Thread S S
> But in general, you can simply use MMapDirectory, the max-map-count setting > is only relevant in *huge* (huge means hundreds of huge indexes per node). In > that case Java 19's preview features would be recommended. > > Uwe > > Am 02.01.2023 um 17:41 schrieb S S

Re: Recurring index corruption

2023-01-02 Thread S S
eneral, you can simply use MMapDirectory, the max-map-count setting > is only relevant in *huge* (huge means hundreds of huge indexes per node). In > that case Java 19's preview features would be recommended. > > Uwe > > Am 02.01.2023 um 17:41 schrieb S S: >> H

Re: Recurring index corruption

2023-01-02 Thread S S
I can try? Many thanks, Seb > On 2 Jan 2023, at 17:55, S S wrote: > > Thank you Uwe, this is great! I am rebuilding the cluster using MMapDirectory > and no enable-preview, as you suggested. Let’s see what happens. > > Cheers, > > Seb > >> On 2 Jan 202

Re: Recurring index corruption

2023-01-02 Thread S S
w >1 GiB (but < 5 GiB) chunks (so lets assume 90 mappings needed for a > shard), so you could have approx 728 shards per node. Sorry raising this > setting is not needed! > > Uwe > > Am 02.01.2023 um 18:24 schrieb S S: >> I also tried enabling preview but no joy, sa

Re: Recurring index corruption

2023-01-03 Thread S S
not have implemented > it correctly in Linux kernel. > > This is the reason for the recommendation to not use network file systems. > You're example just confirmed this recommendation. > > Uwe > > Am 02.01.2023 um 19:24 schrieb S S: >> Hi Uwe, >> >>