Hi David! WAL <https://apacheignite.readme.io/docs/write-ahead-log> is a ring buffer on the filesystem. Each file in it is called a "segment". By default each segment occupies 64 MB on disk (*walSegmentSize* property), and there are 10 such files in the buffer (*walSegments* property). So, WAL for each node may grow up to ~640 MB.
But there is also a WAL archive. When a segment is full, it is copied to the archive. And the number of files in the archive depends on the *walHistSize *property, which is 20 checkpoints by default. It specifies the number of checkpoints, that the history should keep. Checkpoint <https://apacheignite.readme.io/docs/persistence-checkpointing> is a procedure of writing data to the persistent disk space. It is triggered when it didn't happen for a long time, or a lot of updates happened since the last checkpoint. So, you can tune these properties, and it may lead to disk usage decrease. But don't set any of these to very low values. You can find here how you can tune durable memory: https://apacheignite.readme.io/docs/durable-memory-tuning Note, that changing these values may lead to that nodes won't be able to read WAL segments, that are already stored on disk. If WAL directory grows up to 100GB, it means, that you either run many nodes on the same machine, or old unused WAL segments are still stored there. Check, that it doesn't contain WAL for nodes, that are not in the cluster. Denis вт, 21 нояб. 2017 г. в 19:20, David Wimsey <[email protected]>: > Hello all, > > I’m having an issue understanding how to control/limit the size of the WAL > as well as why I’m consuming so much WAL space. > > I’m currently migrating a project from redis to ignite and as part of the > first phase, its pretty much lift and shift without optimizing for ignite. > In this case, I’m passing in a handful of large objects to a cache, most of > these are Map<String, String> types, but one is a more complex structure. > All of these objects combined consume a few GB of RAM but the maps > themselves may contain tens of thousands of entries, though at this stage I > don’t care if ignite is aware of these objects, when I retrieve them from > the cache I simply pull out the full object (see code below). I realize > this is not ideal, and will be refactored to work more efficiently in the > next phase. > > private IgniteCache<String, Serializable> objectCache; > > public LookupDataCacheProvider(final Ignite igniteDataGrid) > { > this.igniteDataGrid = igniteDataGrid; > metadataCache = > igniteDataGrid.getOrCreateCache(LookupDataCacheKey.METADATA.getKey()); > objectCache = > igniteDataGrid.getOrCreateCache(OBJECT_CACHE_NAME).withKeepBinary(); > } > > public void setRateCenters(final RateCenters rateCenters) > { > > setJavaObject(LookupDataCacheKey.RATE_CENTERS, rateCenters); > } > > private <T extends Serializable> void setJavaObject(final LookupDataCacheKey > lookupDataCacheKey, final T value) { > > objectCache.put(lookupDataCacheKey.getKey(), value); > } > > private <T extends Serializable> T getJavaObject(final LookupDataCacheKey > lookupDataCacheKey, final Class<T> tClass) { > return tClass.cast(objectCache.get(lookupDataCacheKey.getKey())); > } > > > My problem is that when the largest of these maps get written to the > cache, it fills up the mount we have dedicated to persistent storage, > looking at the directory structure the majority of the space is in WAL > archives. > > We have a 100G mount for persistence, which we assumed would be plenty for > something that normally fits into a few GB of ram. > > So, can anyone point me at something I can use to calculate the size > needed for the WAL based on ignite settings or what settings I really care > about? Additionally, can anyone give me a general reason for why the WAL > may get so massive for such a small set of data (max 4G data in ram, 100gb > in WAL before it runs out of space). Is this just a case of using ignite > so wrong that it doesn’t work? If I were to make the Map<String, String> a > cache by itself rather than a object inside a cache, as a general rule > would that be more likely or less likely to increase WAL usage. > > What am I doing wrong that is consuming so much WAL? >
