[ https://issues.apache.org/jira/browse/IGNITE-13912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270170#comment-17270170 ]
shivakumar commented on IGNITE-13912: ------------------------------------- Hi [~ktkale...@gridgain.com] I have attached the reproducer, when you unzip reproducer.zip then there will be ignite-config.xml file which is used to install ignite cluster. In my setup I deployed 5 node Ignite cluster with this configuration on kubernetes. In this config file you can disable SSL, change ipFinder (ex: to vm ipFinder if you install on Virtual machine). The JVM OPTS set is *-server -Xms32g -Xmx32g -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC* In the same reproducer there is a client program(inventory_data.zip) which creates tables and ingest data, here is the steps to run the client program once you installed the ignite cluster. *unzip inventory_data.zip* *cd inventory_data/* *mvn clean package* open this config file and update ignite_db_url *vim src/main/resources/config.properties* The config file look like this {code:java} ignite_jdbc_driver = org.apache.ignite.IgniteJdbcThinDriver ignite_db_url = jdbc:ignite:thin://ignite-service.default.svc.cluster.local:10800 ignite_db_url_parameters = distributedJoins=true ignite_user = ignite ignite_pass = ignite generate_rows = 1000000 {code} To run the program(This will connect to Ignite cluster and creates table and ingest one million records) *java -cp target/inventory-1.0-SNAPSHOT.jar:target/libs/* com.test.app.InventoryCreate src/main/resources/config.properties yes* To continuously start data ingestion, call the same program in loop with 2nd argument set to "*no*" (This argument will skip table deletion and re-creation and directly starts data ingestion) *while true ; do java -cp target/libs/*:target/inventory-1.0-SNAPSHOT.jar com.test.app.InventoryCreate src/main/resources/config.properties no; sleep 2s; done* Run this program for at-least 40 to 50 minutes and connect to visor shell in between data ingestion (like every 5 minutes connect to visor shell and run *cache -a* or *cache* command and be in visor for 2 minute and disconnect) To connect to visor shell I'm using the same Ignite config file which is used for server start. *./ignitevisorcmd.sh -cfg=/opt/ignite/conf/ignite-config.xml* Monitor the WAL usage and see if you can re-produce. Please let me know if you need any help. Regards, Shiva > Incorrect calculation of WAL segments that should be deleted from WAL archive > ----------------------------------------------------------------------------- > > Key: IGNITE-13912 > URL: https://issues.apache.org/jira/browse/IGNITE-13912 > Project: Ignite > Issue Type: Bug > Components: persistence > Reporter: Kirill Tkalenko > Assignee: Kirill Tkalenko > Priority: Critical > Fix For: 2.10 > > Attachments: reproducer.zip, server1-full-wal-checkpoint.log, > wal-checkpoint-logs, wal_dir_contents, wal_grows_from_peak.PNG, > wal_issue_reproduced.PNG, wal_usage.PNG, wal_usage_dec12.PNG, > wal_usage_dec22nd_binary.PNG > > Time Spent: 40m > Remaining Estimate: 0h > > Now there is an incorrect calculation of WAL segments that should be deleted > from WAL archive. Since we delete only those segments whose total size should > not exceed *DataStorageConfiguration#maxWalArchiveSize * > IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE*, but should be up to > DataStorageConfiguration#maxWalArchiveSize * > IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE*. Therefore, an excess of > *DataStorageConfiguration#maxWalArchiveSize* occurs. -- This message was sent by Atlassian Jira (v8.3.4#803005)