Hi Yong, Thanks for raising this! Cleaning up or resetting the test for the next run makes sense.
Other than using a delete and delete with purge, is resetting the persistence layer directly an option? It might be easier and faster. Object storage purging probably doesn't matter for the benchmark unless there are name conflicts. The tradeoff is that this approach needs direct operations on the persistence layer, which won't apply easily to different persistence layers. Yufei On Sun, Oct 11, 2026 at 8:38 AM Yong Zheng <[email protected]> wrote: > Here is the implementation: > https://github.com/apache/polaris-tools/pull/286 > > On 2026-10-08 05:52 UTC Yong Zheng wrote: > > Hello all, > > > > The current benchmark tool offers a list of simulations > (create/read/update) and it has the ability to skip creation as people make > change to the benchmark config (e.g. change from 15 tables to 20 tables, > only 5 new tables would be create on the second run of create dataset > simulation). > > > > There is one problem I am seeing with current tool is the lack of the > ability to perform cleanup after runs of benchmarks. When using > non-persisted backend, current tool is great. However, when we are actually > testing against a persistent backends (e.g. metadata and S3), lack of > ability to cleanup can make the re-run of benchmarks be in-accurate due to > larger metadata created on those tables after a set of simulations. Also, > if an operator put the wrong catalog properties and use the benchmark to > create dataset (which would create catalog), this will never move forward > (e.g. forgot to put roleArn for S3 catalog). In this case, the operator > will be responsible to do the catalog update via API or CLI tool or do the > cleanup manually then rerun with updated config. The lack of the ability to > perform cleanup will leave the cleanup tasks to the operators which add > some amount of overhead. > > > > I am thinking to add a delete simulation which does the opposite of > create dataset simulation (it will drop tables/views then ns and catalog > itself). The one catch I am seeing is polaris has DROP_WITH_PURGE_ENABLED > default to false while drop the view would need this to be true. Thus, I am > thinking just to document this change needed on server side before people > can use delete simulation. This also means, the drop table will be purge > instead of de-register. Thoughts? > > > > Thanks, > > Yong Zheng > > >
