I’m fine with the current solution. We could explore alternatives later.
Yufei On Mon, Oct 12, 2026 at 09:30 Jean-Baptiste Onofré <[email protected]> wrote: > Hi > > Cleanup after a run is a real gap, especially with a persistent backend, > leaving "orphan" metadata. It also can leave your catalog half-configured, > with stuff like missing roleArn, etc. > So, I think it's a welcome new feature :) > > If I agree it's faster to reset the persistence layer directly, I would > keep the API-based approach as the default. The reason is that it works > with any backend and keeps the benchmark a plain client of a running > server, I don't think it would be a good idea for benchmark to manipulate > directly the persistence layer (or at least it should be optional as it > would depend of the persistence backend). > > About DROP_WITH_PURGE_ENABLED prerequisite, I think turning on that flag > changes more than view drops. Every table drop becomes a purge, which > deletes data in object storage. If someone points the simulation on the > wrong catalog or a shared bucket, they could lose real data. > I would suggest to limit delete objects to the simulation mode, and also > requiring an explicit opt-in (let's not forget the documentation about > that). > > I'm not sure I understand why dropping a view needs purge enabled? If > that's a server-side weird stuff rather than intended behavior, it may be > better to fix on the server rather than documented as prerequisite. > > Also, last comment: cleanup should be safe to re-run (again and again :)). > If it fails partly, a second run should skip objects that are already gone > and carry on. > > I will take a look on the PR as well. > > Regards > JB > > On Thu, Oct 8, 2026 at 6:52 AM Yong Zheng <[email protected]> wrote: > > > Hello all, > > > > The current benchmark tool offers a list of simulations > > (create/read/update) and it has the ability to skip creation as people > make > > change to the benchmark config (e.g. change from 15 tables to 20 tables, > > only 5 new tables would be create on the second run of create dataset > > simulation). > > > > There is one problem I am seeing with current tool is the lack of the > > ability to perform cleanup after runs of benchmarks. When using > > non-persisted backend, current tool is great. However, when we are > actually > > testing against a persistent backends (e.g. metadata and S3), lack of > > ability to cleanup can make the re-run of benchmarks be in-accurate due > to > > larger metadata created on those tables after a set of simulations. Also, > > if an operator put the wrong catalog properties and use the benchmark to > > create dataset (which would create catalog), this will never move forward > > (e.g. forgot to put roleArn for S3 catalog). In this case, the operator > > will be responsible to do the catalog update via API or CLI tool or do > the > > cleanup manually then rerun with updated config. The lack of the ability > to > > perform cleanup will leave the cleanup tasks to the operators which add > > some amount of overhead. > > > > I am thinking to add a delete simulation which does the opposite of > create > > dataset simulation (it will drop tables/views then ns and catalog > itself). > > The one catch I am seeing is polaris has DROP_WITH_PURGE_ENABLED default > to > > false while drop the view would need this to be true. Thus, I am thinking > > just to document this change needed on server side before people can use > > delete simulation. This also means, the drop table will be purge instead > of > > de-register. Thoughts? > > > > Thanks, > > Yong Zheng > > >
