maytasm commented on pull request #11078: URL: https://github.com/apache/druid/pull/11078#issuecomment-816355663
> > > I have a question. Users have no idea of the size of payload, are there any suggestions that tell users how to set this configuration ? Can we skip storing payload based on value of `type` ? > > > > > > In another PR, I am adding an auto cleanup based on size limit for the druid_audit metadata table. This work similar to druid.indexer.logs.kill.* which periodically clean up task entries in the druid_tasks table. Basically in my other PR, user will be able to configured the max number of entries they want to store in the druid_audit metadata table, and Druid will automatically remove from oldest entry (FIFO) when the table entries exceed this size limit. Using both of these configuration (this PR and the other PR), Druid operator will be able to control the size of the audit metadata table. For example, if you want to limit your audit table metadata to 20GB (i.e. that is how much storage you have on your machine), and say you want keep `x` number of last audit entries, you can then set `maxPayloadSizeBytes` to `20GB / x` > > This make sure that the audit table will not grow uncontrollably and bringing down / effecting your whole Druid cluster. > > That makes sense. Do you have any plan to put these explanation in doc by your another PR ? Yep. This is the other PR https://github.com/apache/druid/pull/11084. Right now it only has the doc change and the code implementation is coming soon. Please let me know if you think any docs is missing. I will add a section on how both configs can be use together to control the size of the metadata tables. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
