Hi All,
We have an application that stores information on multiple users/customers/tenants in a common table. Each tenant has a unique id which we encode in the row key of the records that are stored in the table. We want to apply custom (and dynamically updatable) data retention policies for each tenant. What would be a reasonable way to achieve that? Searching through forums, I came across this link that suggests to either write an external process to retrieve and delete cells based on the retention policy or write a custom compaction policy: https://community.hortonworks.com/questions/14883/best-way-to-achieve-custom-retention-of-some-rows.html We felt that writing an external scanner for managing retention would be simpler but very inefficient as it would require getting the entire data set out of the hbase server and then issuing delete calls back to it. Does any one know if there has been any recent progress on this aspect of data retention in hbase? Additionally, if I go the route of writing my own custom compaction policy, what would be the best place to start? Maybe I could copy/extend the “default” hbase compaction policy and enhance it to look at rowkey inside every Cell to make a call if the cell needs to be deleted? -- cheers, gaurav
