dosubot[bot] commented on issue #38605: URL: https://github.com/apache/superset/issues/38605#issuecomment-4053380956
<!-- Answer --> Great detective work narrowing it down to S3Cache specifically! That's really helpful. The key difference is that [SupersetMetastoreCache actively manages expiration](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/extensions/metastore_cache.py#L74-L78) by converting the timeout to an absolute datetime and storing it in the database — then checking it on retrieval. S3Cache, being an external library (`s3cache.s3cache.S3Cache`), is responsible for its own TTL enforcement, and it appears it's not properly checking expiration when fetching cached values. This is likely a bug in the `s3cache` library itself rather than Superset. The library may be: - Not storing TTL metadata with the S3 objects - Storing it but not checking it on retrieval - Relying entirely on S3 lifecycle policies (which as you noted aren't reliable for precise timing) If you're able to narrow it down further, it would be worth reporting upstream to the `s3cache` library maintainers. In the meantime, `SupersetMetastoreCache` is a solid alternative if your volume permits database-backed caching. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=brand-link).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/4b46eb73-6763-4c9c-9d7c-2cb3b8600eb9?feedback_type=great_response&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/4b46eb73-6763-4c9c-9d7c-2cb3b8600eb9?feedback_type=irrelevant_answer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/4b46eb73-6763-4c9c-9d7c-2cb3b8600eb9?feedback_type=incorrect_sources&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/4b46eb73-6763-4c9c-9d7c-2cb3b8600eb9?feedback_type=too_verbose&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/4b46eb73-6763 -4c9c-9d7c-2cb3b8600eb9?feedback_type=hallucination&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/4b46eb73-6763-4c9c-9d7c-2cb3b8600eb9?feedback_type=bug_report&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-bug_report) | [Other](https://app.dosu.dev/response-feedback/4b46eb73-6763-4c9c-9d7c-2cb3b8600eb9?feedback_type=other&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=chat-badge) [](https://cursor.com/link/prompt?text=Great%20detective%20work%20narrowing%20it%20down%20to%20S3Cache%20specifically%21%20That%27s%20really%20helpful.%0A%0AThe%20key%20difference%20is%20that%20%5BSupersetMetastoreCache%20actively%20manages%20expiration%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/extensions/metastore_cache.py%23L74-L78%29%20by%20converting%20the%20timeout%20to%20an%20absolute%20datetime%20and%20storing%20it%20in%20the%20database%20%E2%80%94%20then%20checking%20it%20on%20retrieval.%20S3Cache%2C%20being%20an%20external%20library%20%28%60s3cache.s3cache.S3Cache%60%29%2C%20is%20responsible%20for%20its%20own%20TTL%20enforcement%2C%20and%20it%20appe ars%20it%27s%20not%20properly%20checking%20expiration%20when%20fetching%20cached%20values.%0A%0AThis%20is%20likely%20a%20bug%20in%20the%20%60s3cache%60%20library%20itself%20rather%20than%20Superset.%20The%20library%20may%20be%3A%0A-%20Not%20storing%20TTL%20metadata%20with%20the%20S3%20objects%0A-%20Storing%20it%20but%20not%20checking%20it%20on%20retrieval%0A-%20Relying%20entirely%20on%20S3%20lifecycle%20policies%20%28which%20as%20you%20noted%20aren%27t%20reliable%20for%20precise%20timing%29%0A%0AIf%20you%27re%20able%20to%20narrow%20it%20down%20further%2C%20it%20would%20be%20worth%20reporting%20upstream%20to%20the%20%60s3cache%60%20library%20maintainers.%20In%20the%20meantime%2C%20%60SupersetMetastoreCache%60%20is%20a%20solid%20alternative%20if%20your%20volume%20permits%20database-backed%20caching.) [](https://go.dosu.dev/discord-bot?utm_source=github&utm_medium=bot-comment&utm_campaign= github-assistant&utm_content=join-discord) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/38605) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
