Yuvraj-cyborg commented on issue #19273:
URL: https://github.com/apache/datafusion/issues/19273#issuecomment-3641971641
@BlakeOrth
ok so should it be like
making the ```DefaultListFilesCache``` "prefix-aware" by modifying the
get_with_extra method to accept the table's base path as the Extra parameter.
And now when a cache lookup occurs for a partition-specific prefix (for say,
table/a=1/), the cache will-
- Try an exact key match (current impl)
- If that fails, check if a parent prefix (the table base path passed as
Extra) exists in the cache
and if found, then will filter the cached ```ObjectMeta``` entries to
only return files whose location starts with the requested prefix.
- And finally return the filtered results without making a storage call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
