zabetak commented on code in PR #6064:
URL: https://github.com/apache/hive/pull/6064#discussion_r2332270127
##########
ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java:
##########
@@ -671,10 +672,15 @@ public void notifyTableChanged(String dbName, String
tableName, long updateTime)
*/
private boolean entryMatches(LookupInfo lookupInfo, CacheEntry entry,
Set<CacheEntry> entriesToRemove) {
QueryInfo queryInfo = entry.getQueryInfo();
+
+ Set<Long> cacheTableIds = new HashSet<>();
for (ReadEntity readEntity : queryInfo.getInputs()) {
- // Check that the tables used do not resolve to temp tables.
if (readEntity.getType() == Type.TABLE) {
Table tableUsed = readEntity.getTable();
+ // collect the table ids of the cache entry
+ cacheTableIds.add(tableUsed.getTTable().getId());
Review Comment:
Do we need to collect **all** IDs in a temporary set? Can't we check if the
current table ID is part of the `lookupInfo` and bail out immediately without
going through all tables in this `entry`?
##########
ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java:
##########
@@ -671,10 +672,15 @@ public void notifyTableChanged(String dbName, String
tableName, long updateTime)
*/
private boolean entryMatches(LookupInfo lookupInfo, CacheEntry entry,
Set<CacheEntry> entriesToRemove) {
QueryInfo queryInfo = entry.getQueryInfo();
+
+ Set<Long> cacheTableIds = new HashSet<>();
for (ReadEntity readEntity : queryInfo.getInputs()) {
- // Check that the tables used do not resolve to temp tables.
if (readEntity.getType() == Type.TABLE) {
Table tableUsed = readEntity.getTable();
Review Comment:
Is there a way to use the table ID to invalidate an cache entry? We we are
doing some kind of invalidation below based on writeids so I was wondering if
we could/should do something similar when a table is dropped. This is not
strictly related to the stale results problem so we can certainly defer it to a
follow-up if it makes sense.
##########
ql/src/test/queries/clientpositive/results_cache_invalidation4.q:
##########
@@ -0,0 +1,16 @@
+set hive.support.concurrency=true;
+set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
+
+set hive.query.results.cache.enabled=true;
+set hive.query.results.cache.nontransactional.tables.enabled=true;
Review Comment:
Just out of curiosity did you confirm that bug is also affecting
nontransactional tables?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
