[ 
https://issues.apache.org/jira/browse/HIVE-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16436165#comment-16436165
 ] 

Jason Dere commented on HIVE-18609:
-----------------------------------

Tried running the test and ran into a hang. One thing I forgot to fix in this 
patch after HIVE-19127 is that the cache cannot attempt to remove an entry 
(which tries to take a write lock) during lookup() (where it has already taken 
a read lock), since ReentrantReadWriteLock does not allow upgrading a read lock 
to a write lock. Will change the logic so that the invalid entries are not 
removed until after the read lock is released.

{noformat}
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000007bcb3f3c0> (a 
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
        at 
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
        at 
org.apache.hadoop.hive.ql.cache.results.QueryResultsCache.removeEntry(QueryResultsCache.java:659)
  <== Tries to take write lock
        at 
org.apache.hadoop.hive.ql.cache.results.QueryResultsCache.entryMatches(QueryResultsCache.java:647)
        at 
org.apache.hadoop.hive.ql.cache.results.QueryResultsCache.lookup(QueryResultsCache.java:408)
    <== Took read lock
        at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.checkResultsCache(SemanticAnalyzer.java:14697)
        at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12055)
        at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:314)
        at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:287)
        at 
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:160)
        at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:287)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:635)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1655)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1602)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1597)
        at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
        at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:200)
        at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
        at 
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1438)
        at 
org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1412)
        at 
org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:177)
        at 
org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
        at 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
{noformat}

> Results cache invalidation based on ACID table updates
> ------------------------------------------------------
>
>                 Key: HIVE-18609
>                 URL: https://issues.apache.org/jira/browse/HIVE-18609
>             Project: Hive
>          Issue Type: Sub-task
>    Affects Versions: 3.1.0
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>            Priority: Major
>         Attachments: HIVE-18609.1.patch, HIVE-18609.2.patch, 
> HIVE-18609.3.patch
>
>
> Look into using the materialized view invalidation mechanisms to 
> automatically invalidate queries in the results cache if the underlying 
> tables used in the cached queries have been modified.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to