[jira] [Updated] (HIVE-16992) LLAP: monitoring and better default lambda for LRFU policy
[ https://issues.apache.org/jira/browse/HIVE-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16992: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master after some cluster smoke testing. Thanks for the review! > LLAP: monitoring and better default lambda for LRFU policy > -- > > Key: HIVE-16992 > URL: https://issues.apache.org/jira/browse/HIVE-16992 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-16992.patch > > > LRFU is currently skewed heavily towards LRU; there are 10k-s or 100k-s of > buffers tracked during a typical workload, but the heap size is around 700. > We should see if making it closer to LFU (by tweaking the lambda) will > improve hit rate with small queries infrequently interleaved with large > scans; and whether it will have negative effects due to perf overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16992) LLAP: monitoring and better default lambda for LRFU policy
[ https://issues.apache.org/jira/browse/HIVE-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16992: Issue Type: Sub-task (was: Bug) Parent: HIVE-18939 > LLAP: monitoring and better default lambda for LRFU policy > -- > > Key: HIVE-16992 > URL: https://issues.apache.org/jira/browse/HIVE-16992 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-16992.patch > > > LRFU is currently skewed heavily towards LRU; there are 10k-s or 100k-s of > buffers tracked during a typical workload, but the heap size is around 700. > We should see if making it closer to LFU (by tweaking the lambda) will > improve hit rate with small queries infrequently interleaved with large > scans; and whether it will have negative effects due to perf overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16992) LLAP: monitoring and better default lambda for LRFU policy
[ https://issues.apache.org/jira/browse/HIVE-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16992: Status: Patch Available (was: Open) I will file some follow-up jiras from this. 1) Better mapping for entities in moitoring, not just based on the split path. 2) Easier to tune/simpler cache policy :) > LLAP: monitoring and better default lambda for LRFU policy > -- > > Key: HIVE-16992 > URL: https://issues.apache.org/jira/browse/HIVE-16992 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-16992.patch > > > LRFU is currently skewed heavily towards LRU; there are 10k-s or 100k-s of > buffers tracked during a typical workload, but the heap size is around 700. > We should see if making it closer to LFU (by tweaking the lambda) will > improve hit rate with small queries infrequently interleaved with large > scans; and whether it will have negative effects due to perf overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16992) LLAP: monitoring and better default lambda for LRFU policy
[ https://issues.apache.org/jira/browse/HIVE-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16992: Attachment: HIVE-16992.patch > LLAP: monitoring and better default lambda for LRFU policy > -- > > Key: HIVE-16992 > URL: https://issues.apache.org/jira/browse/HIVE-16992 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-16992.patch > > > LRFU is currently skewed heavily towards LRU; there are 10k-s or 100k-s of > buffers tracked during a typical workload, but the heap size is around 700. > We should see if making it closer to LFU (by tweaking the lambda) will > improve hit rate with small queries infrequently interleaved with large > scans; and whether it will have negative effects due to perf overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16992) LLAP: monitoring and better default lambda for LRFU policy
[ https://issues.apache.org/jira/browse/HIVE-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16992: Summary: LLAP: monitoring and better default lambda for LRFU policy (was: LLAP: better default lambda for LRFU policy) > LLAP: monitoring and better default lambda for LRFU policy > -- > > Key: HIVE-16992 > URL: https://issues.apache.org/jira/browse/HIVE-16992 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > LRFU is currently skewed heavily towards LRU; there are 10k-s or 100k-s of > buffers tracked during a typical workload, but the heap size is around 700. > We should see if making it closer to LFU (by tweaking the lambda) will > improve hit rate with small queries infrequently interleaved with large > scans; and whether it will have negative effects due to perf overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
