[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-03-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891439#comment-15891439 ] Rui Li commented on HIVE-15882: --- I'll commit shortly if there's no further comments. > HS2 generating high

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-28 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889345#comment-15889345 ] Rui Li commented on HIVE-15882: --- Thanks [~mi...@cloudera.com] for the update. +1 > HS2 generating high

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-28 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1541#comment-1541 ] Misha Dmitriev commented on HIVE-15882: --- [~lirui] thank you for your constructive feedback, I hope

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-27 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886651#comment-15886651 ] Misha Dmitriev commented on HIVE-15882: --- The same tests fail in other builds, so they are not due to

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-27 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886625#comment-15886625 ] Hive QA commented on HIVE-15882: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-25 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884175#comment-15884175 ] Hive QA commented on HIVE-15882: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-23 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882157#comment-15882157 ] Rui Li commented on HIVE-15882: --- The patch looks good to me overall. I left some minor comments on the RB.

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-23 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881788#comment-15881788 ] Misha Dmitriev commented on HIVE-15882: --- [~lirui] sure - the RB for the first change (string

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-23 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881733#comment-15881733 ] Rui Li commented on HIVE-15882: --- Thanks [~mi...@cloudera.com] for the benchmark results. It looks promising.

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-23 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881399#comment-15881399 ] Misha Dmitriev commented on HIVE-15882: --- I've just measured the CPU performance impact of my changes

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-23 Thread Vihang Karajgaonkar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881337#comment-15881337 ] Vihang Karajgaonkar commented on HIVE-15882: 37% is a great improvement [~mi...@cloudera.com].

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-23 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881244#comment-15881244 ] Misha Dmitriev commented on HIVE-15882: --- I've measured how much memory is saved with my change. It

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-23 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881203#comment-15881203 ] Sahil Takiar commented on HIVE-15882: - [~mi...@cloudera.com] you mentioned this earlier, but it would

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-22 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879949#comment-15879949 ] Misha Dmitriev commented on HIVE-15882: --- Hi [~lirui], this is a legitimate concern. Regarding the

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-22 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879917#comment-15879917 ] Rui Li commented on HIVE-15882: --- Hi [~mi...@cloudera.com], I guess one possible issue of String::intern is

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-22 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879349#comment-15879349 ] Misha Dmitriev commented on HIVE-15882: --- Yes, I did take a heap dump and rerun the tool after

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-17 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871561#comment-15871561 ] Rui Li commented on HIVE-15882: --- I also noticed lots of duplicated Strings in {{ListSinkOperator::res}} when

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-15 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868357#comment-15868357 ] Misha Dmitriev commented on HIVE-15882: --- I've checked the failed tests. They either pass for me

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-14 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867157#comment-15867157 ] Hive QA commented on HIVE-15882: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-02-14 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866879#comment-15866879 ] Misha Dmitriev commented on HIVE-15882: --- For convenience, I've created a code review here: