Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[Thejas Nair]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189945
---


Ship it!




Ship It!

- Thejas Nair


On Nov. 2, 2017, 6:25 p.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 2, 2017, 6:25 p.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a3c853a 
>   
> itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestHiveSessionImpl.java
>  ebcf4a8 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionBase.java 
> 9436a25 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
> 0206fe3 
>   
> service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java
>  8975aee 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
>   
> service/src/test/org/apache/hive/service/cli/session/TestPluggableHiveSessionImpl.java
>  47f95c5 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[j . prasanth . j]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/
---

(Updated Nov. 2, 2017, 6:25 p.m.)


Review request for hive, Sergey Shelukhin and Thejas Nair.


Changes
---

Addressed review comments. Handled forwarded addresses case as well.


Bugs: HIVE-16917
https://issues.apache.org/jira/browse/HIVE-16917


Repository: hive-git


Description
---

HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a3c853a 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestHiveSessionImpl.java
 ebcf4a8 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionBase.java 
9436a25 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
0206fe3 
  
service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java
 8975aee 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
9b2ae57 
  service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
6354c8c 
  
service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
 PRE-CREATION 
  
service/src/test/org/apache/hive/service/cli/session/TestPluggableHiveSessionImpl.java
 47f95c5 


Diff: https://reviews.apache.org/r/63458/diff/3/

Changes: https://reviews.apache.org/r/63458/diff/2-3/


Testing
---


Thanks,

Prasanth_J

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[j . prasanth . j]

> On Nov. 2, 2017, 5:01 p.m., Thejas Nair wrote:
> > service/src/java/org/apache/hive/service/cli/session/SessionManager.java
> > Lines 477 (patched)
> > 
> >
> > Can you also log the key on which limit has been reached ? (username 
> > and/or ipaddress).
> > It would help with troubleshooting from the server logs.

Done.


> On Nov. 2, 2017, 5:01 p.m., Thejas Nair wrote:
> > service/src/java/org/apache/hive/service/cli/session/SessionManager.java
> > Lines 508 (patched)
> > 
> >
> > safer to check - if( connectionCount >= limit )

Done.


- Prasanth_J


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189929
---


On Nov. 2, 2017, 8:08 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 2, 2017, 8:08 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a3c853a 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[Thejas Nair]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189929
---




service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 477 (patched)


Can you also log the key on which limit has been reached ? (username and/or 
ipaddress).
It would help with troubleshooting from the server logs.



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 508 (patched)


safer to check - if( connectionCount >= limit )


- Thejas Nair


On Nov. 2, 2017, 8:08 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 2, 2017, 8:08 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a3c853a 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[j . prasanth . j]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/
---

(Updated Nov. 2, 2017, 8:08 a.m.)


Review request for hive, Sergey Shelukhin and Thejas Nair.


Changes
---

Addressed review comments.


Bugs: HIVE-16917
https://issues.apache.org/jira/browse/HIVE-16917


Repository: hive-git


Description
---

HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a3c853a 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
9b2ae57 
  service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
6354c8c 
  
service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/63458/diff/2/

Changes: https://reviews.apache.org/r/63458/diff/1-2/


Testing
---


Thanks,

Prasanth_J

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[j . prasanth . j]

> On Nov. 2, 2017, 3:19 a.m., Alexander Kolbasov wrote:
> > service/src/java/org/apache/hive/service/cli/session/SessionManager.java
> > Lines 477 (patched)
> > 
> >
> > This potentially leak IP addresses of clients in the log file - would 
> > be better to avoid it.

removed it.


- Prasanth_J


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189871
---


On Nov. 1, 2017, 2:04 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 1, 2017, 2:04 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57ee498e9b712cc87b16ba223995519611c 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[j . prasanth . j]

> On Nov. 2, 2017, 1:34 a.m., Sergey Shelukhin wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Lines 2475 (patched)
> > 
> >
> > hmm... would it make sense to have 3 limits instead? per user, per ip, 
> > and per user+ip (if needed). All would be enforced. Or is it going to be 
> > extensible?

Yeah. Makes sense to have 3 limits. Updated patch. All 3 are tracked now.


> On Nov. 2, 2017, 1:34 a.m., Sergey Shelukhin wrote:
> > service/src/java/org/apache/hive/service/cli/session/SessionManager.java
> > Lines 472 (patched)
> > 
> >
> > nit: map is concurrent, it should be possible to use putIfAbsent and 
> > replace w/o synchronize

update with java 8 one liner.


- Prasanth_J


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189860
---


On Nov. 1, 2017, 2:04 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 1, 2017, 2:04 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57ee498e9b712cc87b16ba223995519611c 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[Alexander Kolbasov]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189871
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 2477 (patched)


nit: missing space between words



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 2479 (patched)


nit: missing space between words



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 472 (patched)


Right, you may actually use a combination of concurrent hashmap with atomics



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 477 (patched)


This potentially leak IP addresses of clients in the log file - would be 
better to avoid it.



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 497 (patched)


why is this <= 0 rather then == 0?



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 503 (patched)


This is leaking IP addresses in the log file


- Alexander Kolbasov


On Nov. 1, 2017, 2:04 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 1, 2017, 2:04 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57ee498e9b712cc87b16ba223995519611c 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/#review189860
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 2475 (patched)


hmm... would it make sense to have 3 limits instead? per user, per ip, and 
per user+ip (if needed). All would be enforced. Or is it going to be extensible?



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 472 (patched)


nit: map is concurrent, it should be possible to use putIfAbsent and 
replace w/o synchronize



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
Lines 493 (patched)


nit: the null check is outside for increment but inside for decrement


- Sergey Shelukhin


On Nov. 1, 2017, 2:04 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63458/
> ---
> 
> (Updated Nov. 1, 2017, 2:04 a.m.)
> 
> 
> Review request for hive, Sergey Shelukhin and Thejas Nair.
> 
> 
> Bugs: HIVE-16917
> https://issues.apache.org/jira/browse/HIVE-16917
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 9b2ae57ee498e9b712cc87b16ba223995519611c 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
>   
> service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/63458/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Review Request 63458: HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user

[j . prasanth . j]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63458/
---

Review request for hive, Sergey Shelukhin and Thejas Nair.


Bugs: HIVE-16917
https://issues.apache.org/jira/browse/HIVE-16917


Repository: hive-git


Description
---

HIVE-16917: HiveServer2 guard rails - Limit concurrent connections from user


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
a3c853a5c5a2a3e72ae0141d0ad836ec95516280 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
9b2ae57ee498e9b712cc87b16ba223995519611c 
  service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
6354c8c5450eaa8ab91dd59387f2eddf0eb84389 
  
service/src/test/org/apache/hive/service/cli/TestCLIServiceConnectionLimits.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/63458/diff/1/


Testing
---


Thanks,

Prasanth_J