Re: Review Request 68918: SENTRY-2423: Increase the allocation size for auto-increment of id's for Snapshot tables.

2018-10-09 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68918/#review209369
---


Ship it!




Ship It!

- Na Li


On Oct. 3, 2018, 8:24 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68918/
> ---
> 
> (Updated Oct. 3, 2018, 8:24 p.m.)
> 
> 
> Review request for sentry, Arjun Mishra, Na Li, and Sergio Pena.
> 
> 
> Bugs: SENTRY-2423
> https://issues.apache.org/jira/browse/SENTRY-2423
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Currently sentry uses native strategy for auto generation of ID columns for 
> which the allocation can not be increased.
> This should be change to "increment" strategy and which lets to configure the 
> allocation size. This reduces the delay caused for checking the 
> SEQUENCE_TABLE.
> 
> 
> Diffs
> -
> 
>   
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/service/common/ServiceConstants.java
>  e90fe2df46266d72f1eb3706b948fb163a44c4a1 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  6539e33e021e0f5acaa7827356a8e9b3e4286b18 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1722109e381bc7be81a4673d12c1ac9f81195c47 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  a299e008cc0df92f3d692024c7794aa494b64d2d 
> 
> 
> Diff: https://reviews.apache.org/r/68918/diff/1/
> 
> 
> Testing
> ---
> 
> Made sure all the existing tests passed.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 68918: SENTRY-2423: Increase the allocation size for auto-increment of id's for Snapshot tables.

2018-10-09 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68918/#review209363
---


Ship it!




Looks good

- Sergio Pena


On Oct. 3, 2018, 8:24 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68918/
> ---
> 
> (Updated Oct. 3, 2018, 8:24 p.m.)
> 
> 
> Review request for sentry, Arjun Mishra, Na Li, and Sergio Pena.
> 
> 
> Bugs: SENTRY-2423
> https://issues.apache.org/jira/browse/SENTRY-2423
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Currently sentry uses native strategy for auto generation of ID columns for 
> which the allocation can not be increased.
> This should be change to "increment" strategy and which lets to configure the 
> allocation size. This reduces the delay caused for checking the 
> SEQUENCE_TABLE.
> 
> 
> Diffs
> -
> 
>   
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/service/common/ServiceConstants.java
>  e90fe2df46266d72f1eb3706b948fb163a44c4a1 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  6539e33e021e0f5acaa7827356a8e9b3e4286b18 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1722109e381bc7be81a4673d12c1ac9f81195c47 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  a299e008cc0df92f3d692024c7794aa494b64d2d 
> 
> 
> Diff: https://reviews.apache.org/r/68918/diff/1/
> 
> 
> Testing
> ---
> 
> Made sure all the existing tests passed.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 68918: SENTRY-2423: Increase the allocation size for auto-increment of id's for Snapshot tables.

2018-10-05 Thread kalyan kumar kalvagadda via Review Board 


> On Oct. 4, 2018, 12:47 a.m., Na Li wrote:
> > sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
> > Lines 4575 (patched)
> > 
> >
> > do you have measurement on how long it takes with/without this fix?

When we test locally it may be very significant difference. But the difference 
would be significant under below factors
1. Network Latency
2. When the backend database is busy handling ton of queries.
3. There are a lot connections to the backend database.

When ever the inserts are blocked to get the sequence from the database were 
are adding delay. This delay may not be significant in simple test environment.
The sequence values are collected in a different thread and the AUTHZ_PATH are 
updated in different threads. While the thread is getting the NEXT_VAL, thread 
that is updating the AUTHZ_PATH is blocked. It should be unblocked once 
NEXT_VAL is retrieved. This is happening in the application context as the 
SEQUENCING is managed in the application. Once a thread moved from waiting 
state and back to runnable state, processor may OR may not scheduled the thread 
right away. It depends based on how other threads are loading the processor. 
Even a thread in a infinite loop of doing non-blocking operations, it will be 
moved from "Running" state to "Runnable" as the processor is a shared resource. 
Whenever it happens thread would be in "Runnable" state for some time. In our 
case we are moving thread inserting the entries into AUTHZ_PATH table is moved 
from "Running" state to "Runnable" for every 5 inserts. This bacome much worse 
if the dabtabase is loaded and has a many connections. Let's take a
 n example of mysql, each connection to the datbase would result in a server 
thread. That means that mysql server should be managing that many threads. If 
there is no data on that thread the thread goes to sleep. When there are many 
threads, database would know that there is some data on connection only when 
the thread that is managing that connection is scheduled. This would introduce 
additional delay in reading it from network and executing. More the number of 
connections more delay it might introduce.


- kalyan kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68918/#review209205
---


On Oct. 3, 2018, 8:24 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68918/
> ---
> 
> (Updated Oct. 3, 2018, 8:24 p.m.)
> 
> 
> Review request for sentry, Arjun Mishra, Na Li, and Sergio Pena.
> 
> 
> Bugs: SENTRY-2423
> https://issues.apache.org/jira/browse/SENTRY-2423
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Currently sentry uses native strategy for auto generation of ID columns for 
> which the allocation can not be increased.
> This should be change to "increment" strategy and which lets to configure the 
> allocation size. This reduces the delay caused for checking the 
> SEQUENCE_TABLE.
> 
> 
> Diffs
> -
> 
>   
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/service/common/ServiceConstants.java
>  e90fe2df46266d72f1eb3706b948fb163a44c4a1 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  6539e33e021e0f5acaa7827356a8e9b3e4286b18 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1722109e381bc7be81a4673d12c1ac9f81195c47 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  a299e008cc0df92f3d692024c7794aa494b64d2d 
> 
> 
> Diff: https://reviews.apache.org/r/68918/diff/1/
> 
> 
> Testing
> ---
> 
> Made sure all the existing tests passed.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 68918: SENTRY-2423: Increase the allocation size for auto-increment of id's for Snapshot tables.

2018-10-03 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68918/#review209205
---




sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
Lines 4575 (patched)


do you have measurement on how long it takes with/without this fix?


- Na Li


On Oct. 3, 2018, 8:24 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68918/
> ---
> 
> (Updated Oct. 3, 2018, 8:24 p.m.)
> 
> 
> Review request for sentry, Arjun Mishra, Na Li, and Sergio Pena.
> 
> 
> Bugs: SENTRY-2423
> https://issues.apache.org/jira/browse/SENTRY-2423
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Currently sentry uses native strategy for auto generation of ID columns for 
> which the allocation can not be increased.
> This should be change to "increment" strategy and which lets to configure the 
> allocation size. This reduces the delay caused for checking the 
> SEQUENCE_TABLE.
> 
> 
> Diffs
> -
> 
>   
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/service/common/ServiceConstants.java
>  e90fe2df46266d72f1eb3706b948fb163a44c4a1 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  6539e33e021e0f5acaa7827356a8e9b3e4286b18 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1722109e381bc7be81a4673d12c1ac9f81195c47 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  a299e008cc0df92f3d692024c7794aa494b64d2d 
> 
> 
> Diff: https://reviews.apache.org/r/68918/diff/1/
> 
> 
> Testing
> ---
> 
> Made sure all the existing tests passed.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 68918: SENTRY-2423: Increase the allocation size for auto-increment of id's for Snapshot tables.

2018-10-03 Thread Arjun Mishra via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68918/#review209195
---


Ship it!




Ship It!

- Arjun Mishra


On Oct. 3, 2018, 8:24 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68918/
> ---
> 
> (Updated Oct. 3, 2018, 8:24 p.m.)
> 
> 
> Review request for sentry, Arjun Mishra, Na Li, and Sergio Pena.
> 
> 
> Bugs: SENTRY-2423
> https://issues.apache.org/jira/browse/SENTRY-2423
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Currently sentry uses native strategy for auto generation of ID columns for 
> which the allocation can not be increased.
> This should be change to "increment" strategy and which lets to configure the 
> allocation size. This reduces the delay caused for checking the 
> SEQUENCE_TABLE.
> 
> 
> Diffs
> -
> 
>   
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/service/common/ServiceConstants.java
>  e90fe2df46266d72f1eb3706b948fb163a44c4a1 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  6539e33e021e0f5acaa7827356a8e9b3e4286b18 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1722109e381bc7be81a4673d12c1ac9f81195c47 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  a299e008cc0df92f3d692024c7794aa494b64d2d 
> 
> 
> Diff: https://reviews.apache.org/r/68918/diff/1/
> 
> 
> Testing
> ---
> 
> Made sure all the existing tests passed.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Review Request 68918: SENTRY-2423: Increase the allocation size for auto-increment of id's for Snapshot tables.

2018-10-03 Thread kalyan kumar kalvagadda via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68918/
---

Review request for sentry, Arjun Mishra, Na Li, and Sergio Pena.


Bugs: SENTRY-2423
https://issues.apache.org/jira/browse/SENTRY-2423


Repository: sentry


Description
---

Currently sentry uses native strategy for auto generation of ID columns for 
which the allocation can not be increased.
This should be change to "increment" strategy and which lets to configure the 
allocation size. This reduces the delay caused for checking the SEQUENCE_TABLE.


Diffs
-

  
sentry-core/sentry-core-common/src/main/java/org/apache/sentry/service/common/ServiceConstants.java
 e90fe2df46266d72f1eb3706b948fb163a44c4a1 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
 6539e33e021e0f5acaa7827356a8e9b3e4286b18 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 1722109e381bc7be81a4673d12c1ac9f81195c47 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 a299e008cc0df92f3d692024c7794aa494b64d2d 


Diff: https://reviews.apache.org/r/68918/diff/1/


Testing
---

Made sure all the existing tests passed.


Thanks,

kalyan kumar kalvagadda