[jira] [Commented] (SENTRY-1120) Show role / privileges info in Sentry Service Webpage
[
https://issues.apache.org/jira/browse/SENTRY-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550527#comment-15550527
]
Hadoop QA commented on SENTRY-1120:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12831628/SENTRY-1120.3.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2049/console
This message is automatically generated.
> Show role / privileges info in Sentry Service Webpage
> -
>
> Key: SENTRY-1120
> URL: https://issues.apache.org/jira/browse/SENTRY-1120
> Project: Sentry
> Issue Type: Improvement
>Reporter: Li Li
>Assignee: Li Li
> Fix For: 1.8.0
>
> Attachments: SENTRY-1120.0.patch, SENTRY-1120.1.patch,
> SENTRY-1120.2.patch, SENTRY-1120.3.patch, Screen Shot 2016-04-06 at 9.00.55
> PM.png, showAll.json
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SENTRY-1489) Categorize e2e tests into slow and regular tests, so that can adapt the timeout and etc.
[
https://issues.apache.org/jira/browse/SENTRY-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550466#comment-15550466
]
Hadoop QA commented on SENTRY-1489:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12831835/SENTRY-1489.0.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2048/console
This message is automatically generated.
> Categorize e2e tests into slow and regular tests, so that can adapt the
> timeout and etc.
>
>
> Key: SENTRY-1489
> URL: https://issues.apache.org/jira/browse/SENTRY-1489
> Project: Sentry
> Issue Type: Test
> Components: Sentry
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Anne Yu
>Assignee: Anne Yu
> Fix For: 1.8.0, sentry-ha-redesign
>
> Attachments: SENTRY-1489.0.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SENTRY-1120) Show role / privileges info in Sentry Service Webpage
[ https://issues.apache.org/jira/browse/SENTRY-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Li updated SENTRY-1120: -- Attachment: showAll.json Here is a sample json file via wget hostname:port/admin/showAll > Show role / privileges info in Sentry Service Webpage > - > > Key: SENTRY-1120 > URL: https://issues.apache.org/jira/browse/SENTRY-1120 > Project: Sentry > Issue Type: Improvement >Reporter: Li Li >Assignee: Li Li > Fix For: 1.8.0 > > Attachments: SENTRY-1120.0.patch, SENTRY-1120.1.patch, > SENTRY-1120.2.patch, SENTRY-1120.3.patch, Screen Shot 2016-04-06 at 9.00.55 > PM.png, showAll.json > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SENTRY-1489) Categorize e2e tests into slow and regular tests, so that can adapt the timeout and etc.
[ https://issues.apache.org/jira/browse/SENTRY-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Yu updated SENTRY-1489: Status: Patch Available (was: Open) > Categorize e2e tests into slow and regular tests, so that can adapt the > timeout and etc. > > > Key: SENTRY-1489 > URL: https://issues.apache.org/jira/browse/SENTRY-1489 > Project: Sentry > Issue Type: Test > Components: Sentry >Affects Versions: 1.8.0, sentry-ha-redesign >Reporter: Anne Yu >Assignee: Anne Yu > Fix For: 1.8.0, sentry-ha-redesign > > Attachments: SENTRY-1489.0.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SENTRY-1490) Make tests more robust with respect to isolation
[ https://issues.apache.org/jira/browse/SENTRY-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sravya Tirukkovalur updated SENTRY-1490: Description: Traditionally in our e2e tests, each test class sets up the backend database. And each test creates its own database and performs some operations and then we clean up the databases created as part of the test either within the test or in @After. There are some problems with this approach: - Not possible to run tests parallely, as more than one test can be using same environment(db). - If tests do not clean up well, it leads to cascading test failures due to incorrect assumptions. In the test TestDBNotificationListenerInBuiltDeserializer, I tried solving this problem by creating a random name for db in each test case where dbname= db+randomNumber. But the randomness does not seem to be sufficient as we are not making sure the random numbers are non repeating. I am thinking of either of the below solutions here: 1. Generate non repeating random names. 2. Use an atomic class integer rather than a random number. Can you think of any other simpler solutions? was: Traditionally in our e2e tests, each test class setups the backend database. And each test creates its own database and performs some operations and then we clean up the databases created as part of the test either within the test or in @After. There are some problems with this approach: - Not possible to run tests parallely, as more than one test can be using same environment(db). - If tests do not clean up well, it leads to cascading test failures due to incorrect assumptions. In the test TestDBNotificationListenerInBuiltDeserializer, I tried solving this problem by creating a random name for db in each test case where dbname= db+randomNumber. But the randomness does not seem to be sufficient as we are not making sure the random numbers are non repeating. I am thinking of either of the below solutions here: 1. Generate non repeating random names. 2. Use an atomic class integer rather than a random number. Can you think of any other simpler solutions? > Make tests more robust with respect to isolation > > > Key: SENTRY-1490 > URL: https://issues.apache.org/jira/browse/SENTRY-1490 > Project: Sentry > Issue Type: Test >Reporter: Sravya Tirukkovalur >Assignee: Sravya Tirukkovalur >Priority: Minor > > Traditionally in our e2e tests, each test class sets up the backend database. > And each test creates its own database and performs some operations and then > we clean up the databases created as part of the test either within the test > or in @After. > There are some problems with this approach: > - Not possible to run tests parallely, as more than one test can be using > same environment(db). > - If tests do not clean up well, it leads to cascading test failures due to > incorrect assumptions. > In the test TestDBNotificationListenerInBuiltDeserializer, I tried solving > this problem by creating a random name for db in each test case where dbname= > db+randomNumber. But the randomness does not seem to be sufficient as we are > not making sure the random numbers are non repeating. I am thinking of either > of the below solutions here: > 1. Generate non repeating random names. > 2. Use an atomic class integer rather than a random number. > Can you think of any other simpler solutions? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1490) Make tests more robust with respect to isolation
[ https://issues.apache.org/jira/browse/SENTRY-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550050#comment-15550050 ] Sravya Tirukkovalur commented on SENTRY-1490: - [~anneyu] > Make tests more robust with respect to isolation > > > Key: SENTRY-1490 > URL: https://issues.apache.org/jira/browse/SENTRY-1490 > Project: Sentry > Issue Type: Test >Reporter: Sravya Tirukkovalur >Assignee: Sravya Tirukkovalur >Priority: Minor > > Traditionally in our e2e tests, each test class sets up the backend database. > And each test creates its own database and performs some operations and then > we clean up the databases created as part of the test either within the test > or in @After. > There are some problems with this approach: > - Not possible to run tests parallely, as more than one test can be using > same environment(db). > - If tests do not clean up well, it leads to cascading test failures due to > incorrect assumptions. > In the test TestDBNotificationListenerInBuiltDeserializer, I tried solving > this problem by creating a random name for db in each test case where dbname= > db+randomNumber. But the randomness does not seem to be sufficient as we are > not making sure the random numbers are non repeating. I am thinking of either > of the below solutions here: > 1. Generate non repeating random names. > 2. Use an atomic class integer rather than a random number. > Can you think of any other simpler solutions? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SENTRY-1490) Make tests more robust with respect to isolation
Sravya Tirukkovalur created SENTRY-1490: --- Summary: Make tests more robust with respect to isolation Key: SENTRY-1490 URL: https://issues.apache.org/jira/browse/SENTRY-1490 Project: Sentry Issue Type: Test Reporter: Sravya Tirukkovalur Assignee: Sravya Tirukkovalur Priority: Minor Traditionally in our e2e tests, each test class setups the backend database. And each test creates its own database and performs some operations and then we clean up the databases created as part of the test either within the test or in @After. There are some problems with this approach: - Not possible to run tests parallely, as more than one test can be using same environment(db). - If tests do not clean up well, it leads to cascading test failures due to incorrect assumptions. In the test TestDBNotificationListenerInBuiltDeserializer, I tried solving this problem by creating a random name for db in each test case where dbname= db+randomNumber. But the randomness does not seem to be sufficient as we are not making sure the random numbers are non repeating. I am thinking of either of the below solutions here: 1. Generate non repeating random names. 2. Use an atomic class integer rather than a random number. Can you think of any other simpler solutions? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1486) Sentry should use repeatable-read consistency level
[ https://issues.apache.org/jira/browse/SENTRY-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550027#comment-15550027 ] Alexander Kolbasov commented on SENTRY-1486: Here is what I am using: mysql Ver 14.14 Distrib 5.7.15, for osx10.11 (x86_64) > Sentry should use repeatable-read consistency level > --- > > Key: SENTRY-1486 > URL: https://issues.apache.org/jira/browse/SENTRY-1486 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 1.7.0, sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov > Attachments: SENTRY-1486.001.patch, SENTRY-1486.002.patch > > > Currently Sentry uses the "read-committed" consistency level which is the > default for the Datanucleus JDO library. This causes potential problems since > the state visible to each transaction can actually see updates from another > transactions, so it is very difficult to reason about any code that reads > multiple pieces of data. > Instead it should use repeatable read" consistency which guarantees that any > transaction only sees the state at the beginning of a transaction plus any > updates done within a transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (SENTRY-1486) Sentry should use repeatable-read consistency level
[ https://issues.apache.org/jira/browse/SENTRY-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549889#comment-15549889 ] Sravya Tirukkovalur edited comment on SENTRY-1486 at 10/5/16 8:46 PM: -- [~akolb] Thanks for running a mini bench mark. Thats useful. By the way, which databases did you try it on? was (Author: sravya): [~akolb] Which databases did you try it on? > Sentry should use repeatable-read consistency level > --- > > Key: SENTRY-1486 > URL: https://issues.apache.org/jira/browse/SENTRY-1486 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 1.7.0, sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov > Attachments: SENTRY-1486.001.patch, SENTRY-1486.002.patch > > > Currently Sentry uses the "read-committed" consistency level which is the > default for the Datanucleus JDO library. This causes potential problems since > the state visible to each transaction can actually see updates from another > transactions, so it is very difficult to reason about any code that reads > multiple pieces of data. > Instead it should use repeatable read" consistency which guarantees that any > transaction only sees the state at the beginning of a transaction plus any > updates done within a transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1486) Sentry should use repeatable-read consistency level
[ https://issues.apache.org/jira/browse/SENTRY-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549889#comment-15549889 ] Sravya Tirukkovalur commented on SENTRY-1486: - [~akolb] Which databases did you try it on? > Sentry should use repeatable-read consistency level > --- > > Key: SENTRY-1486 > URL: https://issues.apache.org/jira/browse/SENTRY-1486 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 1.7.0, sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov > Attachments: SENTRY-1486.001.patch, SENTRY-1486.002.patch > > > Currently Sentry uses the "read-committed" consistency level which is the > default for the Datanucleus JDO library. This causes potential problems since > the state visible to each transaction can actually see updates from another > transactions, so it is very difficult to reason about any code that reads > multiple pieces of data. > Instead it should use repeatable read" consistency which guarantees that any > transaction only sees the state at the beginning of a transaction plus any > updates done within a transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1486) Sentry should use repeatable-read consistency level
[ https://issues.apache.org/jira/browse/SENTRY-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549729#comment-15549729 ] Alexander Kolbasov commented on SENTRY-1486: [~sravya], [~hahao] To answer you question, I did a small micro-benchmark using two threads - one constantly updating all records and one reading records. I measured time on the reader thread and didn't see any difference between read-committed and repeatable read. > Sentry should use repeatable-read consistency level > --- > > Key: SENTRY-1486 > URL: https://issues.apache.org/jira/browse/SENTRY-1486 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 1.7.0, sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov > Attachments: SENTRY-1486.001.patch, SENTRY-1486.002.patch > > > Currently Sentry uses the "read-committed" consistency level which is the > default for the Datanucleus JDO library. This causes potential problems since > the state visible to each transaction can actually see updates from another > transactions, so it is very difficult to reason about any code that reads > multiple pieces of data. > Instead it should use repeatable read" consistency which guarantees that any > transaction only sees the state at the beginning of a transaction plus any > updates done within a transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1477) Sentry clients should retry with another server when they get connection errors
[ https://issues.apache.org/jira/browse/SENTRY-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549725#comment-15549725 ] Alexander Kolbasov commented on SENTRY-1477: It works, but may be confusing for someone who sets the value- it is easier to specify number of times to retry as a number of times to cycle through the conn list. > Sentry clients should retry with another server when they get connection > errors > --- > > Key: SENTRY-1477 > URL: https://issues.apache.org/jira/browse/SENTRY-1477 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Li Li > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1477.0-sentry-ha-redesign.patch > > > When Sentry clients get a connection error when talking to a server they > should re-try connection with another server. This should be normal client > behavior when one of the servers goes down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SENTRY-1489) Categorize e2e tests into slow and regular tests, so that can adapt the timeout and etc.
Anne Yu created SENTRY-1489: --- Summary: Categorize e2e tests into slow and regular tests, so that can adapt the timeout and etc. Key: SENTRY-1489 URL: https://issues.apache.org/jira/browse/SENTRY-1489 Project: Sentry Issue Type: Test Components: Sentry Affects Versions: 1.8.0, sentry-ha-redesign Reporter: Anne Yu Assignee: Anne Yu Fix For: 1.8.0, sentry-ha-redesign -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1477) Sentry clients should retry with another server when they get connection errors
[ https://issues.apache.org/jira/browse/SENTRY-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549616#comment-15549616 ] Li Li commented on SENTRY-1477: --- So the total retry num = num of sentry server * max_retry_num ? > Sentry clients should retry with another server when they get connection > errors > --- > > Key: SENTRY-1477 > URL: https://issues.apache.org/jira/browse/SENTRY-1477 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Li Li > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1477.0-sentry-ha-redesign.patch > > > When Sentry clients get a connection error when talking to a server they > should re-try connection with another server. This should be normal client > behavior when one of the servers goes down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1477) Sentry clients should retry with another server when they get connection errors
[ https://issues.apache.org/jira/browse/SENTRY-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549612#comment-15549612 ] Li Li commented on SENTRY-1477: --- Yes, I agree with randomly shuffle and small random sleep. > Sentry clients should retry with another server when they get connection > errors > --- > > Key: SENTRY-1477 > URL: https://issues.apache.org/jira/browse/SENTRY-1477 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Li Li > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1477.0-sentry-ha-redesign.patch > > > When Sentry clients get a connection error when talking to a server they > should re-try connection with another server. This should be normal client > behavior when one of the servers goes down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1477) Sentry clients should retry with another server when they get connection errors
[ https://issues.apache.org/jira/browse/SENTRY-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549600#comment-15549600 ] Alexander Kolbasov commented on SENTRY-1477: It may be a good idea to randomly shuffle the list of connections if its length is greater than two. Otherwise all clients will move at once to the same new server when something fails. Also it may be a good idea to introduce some small random sleep after going through the full cycle of connections. > Sentry clients should retry with another server when they get connection > errors > --- > > Key: SENTRY-1477 > URL: https://issues.apache.org/jira/browse/SENTRY-1477 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Li Li > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1477.0-sentry-ha-redesign.patch > > > When Sentry clients get a connection error when talking to a server they > should re-try connection with another server. This should be normal client > behavior when one of the servers goes down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SENTRY-1477) Sentry clients should retry with another server when they get connection errors
[ https://issues.apache.org/jira/browse/SENTRY-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549592#comment-15549592 ] Alexander Kolbasov commented on SENTRY-1477: I think that client should attempt to connect to any available address, so the retry count shouldn't be really "number of times to cycle through the full connection list". > Sentry clients should retry with another server when they get connection > errors > --- > > Key: SENTRY-1477 > URL: https://issues.apache.org/jira/browse/SENTRY-1477 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Li Li > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1477.0-sentry-ha-redesign.patch > > > When Sentry clients get a connection error when talking to a server they > should re-try connection with another server. This should be normal client > behavior when one of the servers goes down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
