Review Request 69914: HIVE-21227: HIVE-20776 causes view access regression
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69914/ --- Review request for hive and Vihang Karajgaonkar. Bugs: hive-21227 https://issues.apache.org/jira/browse/hive-21227 Repository: hive-git Description --- HIVE-20776 introduces a change that causes regression for view access. Before the change, a user with select access of a view can get all columns of a view with select access of a view that is derived from a partitioned table. With the change, that user cannot access that view. The reason is that When user accesses columns of a view, Hive needs to get the partitions of the table that the view is derived from. The user name is the user who issues the query to access the view. The change in HIVE-20776 checks if user has access to a table before getting its partitions. When user only has access of a view, not the access of a table itself, this change denies the user access of the view. The solution is when getting table partitions, do not filter on table at HMS client Diffs - standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java a1826fa259d424c9f3d5a2f58a18f617355d586f Diff: https://reviews.apache.org/r/69914/diff/1/ Testing --- TestGetPartitions and TestListPartitions pass Thanks, Na Li
[jira] [Created] (HIVE-21227) HIVE-20776 causes view access regression
Na Li created HIVE-21227: Summary: HIVE-20776 causes view access regression Key: HIVE-21227 URL: https://issues.apache.org/jira/browse/HIVE-21227 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 4.0.0 Reporter: Na Li Assignee: Na Li HIVE-20776 introduces a change that causes regression for view access. Before the change, a user with select access of a view can get all columns of a view with select access of a view that is derived from a partitioned table. With the change, that user cannot access that view. The reason is that * When user accesses columns of a view, Hive needs to get the partitions of the table that the view is derived from. The user name is the user who issues the query to access the view. * The change in HIVE-20776 checks if user has access to a table before getting its partitions. When user only has access of a view, not the access of a table itself, this change denies the user access of the view. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21226) Exclude read-only transactions from ValidTxnList
Eugene Koifman created HIVE-21226: - Summary: Exclude read-only transactions from ValidTxnList Key: HIVE-21226 URL: https://issues.apache.org/jira/browse/HIVE-21226 Project: Hive Issue Type: Improvement Components: Transactions Affects Versions: 3.0.0 Reporter: Eugene Koifman Once HIVE-21114 is done, we should make sure that ValidTxnList doesn't contain any read-only txns in the exceptions list since by definition there is no data tagged with such txnid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
Gopal V created HIVE-21225: -- Summary: ACID: getAcidState() should cache a recursive dir listing locally Key: HIVE-21225 URL: https://issues.apache.org/jira/browse/HIVE-21225 Project: Hive Issue Type: Bug Components: Transactions Reporter: Gopal V Currently getAcidState() makes 3 calls into the FS api which could be answered by making a single recursive listDir call and reusing the same data to check for isRawFormat() and isValidBase(). All delta operations for a single partition can go against a single listed directory snapshot instead of interacting with the NameNode or ObjectStore within the inner loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 69683: [HIVE-21071] Improve getInputSummary
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69683/ --- (Updated Feb. 6, 2019, 9:17 p.m.) Review request for hive. Repository: hive-git Description --- Improve performance of method getInputSummary by changing data structures and allowing multiple threads to do calculations. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 8937b43 ql/src/test/org/apache/hadoop/hive/ql/exec/TestGetInputSummary.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 90eb45b Diff: https://reviews.apache.org/r/69683/diff/4/ Changes: https://reviews.apache.org/r/69683/diff/3-4/ Testing --- Unit Thanks, David Mollitor
Re: Review Request 69683: [HIVE-21071] Improve getInputSummary
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69683/ --- (Updated Feb. 6, 2019, 8:34 p.m.) Review request for hive. Repository: hive-git Description --- Improve performance of method getInputSummary by changing data structures and allowing multiple threads to do calculations. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 8937b43 ql/src/test/org/apache/hadoop/hive/ql/exec/TestGetInputSummary.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 90eb45b Diff: https://reviews.apache.org/r/69683/diff/3/ Changes: https://reviews.apache.org/r/69683/diff/2-3/ Testing --- Unit Thanks, David Mollitor
Re: Review Request 69683: [HIVE-21071] Improve getInputSummary
> On Jan. 8, 2019, 10:04 a.m., Peter Vary wrote: > > Thanks for the patch! > > Two nits below. > > Also a bit concerned about the size calculation - seems ok, but it would be > > good to have a few test case which validates the contentsummary > > calculations (when every path is cached/only few patch is cached/no path is > > cached), so we can be sure that further changes will not break the > > functionality. > > > > What do you think? > > > > Peter > > David Mollitor wrote: > Thank you Peter for the review. This functionality is unit tested > already. Do you have suggestions for additional unit tests? > > > > https://github.com/apache/hive/blob/ae008b79b5d52ed6a38875b73025a505725828eb/ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java > > Peter Vary wrote: > I missed the ones which are testing the full getInputSummary method, and > found only the ones (testGetInputSummaryPool, > testGetInputSummaryPoolAndFailure) using the getInputSummaryWithPool. > Seeing those tests I feel much better. Maybe it is an edge case, but it > might be good to add a test where we call the getInputSummary twice. First > with one set of path (p1, p2) and then again with another set of path (p1, > p2, p3, p4) so we can check that the merge of the cached results and the > newly fetched ones is working. I created a new patch to include a unit test where a file path's summary status is stored in the cache. I moved all of the unit tests for getInputSummary into their own file because it made it easier since its functionality (and therefore testing) are much different than all the other Utility methods. This allowed me to implement JUnit setup() and teardown() methods. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69683/#review211758 --- On Feb. 6, 2019, 8:34 p.m., David Mollitor wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69683/ > --- > > (Updated Feb. 6, 2019, 8:34 p.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > --- > > Improve performance of method getInputSummary by changing data structures and > allowing multiple threads to do calculations. > > > Diffs > - > > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 8937b43 > ql/src/test/org/apache/hadoop/hive/ql/exec/TestGetInputSummary.java > PRE-CREATION > ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 90eb45b > > > Diff: https://reviews.apache.org/r/69683/diff/3/ > > > Testing > --- > > Unit > > > Thanks, > > David Mollitor > >
[jira] [Created] (HIVE-21224) Upgrade tests JUnit3 to JUnit4
Bruno Pusztahazi created HIVE-21224: --- Summary: Upgrade tests JUnit3 to JUnit4 Key: HIVE-21224 URL: https://issues.apache.org/jira/browse/HIVE-21224 Project: Hive Issue Type: Improvement Reporter: Bruno Pusztahazi Assignee: Bruno Pusztahazi Old JUnit3 tests should be upgraded to JUnit4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
