[jira] [Resolved] (IMPALA-6392) Explain format for parquet predicate statistics should be consistent with predicates
[ https://issues.apache.org/jira/browse/IMPALA-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6392. -- Resolution: Fixed > Explain format for parquet predicate statistics should be consistent with > predicates > > > Key: IMPALA-6392 > URL: https://issues.apache.org/jira/browse/IMPALA-6392 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.11.0 >Reporter: Vuk Ercegovac >Assignee: Fredy Wijaya >Priority: Minor > Labels: ramp-up > > In EXPLAIN_LEVEL 2, "parquet predicate statistics" combine conjuncts from > multiple tuple descriptors into one line: > parquet statistics predicates: c_custkey > 0, o.o_orderkey > 0, > l.l_partkey > 0 > However, "predicates" separate the conjuncts into a line per tuple descriptor: > predicates: c_custkey > 0, !empty(c.c_orders) > predicates on o: !empty(o.o_lineitems), o.o_orderkey > 0 > predicates on l: l.l_partkey > 0 > These two cases should be consistent. Prefer the line per tuple descriptor > style. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-5269) Comment on Final Line of Query Breaks Parsing
[ https://issues.apache.org/jira/browse/IMPALA-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-5269. -- Resolution: Fixed Fix Version/s: Impala 2.12.0 Impala 3.0 > Comment on Final Line of Query Breaks Parsing > - > > Key: IMPALA-5269 > URL: https://issues.apache.org/jira/browse/IMPALA-5269 > Project: IMPALA > Issue Type: Bug > Components: Clients >Reporter: Alan Jackoway >Assignee: Fredy Wijaya >Priority: Major > Labels: newbie > Fix For: Impala 3.0, Impala 2.12.0 > > > One of our users made a query with comments inline, including one on the end > of the query. This causes the query not to launch from impala shell. > To reproduce, open impala shell and run {{select 1 + 1; -- Not much of a > query}}. When you hit enter it won't run, then you have to add another > semicolon and hit enter again to get it to go. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6275) Successful CTAS logs warning
[ https://issues.apache.org/jira/browse/IMPALA-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6275. -- Resolution: Fixed > Successful CTAS logs warning > > > Key: IMPALA-6275 > URL: https://issues.apache.org/jira/browse/IMPALA-6275 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.11.0 >Reporter: Tim Armstrong >Assignee: Fredy Wijaya >Priority: Major > Labels: supportability > > I just noticed this on my development branch (branched off master at > 63f17e9ceaed92a28ea12567a36b746e54fffdb3). I'm not sure if it's new > behaviour, but it's potentially confusing. I ran a successful CTAS for a > table that didn't exist and got a stack trace in the warning log. > {noformat} > I1205 10:42:21.316118 14901 Frontend.java:909] Compiling query: create table > new_table as select "foo" > W1205 10:42:21.320310 14901 Table.java:187] Could not load column statistics > for: default.new_table > Java exception follows: > NoSuchObjectException(message:Specified database/table does not exist : > default.new_table) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result$get_table_statistics_req_resultStandardScheme.read(ThriftHiveMetastore.java) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result$get_table_statistics_req_resultStandardScheme.read(ThriftHiveMetastore.java) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result.read(ThriftHiveMetastore.java) > at > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_statistics_req(ThriftHiveMetastore.java:3117) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_statistics_req(ThriftHiveMetastore.java:3104) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTableColumnStatistics(HiveMetaStoreClient.java:1557) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:101) > at com.sun.proxy.$Proxy13.getTableColumnStatistics(Unknown Source) > at org.apache.impala.catalog.Table.loadAllColumnStats(Table.java:185) > at org.apache.impala.catalog.HdfsTable.loadSchema(HdfsTable.java:1527) > at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1182) > at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1146) > at > org.apache.impala.analysis.CreateTableAsSelectStmt.analyze(CreateTableAsSelectStmt.java:201) > at > org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:388) > at > org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:369) > at org.apache.impala.service.Frontend.analyzeStmt(Frontend.java:920) > at > org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1069) > at > org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:156) > I1205 10:42:21.320755 14901 Frontend.java:948] Compiled query. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6571) NullPointerException in SHOW CREATE TABLE for HBase table
Fredy Wijaya created IMPALA-6571: Summary: NullPointerException in SHOW CREATE TABLE for HBase table Key: IMPALA-6571 URL: https://issues.apache.org/jira/browse/IMPALA-6571 Project: IMPALA Issue Type: Bug Components: Frontend Reporter: Fredy Wijaya {code:java} [localhost:21000] > show create table functional_hbase.alltypes; Query: show create table functional_hbase.alltypes ERROR: NullPointerException: null {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6572) Upgrade Sentry version to 2.0.0
Fredy Wijaya created IMPALA-6572: Summary: Upgrade Sentry version to 2.0.0 Key: IMPALA-6572 URL: https://issues.apache.org/jira/browse/IMPALA-6572 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Fredy Wijaya Sentry version 2.0.0 is required to have fine-grained privileges such as granting/revoking on CREATE, ALTER, and DROP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6615) An insert query using a CTE does not show the expected output when executed in Impala-shell
[ https://issues.apache.org/jira/browse/IMPALA-6615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6615. -- Resolution: Fixed > An insert query using a CTE does not show the expected output when executed > in Impala-shell > --- > > Key: IMPALA-6615 > URL: https://issues.apache.org/jira/browse/IMPALA-6615 > Project: IMPALA > Issue Type: Bug > Components: Clients >Affects Versions: Impala 2.9.0 >Reporter: Luis E Martinez-Poblete >Assignee: Fredy Wijaya >Priority: Trivial > > Synopsis: > = > An insert query using a CTE does not show the expected output when executed > in Impala-shell > Problem: > > An insert query using a CTE does not produces the expected output when > executed in Impala-shell. In the bad case, the output does not show the web > links for "Query submitted" and " Query progress". > Reproduction case: > == > {noformat} > create table mytestCTE (c1 int, c2 int); > --Good case: > insert overwrite mytestCTE select * from mytestCTE; > --Output: > [host-10-17-101-88.coe.cloudera.com:21000] > insert overwrite mytestCTE > select * from mytestCTE; > Query: insert overwrite mytestCTE select * from mytestCTE > Query submitted at: 2018-03-06 12:19:02 (Coordinator: > http://host-10-17-101-88:25000) > Query progress can be monitored at: > http://host-10-17-101-88:25000/query_plan?query_id=f240a1e76c9edb69:3189ef56 > Modified 0 row(s) in 0.23s > --Bad case: > with dfl as (select * from mytestCTE) > insert overwrite mytestCTE > select * from dfl; > --Output: > [host-10-17-101-88.coe.cloudera.com:21000] > with dfl as (select * from > mytestCTE) > > insert overwrite mytestCTE > > select * from dfl; > Query: with dfl as (select * from mytestCTE) > insert overwrite mytestCTE > select * from dfl > Modified 0 row(s) in 0.12s{noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6643) Add fine-grained REFRESH privilege
Fredy Wijaya created IMPALA-6643: Summary: Add fine-grained REFRESH privilege Key: IMPALA-6643 URL: https://issues.apache.org/jira/browse/IMPALA-6643 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Fredy Wijaya Currently we only need ANY privilege in order to execute INVALIDATE METADATA or REFRESH . We need to have a more fine-grained privilege, i.e. REFRESH to allow executing INVALIDATE METADATA/REFRESH. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6647) Add fine-grained privileges to role
Fredy Wijaya created IMPALA-6647: Summary: Add fine-grained privileges to role Key: IMPALA-6647 URL: https://issues.apache.org/jira/browse/IMPALA-6647 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Fredy Wijaya Add CREATE, ALTER, DROP, and REFRESH fine-grained privileges to role. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6648) Add fine-grained privileges to role
Fredy Wijaya created IMPALA-6648: Summary: Add fine-grained privileges to role Key: IMPALA-6648 URL: https://issues.apache.org/jira/browse/IMPALA-6648 Project: IMPALA Issue Type: Epic Reporter: Fredy Wijaya Add CREATE, ALTER, DROP, and REFRESH fine-grained privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6649) Add fine-grained ALTER privilege
Fredy Wijaya created IMPALA-6649: Summary: Add fine-grained ALTER privilege Key: IMPALA-6649 URL: https://issues.apache.org/jira/browse/IMPALA-6649 Project: IMPALA Issue Type: Improvement Reporter: Fredy Wijaya Currently ALL privilege is required to alter a table. We need to have a more fine-grained privilege, i.e. ALTER to alter a table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6650) Add fine-grained DROP privilege
Fredy Wijaya created IMPALA-6650: Summary: Add fine-grained DROP privilege Key: IMPALA-6650 URL: https://issues.apache.org/jira/browse/IMPALA-6650 Project: IMPALA Issue Type: Improvement Reporter: Fredy Wijaya Currently ALL privilege is required to drop a database/table. We need to have a more fine-grained privilege, i.e. DROP to drop a database/table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6655) Set owner information on database creation
Fredy Wijaya created IMPALA-6655: Summary: Set owner information on database creation Key: IMPALA-6655 URL: https://issues.apache.org/jira/browse/IMPALA-6655 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya Currently Impala only shows owner information when using DESCRIBE DATABASE EXTENDED for databases created outside Impala. When a database is created inside Impala, the owner information is never set. For table creation, Impala always sets the owner information which can be shown by using DESCRIBE EXTENDED . To make the behavior consistent, we need to set the owner information on database creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6619) Alter table recover partitions creates unneeded partitions when faces percent sign
[ https://issues.apache.org/jira/browse/IMPALA-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6619. -- Resolution: Fixed Fix Version/s: Impala 2.13.0 Impala 2.12.0 Impala 3.0 > Alter table recover partitions creates unneeded partitions when faces percent > sign > -- > > Key: IMPALA-6619 > URL: https://issues.apache.org/jira/browse/IMPALA-6619 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.11.0 >Reporter: Miklos Szurap >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.0, Impala 2.12.0, Impala 2.13.0 > > > When a table has a partition with a special character in it's name, then the > HDFS directory contains a percent sign (due to an escaped/UrlEncoded > sequence). This is not decoded and compared properly when running {{alter > table recover partitions}}. This creates new, unneeded partitions on each > execution. > The steps to reproduce/demonstrate the issue: > {noformat} > [nightly-2:21000] > CREATE TABLE tbl_with_partition(col1 string) partitioned > by (p string); > Query: CREATE TABLE tbl_with_partition(col1 string) partitioned by (p string) > Fetched 0 row(s) in 1.08s > [nightly-2:21000] > ALTER TABLE tbl_with_partition add partition (p='100%'); > Query: ALTER TABLE tbl_with_partition add partition (p='100%') > Fetched 0 row(s) in 5.72s > [nightly-2:21000] > show partitions tbl_with_partition; > Query: show partitions tbl_with_partition > +---+---++--+--+---++---++ > | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | > Incremental stats | Location | > +---+---++--+--+---++---++ > | 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 | > | Total | -1 | 0 | 0B | 0B | | | | | > +---+---++--+--+---++---++ > Fetched 2 row(s) in 0.02s > [nightly-2:21000] > ALTER TABLE tbl_with_partition recover partitions; > Query: ALTER TABLE tbl_with_partition recover partitions > Fetched 0 row(s) in 0.29s > [nightly-2:21000] > show partitions tbl_with_partition; > Query: show partitions tbl_with_partition > ++---++--+--+---++---+--+ > | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | > Incremental stats | Location | > ++---++--+--+---++---+--+ > | 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 | > | 100%25 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%2525 | > | Total | -1 | 0 | 0B | 0B | | | | | > ++---++--+--+---++---+--+ > Fetched 3 row(s) in 0.02s > [nightly-2:21000] > ALTER TABLE tbl_with_partition recover partitions; > Query: ALTER TABLE tbl_with_partition recover partitions > Fetched 0 row(s) in 0.27s > [nightly-2:21000] > show partitions tbl_with_partition; > Query: show partitions tbl_with_partition > +--+---++--+--+---++---++ > | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | > Incremental stats | Location | > +--+---++--+--+---++---++ > | 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 | > | 100%25 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%2525 | > | 100%2525 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%252525 | > | Total | -1 | 0 | 0B | 0B | | | | | > +--+---++--+--+---++---+
[jira] [Created] (IMPALA-6672) test_recover_partitions.py should not be skipped on local FS
Fredy Wijaya created IMPALA-6672: Summary: test_recover_partitions.py should not be skipped on local FS Key: IMPALA-6672 URL: https://issues.apache.org/jira/browse/IMPALA-6672 Project: IMPALA Issue Type: Bug Components: Infrastructure Reporter: Fredy Wijaya Assignee: Fredy Wijaya [Remove @SkipIfLocal.hdfs_client|mailto:Remove%C2%A0@SkipIfLocal.hdfs_client] in test_recover_partitions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6674) Add end-to-end authorization testing
Fredy Wijaya created IMPALA-6674: Summary: Add end-to-end authorization testing Key: IMPALA-6674 URL: https://issues.apache.org/jira/browse/IMPALA-6674 Project: IMPALA Issue Type: Improvement Components: Infrastructure Reporter: Fredy Wijaya Currently most authorization testing is done in the unit tests. We need an end-to-end testing for the authorization. This allows us to tests few things related to authorization, such as show grant, grant/revoke, some queries, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6686) Change the DESCRIBE output to look more like Hive output
Fredy Wijaya created IMPALA-6686: Summary: Change the DESCRIBE output to look more like Hive output Key: IMPALA-6686 URL: https://issues.apache.org/jira/browse/IMPALA-6686 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6655) Set owner information on database creation
[ https://issues.apache.org/jira/browse/IMPALA-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6655. -- Resolution: Done > Set owner information on database creation > -- > > Key: IMPALA-6655 > URL: https://issues.apache.org/jira/browse/IMPALA-6655 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Currently Impala only shows owner information when using DESCRIBE DATABASE > EXTENDED for databases created outside Impala. When a database is > created inside Impala, the owner information is never set. For table > creation, Impala always sets the owner information which can be shown by > using DESCRIBE EXTENDED . To make the behavior consistent, we need to > set the owner information on database creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6707) Create an upgrade script
Fredy Wijaya created IMPALA-6707: Summary: Create an upgrade script Key: IMPALA-6707 URL: https://issues.apache.org/jira/browse/IMPALA-6707 Project: IMPALA Issue Type: Sub-task Reporter: Fredy Wijaya To ensure smooth upgrade for these new fine-grained privileges, we need to provide an upgrade script. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6643) Add fine-grained REFRESH privilege
[ https://issues.apache.org/jira/browse/IMPALA-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6643. -- Resolution: Done > Add fine-grained REFRESH privilege > -- > > Key: IMPALA-6643 > URL: https://issues.apache.org/jira/browse/IMPALA-6643 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Currently we only need ANY privilege in order to execute INVALIDATE METADATA > or REFRESH. We need to have a more fine-grained privilege, i.e. REFRESH to > allow executing INVALIDATE METADATA/REFRESH. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6724) Incorrect exception handling in creating a function
Fredy Wijaya created IMPALA-6724: Summary: Incorrect exception handling in creating a function Key: IMPALA-6724 URL: https://issues.apache.org/jira/browse/IMPALA-6724 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.11.0 Reporter: Fredy Wijaya {noformat} [localhost:21000] > create function foo.sin() returns int location '/test-warehouse/libTestUdfs.so' symbol='NoArgs'; Query: create function foo.sin() returns int location '/test-warehouse/libTestUdfs.so' symbol='NoArgs' Fetched 0 row(s) in 0.02s {noformat} To match the behavior above, the statement below should be allowed. {noformat} [localhost:21000] > use foo; Query: use foo [localhost:21000] > create function sin() returns int location '/test-warehouse/libTestUdfs.so' symbol='NoArgs'; Query: create function sin() returns int location '/test-warehouse/libTestUdfs.so' symbol='NoArgs' ERROR: AnalysisException: Function cannot have the same name as a builtin: sin {noformat} The statement below should trigger an AuthorizationException("Can not modify system database.") {noformat} [localhost:21000] > create function _impala_builtins.sin() returns int location '/test-warehouse/libTestUdfs.so' symbol='NoArgs'; Query: create function _impala_builtins.sin() returns int location '/test-warehouse/libTestUdfs.so' symbol='NoArgs' ERROR: AnalysisException: Function cannot have the same name as a builtin: sin {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6647) Add fine-grained CREATE privilege
[ https://issues.apache.org/jira/browse/IMPALA-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6647. -- Resolution: Done > Add fine-grained CREATE privilege > - > > Key: IMPALA-6647 > URL: https://issues.apache.org/jira/browse/IMPALA-6647 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Currently ALL privilege is required to create a table or database. We need to > have a more fine-grained privilege, i.e. CREATE for database/table creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6749) Implement ALTER DATABASE SET DBPROPERTIES
Fredy Wijaya created IMPALA-6749: Summary: Implement ALTER DATABASE SET DBPROPERTIES Key: IMPALA-6749 URL: https://issues.apache.org/jira/browse/IMPALA-6749 Project: IMPALA Issue Type: New Feature Components: Frontend Reporter: Fredy Wijaya Syntax: {noformat} ALTER DATABASE database_name SET DBPROPERTIES (property_name=property_value, ...) {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6672) @SkipIfLocal.hdfs_client should not be enabled for tests that can be executed on local FS
[ https://issues.apache.org/jira/browse/IMPALA-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6672. -- Resolution: Cannot Reproduce > @SkipIfLocal.hdfs_client should not be enabled for tests that can be executed > on local FS > - > > Key: IMPALA-6672 > URL: https://issues.apache.org/jira/browse/IMPALA-6672 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > > A follow-up task for this review: > https://gerrit.cloudera.org/c/9564/3/tests/metadata/test_recover_partitions.py#343 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6775) Inconsistent ALTER TABLE SET LOCATION output format
Fredy Wijaya created IMPALA-6775: Summary: Inconsistent ALTER TABLE SET LOCATION output format Key: IMPALA-6775 URL: https://issues.apache.org/jira/browse/IMPALA-6775 Project: IMPALA Issue Type: Bug Components: Frontend Reporter: Fredy Wijaya No tabular output. {noformat} [localhost:21000] > alter table functional.alltypesagg partition(year=2009, month=1) set location 'hdfs://localhost:20500/test-warehouse/new_table'; Query: alter table functional.alltypesagg partition(year=2009, month=1) set location 'hdfs://localhost:20500/test-warehouse/new_table' Fetched 0 row(s) in 0.05s {noformat} vs others: {noformat} [localhost:21000] > alter table functional.alltypesagg partition(year=2009, month=1) set fileformat parquet; Query: alter table functional.alltypesagg partition(year=2009, month=1) set fileformat parquet +-+ | summary | +-+ | Updated 0 partition(s). | +-+ Fetched 1 row(s) in 0.02s {noformat} {noformat} [localhost:21000] > alter table functional.alltypesagg partition(year=2009, month=1) set tblproperties ('key'='value'); Query: alter table functional.alltypesagg partition(year=2009, month=1) set tblproperties ('key'='value') +-+ | summary | +-+ | Updated 0 partition(s). | +-+ Fetched 1 row(s) in 0.02s {noformat} {noformat} [localhost:21000] > alter table functional.alltypesagg partition(year=2009, month=1) set tblproperties ('key'='value'); Query: alter table functional.alltypesagg partition(year=2009, month=1) set tblproperties ('key'='value') +-+ | summary | +-+ | Updated 0 partition(s). | +-+ Fetched 1 row(s) in 0.02s {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6739) Exception in ALTER TABLE SET statements
[ https://issues.apache.org/jira/browse/IMPALA-6739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6739. -- Resolution: Fixed Fix Version/s: Impala 2.13.0 Impala 2.12.0 Impala 3.0 > Exception in ALTER TABLE SET statements > --- > > Key: IMPALA-6739 > URL: https://issues.apache.org/jira/browse/IMPALA-6739 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Reporter: Adam Holley >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 3.0, Impala 2.12.0, Impala 2.13.0 > > > {noformat} > [localhost:21000] > alter table functional.alltypesagg partition(year=2009, > month=1) set location 'hdfs://localhost:20500/test-warehouse/new_table'; > Query: alter table functional.alltypesagg partition(year=2009, month=1) set > location 'hdfs://localhost:20500/test-warehouse/new_table' > ERROR: IndexOutOfBoundsException: toIndex = 3 > {noformat} > {noformat} > [localhost:21000] > alter table functional.alltypesagg partition(year=2009, > month=1) set fileformat parquet; > Query: alter table functional.alltypesagg partition(year=2009, month=1) set > fileformat parquet > ERROR: IllegalStateException: null > {noformat} > {noformat} > [localhost:21000] > alter table functional.alltypesagg partition(year=2009, > month=1) set row format delimited fields terminated by ' '; > Query: alter table functional.alltypesagg partition(year=2009, month=1) set > row format delimited fields terminated by ' ' > ERROR: IllegalStateException: null > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6719) refresh function case sensitivity
[ https://issues.apache.org/jira/browse/IMPALA-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6719. -- Resolution: Fixed Fix Version/s: Impala 2.13.0 Impala 2.12.0 Impala 3.0 > refresh function case sensitivity > - > > Key: IMPALA-6719 > URL: https://issues.apache.org/jira/browse/IMPALA-6719 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.0 >Reporter: Vuk Ercegovac >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.0, Impala 2.12.0, Impala 2.13.0 > > > "refresh functions " has odd behavior for upper-case : > create database FOO; > OK > refresh functions FOO; > ERROR: CatalogException: Error refreshing functions in FOO: > CAUSED BY: DatabaseNotFoundException: Database does not exist: FOO > refresh functions foo; > OK > drop database FOO; > OK -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6724) Allow creating/dropping functions with the same name as built-ins
[ https://issues.apache.org/jira/browse/IMPALA-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6724. -- Resolution: Fixed Fix Version/s: Impala 3.0 > Allow creating/dropping functions with the same name as built-ins > - > > Key: IMPALA-6724 > URL: https://issues.apache.org/jira/browse/IMPALA-6724 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.0 > > > {noformat} > [localhost:21000] > create function foo.sin() returns int location > '/test-warehouse/libTestUdfs.so' symbol='NoArgs'; > Query: create function foo.sin() returns int location > '/test-warehouse/libTestUdfs.so' symbol='NoArgs' > Fetched 0 row(s) in 0.02s > {noformat} > To match the behavior above, the statement below should be allowed. > {noformat} > [localhost:21000] > use foo; Query: use foo [localhost:21000] > create > function sin() returns int location '/test-warehouse/libTestUdfs.so' > symbol='NoArgs'; Query: create function sin() returns int location > '/test-warehouse/libTestUdfs.so' symbol='NoArgs' ERROR: AnalysisException: > Function cannot have the same name as a builtin: sin > {noformat} > The statement below should trigger an AuthorizationException("Can not modify > system database.") > {noformat} > [localhost:21000] > create function _impala_builtins.sin() returns int > location '/test-warehouse/libTestUdfs.so' symbol='NoArgs'; > Query: create function _impala_builtins.sin() returns int location > '/test-warehouse/libTestUdfs.so' symbol='NoArgs' > ERROR: AnalysisException: Function cannot have the same name as a builtin: sin > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6674) Add end-to-end authorization testing
[ https://issues.apache.org/jira/browse/IMPALA-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6674. -- Resolution: Information Provided > Add end-to-end authorization testing > > > Key: IMPALA-6674 > URL: https://issues.apache.org/jira/browse/IMPALA-6674 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Priority: Major > > Currently most authorization testing is done in the unit tests. We need an > end-to-end testing for the authorization. This allows us to tests few things > related to authorization, such as show grant, grant/revoke, some queries, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6800) Add missing test cases in statements that require ALTER privilege
Fredy Wijaya created IMPALA-6800: Summary: Add missing test cases in statements that require ALTER privilege Key: IMPALA-6800 URL: https://issues.apache.org/jira/browse/IMPALA-6800 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya Some missing test cases include COMPUTE STATS, DROP STATS, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6571) NullPointerException in SHOW CREATE TABLE for HBase tables
[ https://issues.apache.org/jira/browse/IMPALA-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6571. -- Resolution: Fixed Fix Version/s: Impala 2.13.0 Impala 2.12.0 Impala 3.0 > NullPointerException in SHOW CREATE TABLE for HBase tables > -- > > Key: IMPALA-6571 > URL: https://issues.apache.org/jira/browse/IMPALA-6571 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.10.0, Impala 2.11.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.0, Impala 2.12.0, Impala 2.13.0 > > > {code:java} > [localhost:21000] > show create table functional_hbase.alltypes; > Query: show create table functional_hbase.alltypes > ERROR: NullPointerException: null{code} > {color:#33}Stack trace:{color} > {code:java} > I0223 09:16:10.156831 29372 jni-util.cc:230] java.lang.NullPointerException > at > com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187) > at > org.apache.impala.catalog.HdfsFileFormat.fromHdfsInputFormatClass(HdfsFileFormat.java:128) > at > org.apache.impala.analysis.ToSqlUtils.getCreateTableSql(ToSqlUtils.java:234) > at > org.apache.impala.service.JniFrontend.showCreateTable(JniFrontend.java:458) > I0223 09:16:10.233009 29372 status.cc:125] NullPointerException: null > @ 0x167f6cd impala::Status::Status() > @ 0x1adb174 impala::JniUtil::GetJniExceptionMsg() > @ 0x1974ea7 impala::JniUtil::CallJniMethod<>() > @ 0x19728d1 impala::Frontend::ShowCreateTable() > @ 0x19f908b impala::ClientRequestState::ExecLocalCatalogOp() > @ 0x19fa8f5 impala::ClientRequestState::ExecDdlRequest() > @ 0x19f6d9e impala::ClientRequestState::Exec() > @ 0x1996350 impala::ImpalaServer::ExecuteInternal() > @ 0x1995b32 impala::ImpalaServer::Execute() > @ 0x1a1dd48 impala::ImpalaServer::query() > @ 0x2b2aace beeswax::BeeswaxServiceProcessor::process_query() > @ 0x2b2a81c beeswax::BeeswaxServiceProcessor::dispatchCall() > @ 0x2b1280b impala::ImpalaServiceProcessor::dispatchCall() > @ 0x1624290 apache::thrift::TDispatchProcessor::process() > @ 0x1801a5f > apache::thrift::server::TAcceptQueueServer::Task::run() > @ 0x17f96c3 impala::ThriftThread::RunRunnable() > @ 0x17fadc7 boost::_mfi::mf2<>::operator()() > @ 0x17fac5d boost::_bi::list3<>::operator()<>(){code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6800) Add missing test cases in statements that require ALTER privilege
[ https://issues.apache.org/jira/browse/IMPALA-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6800. -- Resolution: Done > Add missing test cases in statements that require ALTER privilege > - > > Key: IMPALA-6800 > URL: https://issues.apache.org/jira/browse/IMPALA-6800 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > > Some missing test cases include COMPUTE STATS, DROP STATS, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6649) Add fine-grained ALTER privilege
[ https://issues.apache.org/jira/browse/IMPALA-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6649. -- Resolution: Done > Add fine-grained ALTER privilege > > > Key: IMPALA-6649 > URL: https://issues.apache.org/jira/browse/IMPALA-6649 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Adam Holley >Priority: Major > Fix For: Impala 3.0, Impala 2.13.0 > > > Currently ALL privilege is required to alter a table. We need to have a more > fine-grained privilege, i.e. ALTER to alter a table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6802) Clean up tests in AuthorizationTest
Fredy Wijaya created IMPALA-6802: Summary: Clean up tests in AuthorizationTest Key: IMPALA-6802 URL: https://issues.apache.org/jira/browse/IMPALA-6802 Project: IMPALA Issue Type: Improvement Reporter: Fredy Wijaya The tests in AuthorizationTest can be refactored to be more condensed. We also need to increase the code coverage. It is also a good idea to refactor these tests to provide flexibility in supporting tests for user-level privileges in the future: https://issues.apache.org/jira/browse/IMPALA-6794 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6804) Allow SELECT and INSERT privileges on SERVER
Fredy Wijaya created IMPALA-6804: Summary: Allow SELECT and INSERT privileges on SERVER Key: IMPALA-6804 URL: https://issues.apache.org/jira/browse/IMPALA-6804 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6805) Show current database in Impala shell
Fredy Wijaya created IMPALA-6805: Summary: Show current database in Impala shell Key: IMPALA-6805 URL: https://issues.apache.org/jira/browse/IMPALA-6805 Project: IMPALA Issue Type: Improvement Components: Clients Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6650) Add fine-grained DROP privilege
[ https://issues.apache.org/jira/browse/IMPALA-6650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6650. -- Resolution: Done > Add fine-grained DROP privilege > --- > > Key: IMPALA-6650 > URL: https://issues.apache.org/jira/browse/IMPALA-6650 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Currently ALL privilege is required to drop a database/table. We need to have > a more fine-grained privilege, i.e. DROP to drop a database/table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6817) Clean up Impala authorization model
Fredy Wijaya created IMPALA-6817: Summary: Clean up Impala authorization model Key: IMPALA-6817 URL: https://issues.apache.org/jira/browse/IMPALA-6817 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya A follow up for this review: [https://gerrit.cloudera.org/c/9716/8/fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaActionFactory.java#32] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6817) Clean up Impala privilege model
[ https://issues.apache.org/jira/browse/IMPALA-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6817. -- Resolution: Done Fix Version/s: Impala 2.13.0 Impala 3.0 > Clean up Impala privilege model > --- > > Key: IMPALA-6817 > URL: https://issues.apache.org/jira/browse/IMPALA-6817 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.0, Impala 2.13.0 > > > A follow up for this review: > [https://gerrit.cloudera.org/c/9716/8/fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaActionFactory.java#32] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6572) Upgrade Sentry version to 2.0.0
[ https://issues.apache.org/jira/browse/IMPALA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6572. -- Resolution: Duplicate > Upgrade Sentry version to 2.0.0 > --- > > Key: IMPALA-6572 > URL: https://issues.apache.org/jira/browse/IMPALA-6572 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Fredy Wijaya >Priority: Critical > > Sentry version 2.0.0 is required to have fine-grained privileges such as > granting/revoking on CREATE, ALTER, and DROP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6805) Show current database in Impala shell prompt
[ https://issues.apache.org/jira/browse/IMPALA-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6805. -- Resolution: Done Fix Version/s: Impala 2.12.0 Impala 3.0 > Show current database in Impala shell prompt > > > Key: IMPALA-6805 > URL: https://issues.apache.org/jira/browse/IMPALA-6805 > Project: IMPALA > Issue Type: Improvement > Components: Clients >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Labels: shell > Fix For: Impala 3.0, Impala 2.12.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6809) bootstrap_system.sh does an unconditional git clone to ~/Impala
[ https://issues.apache.org/jira/browse/IMPALA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6809. -- Resolution: Done Fix Version/s: Impala 2.12.0 Impala 3.0 > bootstrap_system.sh does an unconditional git clone to ~/Impala > --- > > Key: IMPALA-6809 > URL: https://issues.apache.org/jira/browse/IMPALA-6809 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 2.11.0 >Reporter: Gabor Kaszab >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.0, Impala 2.12.0 > > > bootstrap_system.sh checks if ~/Impala is an existing directory, and if it > isn't then clones Impala to that path. Once this is done this script jumps to > ~/Impala and starts a build. > In case someone clones the Impala sources to a custom directory, sources the > Impala configs there and then invokes bootstrap_development.sh or > bootstrap_system.sh then in the background the build is actually performed in > ~/Impala instead of the custom dir. Taking into account that Impala is not > sourced from ~/Impala, but the build is done there, this can result some > strange build failures. > My proposal is to move the following code behind a command line parameter > that is off by default: > {code:java} > # If there is no Impala git repo, get one now > if ! [[ -d ~/Impala ]] > then > time -p git clone https://git-wip-us.apache.org/repos/asf/impala.git > ~/Impala > fi > cd ~/Impala > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6804) Allow SELECT and INSERT privileges on SERVER
[ https://issues.apache.org/jira/browse/IMPALA-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6804. -- Resolution: Done Fix Version/s: Impala 3.0 > Allow SELECT and INSERT privileges on SERVER > > > Key: IMPALA-6804 > URL: https://issues.apache.org/jira/browse/IMPALA-6804 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Adam Holley >Priority: Major > Fix For: Impala 3.0 > > > {noformat} > grant select on server to role test_role;{noformat} > {noformat} > grant insert on server to role test_role;{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6850) Print the actual error message to the console when Sentry fails
Fredy Wijaya created IMPALA-6850: Summary: Print the actual error message to the console when Sentry fails Key: IMPALA-6850 URL: https://issues.apache.org/jira/browse/IMPALA-6850 Project: IMPALA Issue Type: Improvement Components: Infrastructure Reporter: Fredy Wijaya Assignee: Fredy Wijaya When Sentry fails to start, this is currently what gets printed into the console. {noformat} /home/user/Impala/testdata/bin/run-all.sh at line 60: $IMPALA_HOME/testdata/bin/run-sentry-service.sh > \ {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6451) Creating a Kudu table with CTAS fails with AuthorizationException: User 'username' does not have privileges to access: server1
[ https://issues.apache.org/jira/browse/IMPALA-6451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6451. -- Resolution: Fixed Fix Version/s: Impala 2.12.0 Impala 3.0 > Creating a Kudu table with CTAS fails with AuthorizationException: User > 'username' does not have privileges to access: server1 > -- > > Key: IMPALA-6451 > URL: https://issues.apache.org/jira/browse/IMPALA-6451 > Project: IMPALA > Issue Type: Bug > Components: Frontend, Security >Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0 >Reporter: Tomas Farkas >Assignee: Fredy Wijaya >Priority: Critical > Labels: security, usability > Fix For: Impala 3.0, Impala 2.12.0 > > > In a Sentry enabled Kerberized environment user cannot create a Kudu table > from a SELECT statement if it has a function, even though it has an ALL > privilege on work database. > > {quote}create table work.tfsource ( i bigint , s string ); > insert into work.tfsource select 1, 'Test row'; > create table work.tfdest primary key ( i ) partition by hash ( i ) partitions > 5 stored as kudu as > select `i`, `s`, {color:#205081}current_timestamp(){color} as ins_date from > `work`.`tfsource` limit 5; > {quote} > {color:#ff}AuthorizationException: User '' does not have privileges > to access: server1{color} > {quote}create table work.tfsource ( i bigint , s string ); > insert into work.tfsource select 1, 'Test row'; > create table work.tfdest primary key ( i ) partition by hash ( i ) partitions > 5 stored as kudu as > select `i`, `s`, 0 as ins_date from `work`.`tfsource` limit 5; > {quote} > {color:#14892c}Inserted 1 row(s){color} > {color:#14892c} {color} > *It seems that if in the select statement is any function (tested now(), > day("2018-02-16"0) then it requires server level ALL priviliges.* > *{{Kudu version:}}* > {{kudu 1.5.0-cdh5.13.1 revision 9044f1f377a6f2044d800600e5c2dc5e989c4dd4 > build type RELEASE built by jenkins at 09 Nov 2017 08:50:42 PST on > impala-ec2-pkg-centos-7-0c27.vpc.cloudera.com build id 2017-11-09_08-09-26}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6850) Print the actual error message to the console when Sentry fails
[ https://issues.apache.org/jira/browse/IMPALA-6850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6850. -- Resolution: Done > Print the actual error message to the console when Sentry fails > --- > > Key: IMPALA-6850 > URL: https://issues.apache.org/jira/browse/IMPALA-6850 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 3.0, Impala 2.12.0 > > > When Sentry fails to start, this is currently what gets printed into the > console. > {noformat} > /home/user/Impala/testdata/bin/run-all.sh at line 60: > $IMPALA_HOME/testdata/bin/run-sentry-service.sh > \ > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6862) Privilege.java needs to support Sentry 1.5.1 and Sentry 2.0.0 API
Fredy Wijaya created IMPALA-6862: Summary: Privilege.java needs to support Sentry 1.5.1 and Sentry 2.0.0 API Key: IMPALA-6862 URL: https://issues.apache.org/jira/browse/IMPALA-6862 Project: IMPALA Issue Type: Bug Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya In Sentry 1.5.1, BitFieldAction is an abstract class: [https://github.com/apache/sentry/blob/release-1.5.1/sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/BitFieldAction.java#L22]. Sentry removed the abstract requirement in Sentry 2.0.0: [https://github.com/apache/sentry/blob/master/sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/BitFieldAction.java#L23] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6775) Inconsistent ALTER TABLE SET LOCATION output format
[ https://issues.apache.org/jira/browse/IMPALA-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6775. -- Resolution: Duplicate > Inconsistent ALTER TABLE SET LOCATION output format > --- > > Key: IMPALA-6775 > URL: https://issues.apache.org/jira/browse/IMPALA-6775 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.9.0, Impala 2.10.0, Impala 2.11.0 >Reporter: Fredy Wijaya >Priority: Major > > No tabular output. > {noformat} > [localhost:21000] > alter table functional.alltypesagg partition(year=2009, > month=1) set location 'hdfs://localhost:20500/test-warehouse/new_table'; > Query: alter table functional.alltypesagg partition(year=2009, month=1) set > location 'hdfs://localhost:20500/test-warehouse/new_table' > Fetched 0 row(s) in 0.05s > {noformat} > vs others: > {noformat} > [localhost:21000] > alter table functional.alltypesagg partition(year=2009, > month=1) set fileformat parquet; > Query: alter table functional.alltypesagg partition(year=2009, month=1) set > fileformat parquet > +-+ > | summary | > +-+ > | Updated 0 partition(s). | > +-+ > Fetched 1 row(s) in 0.02s > {noformat} > {noformat} > [localhost:21000] > alter table functional.alltypesagg partition(year=2009, > month=1) set tblproperties ('key'='value'); > Query: alter table functional.alltypesagg partition(year=2009, month=1) set > tblproperties ('key'='value') > +-+ > | summary | > +-+ > | Updated 0 partition(s). | > +-+ > Fetched 1 row(s) in 0.02s > {noformat} > {noformat} > [localhost:21000] > alter table functional.alltypesagg partition(year=2009, > month=1) set tblproperties ('key'='value'); > Query: alter table functional.alltypesagg partition(year=2009, month=1) set > tblproperties ('key'='value') > +-+ > | summary | > +-+ > | Updated 0 partition(s). | > +-+ > Fetched 1 row(s) in 0.02s > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6749) Implement ALTER DATABASE SET DBPROPERTIES
[ https://issues.apache.org/jira/browse/IMPALA-6749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6749. -- Resolution: Won't Do > Implement ALTER DATABASE SET DBPROPERTIES > - > > Key: IMPALA-6749 > URL: https://issues.apache.org/jira/browse/IMPALA-6749 > Project: IMPALA > Issue Type: New Feature > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Syntax: > {noformat} > ALTER DATABASE database_name SET DBPROPERTIES (property_name=property_value, > ...) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6862) Privilege.java needs to support Sentry 1.5.1 and Sentry 2.0.0 API
[ https://issues.apache.org/jira/browse/IMPALA-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6862. -- Resolution: Fixed Fix Version/s: Impala 3.0 > Privilege.java needs to support Sentry 1.5.1 and Sentry 2.0.0 API > - > > Key: IMPALA-6862 > URL: https://issues.apache.org/jira/browse/IMPALA-6862 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.0 > > > In Sentry 1.5.1, BitFieldAction is an abstract class: > [https://github.com/apache/sentry/blob/release-1.5.1/sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/BitFieldAction.java#L22]. > Sentry removed the abstract requirement in Sentry 2.0.0: > [https://github.com/apache/sentry/blob/master/sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/BitFieldAction.java#L23] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6878) SentryServicePinger should not print stacktrace when retrying
Fredy Wijaya created IMPALA-6878: Summary: SentryServicePinger should not print stacktrace when retrying Key: IMPALA-6878 URL: https://issues.apache.org/jira/browse/IMPALA-6878 Project: IMPALA Issue Type: Improvement Components: Infrastructure Reporter: Fredy Wijaya Assignee: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6881) COMPUTE STATS should check for SELECT privilege at analysis
Fredy Wijaya created IMPALA-6881: Summary: COMPUTE STATS should check for SELECT privilege at analysis Key: IMPALA-6881 URL: https://issues.apache.org/jira/browse/IMPALA-6881 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.11.0, Impala 2.10.0, Impala 2.9.0 Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6883) [DOCS] Clean up Impala authorization doc
Fredy Wijaya created IMPALA-6883: Summary: [DOCS] Clean up Impala authorization doc Key: IMPALA-6883 URL: https://issues.apache.org/jira/browse/IMPALA-6883 Project: IMPALA Issue Type: Improvement Components: Docs Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6887) Typo in authz-policy.ini.template
Fredy Wijaya created IMPALA-6887: Summary: Typo in authz-policy.ini.template Key: IMPALA-6887 URL: https://issues.apache.org/jira/browse/IMPALA-6887 Project: IMPALA Issue Type: Bug Components: Infrastructure Reporter: Fredy Wijaya Assignee: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6878) SentryServicePinger should not print stacktrace at every retry
[ https://issues.apache.org/jira/browse/IMPALA-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6878. -- Resolution: Done > SentryServicePinger should not print stacktrace at every retry > -- > > Key: IMPALA-6878 > URL: https://issues.apache.org/jira/browse/IMPALA-6878 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > > The stack trace looks like this and is misleading as to whether the service > started successfully or not: > {code:java} > 18/04/18 12:03:23 INFO transport.SentryTransportPool: Creating pool for > localhost with default port 30911 > 18/04/18 12:03:23 INFO transport.SentryTransportPool: Adding endpoint > localhost:30911 > 18/04/18 12:03:23 INFO transport.SentryTransportPool: Connection pooling is > enabled > 18/04/18 12:03:23 ERROR transport.SentryTransportPool: Failed to obtain > transport for localhost:30911: java.net.ConnectException: Connection refused > (Connection refused) > 18/04/18 12:03:23 ERROR transport.RetryClientInvocationHandler: Failed to > connect > sentry.org.apache.thrift.transport.TTransportException: > java.net.ConnectException: Connection refused (Connection refused) > at sentry.org.apache.thrift.transport.TSocket.open(TSocket.java:226) > at > sentry.org.apache.sentry.core.common.transport.SentryTransportFactory.connectToServer(SentryTransportFactory.java:99) > at > sentry.org.apache.sentry.core.common.transport.SentryTransportFactory.getTransport(SentryTransportFactory.java:86) > at > sentry.org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:302) > at > sentry.org.apache.sentry.core.common.transport.SentryTransportPool$PoolFactory.create(SentryTransportPool.java:271) > at > org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62) > at > org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041) > at > org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:380) > at > org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:279) > at > sentry.org.apache.sentry.core.common.transport.SentryTransportPool.getTransport(SentryTransportPool.java:183) > at > org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.connect(SentryPolicyServiceClientDefaultImpl.java:90) > at > sentry.org.apache.sentry.core.common.transport.RetryClientInvocationHandler.connect(RetryClientInvocationHandler.java:141) > at > sentry.org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:90) > at > sentry.org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41) > at com.sun.proxy.$Proxy5.listAllRoles(Unknown Source) > at org.apache.impala.util.SentryUtil.listRoles(SentryUtil.java:52) > at > org.apache.impala.util.SentryPolicyService.listAllRoles(SentryPolicyService.java:398) > at > org.apache.impala.testutil.SentryServicePinger.main(SentryServicePinger.java:75) > Caused by: java.net.ConnectException: Connection refused (Connection refused) > at java.net.PlainSocketImpl.socketConnect(Native Method) > at > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) > at > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) > at > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:589) > at sentry.org.apache.thrift.transport.TSocket.open(TSocket.java:221) > ... 17 more > 18/04/18 12:03:26 ERROR transport.SentryTransportPool: Failed to obtain > transport for localhost:30911: java.net.ConnectException: Connection refused > (Connection refused) > 18/04/18 12:03:26 ERROR transport.RetryClientInvocationHandler: Failed to > connect > sentry.org.apache.thrift.transport.TTransportException: > java.net.ConnectException: Connection refused (Connection refused) > at sentry.org.apache.thrift.transport.TSocket.open(TSocket.java:226) > at > sentry.org.apache.sentry.core.common.transport.SentryTransportFactory.connectToServer(SentryTransportFactory.java:99) > at > sentry.org.apache.sentry.core.common.transport.SentryTransportFactory.getTransport(SentryTransportFactory.java:86) > at > sentry.org.apache.sentry.core.common.transport.SentryTransportPool$Po
[jira] [Resolved] (IMPALA-6887) Typo in authz-policy.ini.template
[ https://issues.apache.org/jira/browse/IMPALA-6887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6887. -- Resolution: Fixed Fix Version/s: Impala 3.0 > Typo in authz-policy.ini.template > - > > Key: IMPALA-6887 > URL: https://issues.apache.org/jira/browse/IMPALA-6887 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 3.0 > > > {noformat} > alter_functionl_text_lzo = > server=server1->db=functional_text_lzo->action=alter > {noformat} > Although it does not affect the tests directly, this typo could potentially > be an issue since we register the role with alter_functional_text_lzo. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6891) AuthorizationException in CROSS JOIN
Fredy Wijaya created IMPALA-6891: Summary: AuthorizationException in CROSS JOIN Key: IMPALA-6891 URL: https://issues.apache.org/jira/browse/IMPALA-6891 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.11.0, Impala 2.10.0, Impala 2.9.0 Reporter: Fredy Wijaya {noformat} [localhost:21000] foo> create table t1(i int, j int); [localhost:21000] foo> create table t2(i int, j int); [localhost:21000] foo> grant select(i) on table foo.t1 to role test_role; [localhost:21000] foo> grant select(j) on table foo.t1 to role test_role; [localhost:21000] foo> grant select(i) on table foo.t2 to role test_role; [localhost:21000] foo> grant select(j) on table foo.t2 to role test_role;{noformat} {noformat} [localhost:21000] foo> select * from foo.t1 a cross join foo.t2 b; Fetched 0 row(s) in 0.14s {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6896) NullPointerException in DESCRIBE FORMATTED on views
Fredy Wijaya created IMPALA-6896: Summary: NullPointerException in DESCRIBE FORMATTED on views Key: IMPALA-6896 URL: https://issues.apache.org/jira/browse/IMPALA-6896 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.11.0 Reporter: Fredy Wijaya {noformat} impala-shell -i localhost:21000 (first impalad) [localhost:21000] default> create view v1 as select * from functional.alltypes; [localhost:21000] default> alter view v1 as select * from tpch.customer; {noformat} {noformat} impala-shell.sh -i localhost:21001 (second impalad) [localhost:21001] default> describe formatted v1; Query: describe formatted v1 ERROR: NullPointerException: null{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6896) NullPointerException in DESCRIBE FORMATTED on views
[ https://issues.apache.org/jira/browse/IMPALA-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6896. -- Resolution: Fixed > NullPointerException in DESCRIBE FORMATTED on views > --- > > Key: IMPALA-6896 > URL: https://issues.apache.org/jira/browse/IMPALA-6896 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Blocker > Labels: regression > Fix For: Impala 3.0, Impala 2.12.0 > > > {noformat} > impala-shell -i localhost:21000 (first impalad) > [localhost:21000] default> create view v1 as select * from > functional.alltypes; > [localhost:21000] default> alter view v1 as select * from tpch.customer; > {noformat} > {noformat} > impala-shell.sh -i localhost:21001 (second impalad) > [localhost:21001] default> describe formatted v1; > Query: describe formatted v1 > ERROR: NullPointerException: null{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6916) Implement COMMENT ON DATABASE
Fredy Wijaya created IMPALA-6916: Summary: Implement COMMENT ON DATABASE Key: IMPALA-6916 URL: https://issues.apache.org/jira/browse/IMPALA-6916 Project: IMPALA Issue Type: Sub-task Reporter: Fredy Wijaya Assignee: Fredy Wijaya Syntax: {noformat} COMMENT ON DATABASE my_database IS 'Development Database';{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6917) Implement COMMENT ON TABLE
Fredy Wijaya created IMPALA-6917: Summary: Implement COMMENT ON TABLE Key: IMPALA-6917 URL: https://issues.apache.org/jira/browse/IMPALA-6917 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Syntax: {noformat} COMMENT ON TABLE my_db.my_table IS 'Employee Information';{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6918) Implement COMMENT ON COLUMN
Fredy Wijaya created IMPALA-6918: Summary: Implement COMMENT ON COLUMN Key: IMPALA-6918 URL: https://issues.apache.org/jira/browse/IMPALA-6918 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Syntax: {noformat} COMMENT ON COLUMN my_table.my_column IS 'Employee ID Number';{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6999) Upgrade to sqlparse 0.1.19 in Impala Shell
Fredy Wijaya created IMPALA-6999: Summary: Upgrade to sqlparse 0.1.19 in Impala Shell Key: IMPALA-6999 URL: https://issues.apache.org/jira/browse/IMPALA-6999 Project: IMPALA Issue Type: Improvement Components: Clients Reporter: Fredy Wijaya Assignee: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7001) Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Fredy Wijaya created IMPALA-7001: Summary: Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS Key: IMPALA-7001 URL: https://issues.apache.org/jira/browse/IMPALA-7001 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.12.0, Impala 2.11.0, Impala 2.10.0 Reporter: Fredy Wijaya {noformat} > grant create on database functional to role; > show tables in functional; -- this is allowed > show functions in functional; ERROR: AuthorizationException: User 'impdev' does not have privileges to access: functional {noformat} In "show tables", we use ANY privilege whereas we use VIEW_METADATA in "show functions". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7002) Referencing non-existent table/database in CTE without any privilege results in AnalysisException and not AuthorizationException
Fredy Wijaya created IMPALA-7002: Summary: Referencing non-existent table/database in CTE without any privilege results in AnalysisException and not AuthorizationException Key: IMPALA-7002 URL: https://issues.apache.org/jira/browse/IMPALA-7002 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.12.0, Impala 2.11.0, Impala 2.10.0 Reporter: Fredy Wijaya {noformat} A CTE from non-existent database. [localhost:21000] default> with t as (select id from nodb.alltypes) select * from t; Query: with t as (select id from nodb.alltypes) select * from t Query submitted at: 2018-05-09 12:31:29 (Coordinator: http://impdev-dev:25000) ERROR: AnalysisException: Could not resolve table reference: 'nodb.alltypes' A normal select from non-existent database. [localhost:21000] default> select id from nodb.alltypes; Query: select id from nodb.alltypes Query submitted at: 2018-05-09 12:32:58 (Coordinator: http://fwijaya-dev:25000) ERROR: AuthorizationException: User 'impdev' does not have privileges to execute 'SELECT' on: nodb.alltypes {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6999) Upgrade to sqlparse 0.1.19 in Impala Shell
[ https://issues.apache.org/jira/browse/IMPALA-6999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6999. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Upgrade to sqlparse 0.1.19 in Impala Shell > -- > > Key: IMPALA-6999 > URL: https://issues.apache.org/jira/browse/IMPALA-6999 > Project: IMPALA > Issue Type: Improvement > Components: Clients >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 2.13.0, Impala 3.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6337) Infinite loop in impala_shell.py
[ https://issues.apache.org/jira/browse/IMPALA-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6337. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Infinite loop in impala_shell.py > > > Key: IMPALA-6337 > URL: https://issues.apache.org/jira/browse/IMPALA-6337 > Project: IMPALA > Issue Type: Bug > Components: Clients >Affects Versions: Impala 2.10.0 >Reporter: Jinchul Kim >Assignee: Fredy Wijaya >Priority: Major > Labels: newbie > Fix For: Impala 2.13.0, Impala 3.1.0 > > > I faced the infinite loop when I ran an example below. In my initial > investigation using pdb, sanitise_input -> precmd -> onecmd -> postcmd -> > santise_input -> ... > Here is a reproduction scenario: > {noformat} > select '1\'23\'4'"; > ; > " > ; > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7026) buildall.sh make shell fails due to sqlparse problem
[ https://issues.apache.org/jira/browse/IMPALA-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7026. -- Resolution: Not A Bug > buildall.sh make shell fails due to sqlparse problem > > > Key: IMPALA-7026 > URL: https://issues.apache.org/jira/browse/IMPALA-7026 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.1.0 >Reporter: Michael Brown >Assignee: Fredy Wijaya >Priority: Blocker > Labels: broken-build > Fix For: Impala 3.1.0 > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2159/ > {noformat} > 22:43:05 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.14 > 22:43:05 [ 47%] Building CXX object > be/src/udf_samples/CMakeFiles/udf-sample-test.dir/udf-sample-test.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/util/CMakeFiles/Util.dir/hdr-histogram.cc.o > 22:43:05 [ 47%] Built target uda-sample-test > 22:43:05 [ 47%] Building CXX object > be/src/util/CMakeFiles/Util.dir/impalad-metrics.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/exprs/CMakeFiles/Exprs.dir/math-functions-ir.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/testutil/CMakeFiles/TestUtil.dir/desc-tbl-builder.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/testutil/CMakeFiles/TestUtil.dir/fault-injection-util.cc.o > 22:43:05 [ 47%] Linking CXX executable > ../../build/debug/udf_samples/udf-sample-test > 22:43:05 [ 47%] Built target udf-sample-test > 22:43:05 Scanning dependencies of target krpc > 22:43:05 [ 47%] Building CXX object > be/src/exprs/CMakeFiles/Exprs.dir/null-literal.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/util/CMakeFiles/Util.dir/jni-util.cc.o > 22:43:05 [ 48%] Building CXX object > be/src/testutil/CMakeFiles/TestUtil.dir/impalad-query-executor.cc.o > 22:43:05 warning: no files found matching 'docs/Makefile' > 22:43:05 warning: no files found matching 'Makefile' > 22:43:05 [ 48%] Building CXX object > be/src/runtime/CMakeFiles/Runtime.dir/hbase-table.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/kudu/rpc/CMakeFiles/krpc.dir/acceptor_pool.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/testutil/CMakeFiles/TestUtil.dir/in-process-servers.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/kudu/rpc/CMakeFiles/krpc.dir/blocking_ops.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/kudu/rpc/CMakeFiles/krpc.dir/client_negotiation.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/runtime/CMakeFiles/Runtime.dir/hbase-table-factory.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/util/CMakeFiles/Util.dir/logging-support.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/util/CMakeFiles/Util.dir/mem-info.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/util/CMakeFiles/Util.dir/memory-metrics.cc.o > 22:43:05 zip_safe flag not set; analyzing archive contents... > 22:43:05 Cleaning up old build artifacts. > 22:43:06 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.19 > 22:43:06 [ 51%] Building CXX object > be/src/runtime/CMakeFiles/Runtime.dir/hdfs-fs-cache.cc.o > 22:43:06 [ 51%] Building CXX object > be/src/runtime/CMakeFiles/Runtime.dir/initial-reservations.cc.o > 22:43:06 python: can't open file 'setup.py': [Errno 2] No such file or > directory > 22:43:06 Error in shell/make_shell_tarball.sh at line 96: python setup.py -q > bdist_egg clean > 22:43:06 CMakeFiles/shell_tarball.dir/build.make:57: recipe for target > 'CMakeFiles/shell_tarball' failed > 22:43:06 make[2]: *** [CMakeFiles/shell_tarball] Error 2 > 22:43:06 CMakeFiles/Makefile2:141: recipe for target > 'CMakeFiles/shell_tarball.dir/all' failed > 22:43:06 make[1]: *** [CMakeFiles/shell_tarball.dir/all] Error 2 > 22:43:06 make[1]: *** Waiting for unfinished jobs > {noformat} > I just reproduced this locally using {{./buildall.sh -notests > -start_minicluster -start_impala_cluster}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6916) Implement COMMENT ON DATABASE
[ https://issues.apache.org/jira/browse/IMPALA-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6916. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Implement COMMENT ON DATABASE > - > > Key: IMPALA-6916 > URL: https://issues.apache.org/jira/browse/IMPALA-6916 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Syntax: > {noformat} > COMMENT ON DATABASE my_database IS 'Development Database';{noformat} > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7059) DESCRIBE table without ALL or SELECT privilege shows 0 results
Fredy Wijaya created IMPALA-7059: Summary: DESCRIBE table without ALL or SELECT privilege shows 0 results Key: IMPALA-7059 URL: https://issues.apache.org/jira/browse/IMPALA-7059 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.0 Reporter: Fredy Wijaya {noformat} [localhost:21000] default> grant insert on server to role foo_role; [localhost:21000] default> describe functional.alltypes; Query: describe functional.alltypes Fetched 0 row(s) in 0.01s {noformat} For consistency with privileges on other metadata SQL statements, we should register VIEW_METADATA privilege instead of just SELECT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7074) Add OWNER privilege on database, table, and view creation
Fredy Wijaya created IMPALA-7074: Summary: Add OWNER privilege on database, table, and view creation Key: IMPALA-7074 URL: https://issues.apache.org/jira/browse/IMPALA-7074 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7075) Add support for object ownership for Impala
Fredy Wijaya created IMPALA-7075: Summary: Add support for object ownership for Impala Key: IMPALA-7075 URL: https://issues.apache.org/jira/browse/IMPALA-7075 Project: IMPALA Issue Type: Epic Components: Frontend Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6707) Create an upgrade script for Sentry
[ https://issues.apache.org/jira/browse/IMPALA-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6707. -- Resolution: Won't Do > Create an upgrade script for Sentry > --- > > Key: IMPALA-6707 > URL: https://issues.apache.org/jira/browse/IMPALA-6707 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Fredy Wijaya >Priority: Major > Labels: security > > To ensure smooth upgrade for these new fine-grained privileges, we need to > provide an upgrade script. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6648) Add fine-grained privileges to role for Impala
[ https://issues.apache.org/jira/browse/IMPALA-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6648. -- Resolution: Fixed > Add fine-grained privileges to role for Impala > -- > > Key: IMPALA-6648 > URL: https://issues.apache.org/jira/browse/IMPALA-6648 > Project: IMPALA > Issue Type: Epic > Components: Frontend >Reporter: Fredy Wijaya >Priority: Major > Labels: security > Fix For: Impala 3.0 > > > Add CREATE, ALTER, DROP, and REFRESH METADATA fine-grained privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7076) [DOCS] Object ownership for Impala
Fredy Wijaya created IMPALA-7076: Summary: [DOCS] Object ownership for Impala Key: IMPALA-7076 URL: https://issues.apache.org/jira/browse/IMPALA-7076 Project: IMPALA Issue Type: Sub-task Components: Docs Reporter: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-2751) quote in WITH block's comment breaks shell
[ https://issues.apache.org/jira/browse/IMPALA-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-2751. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > quote in WITH block's comment breaks shell > -- > > Key: IMPALA-2751 > URL: https://issues.apache.org/jira/browse/IMPALA-2751 > Project: IMPALA > Issue Type: Bug > Components: Clients >Affects Versions: Impala 2.2 > Environment: CDH5.4.8 >Reporter: Marcell Szabo >Assignee: Fredy Wijaya >Priority: Minor > Labels: impala-shell, shell, usability > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Steps to reproduce: > $ cat > test.sql > with a as ( > select 'a' > -- shouldn't matter > ) > select * from a; > $ impala-shell -f test.sql > /usr/bin/impala-shell: line 32: warning: setlocale: LC_CTYPE: cannot change > locale (UTF-8): No such file or directory > /usr/bin/impala-shell: line 32: warning: setlocale: LC_CTYPE: cannot change > locale (UTF-8): No such file or directory > Starting Impala Shell without Kerberos authentication > Connected to host:21000 > Server version: impalad version 2.2.0-cdh5 RELEASE (build > 1d0b017e2441dd8950924743d839f14b3995e259) > Traceback (most recent call last): > File "/usr/lib/impala-shell/impala_shell.py", line 1006, in > execute_queries_non_interactive_mode(options) > File "/usr/lib/impala-shell/impala_shell.py", line 922, in > execute_queries_non_interactive_mode > if shell.onecmd(query) is CmdStatus.ERROR: > File "/usr/lib64/python2.6/cmd.py", line 219, in onecmd > return func(arg) > File "/usr/lib/impala-shell/impala_shell.py", line 762, in do_with > tokens = list(lexer) > File "/usr/lib64/python2.6/shlex.py", line 269, in next > token = self.get_token() > File "/usr/lib64/python2.6/shlex.py", line 96, in get_token > raw = self.read_token() > File "/usr/lib64/python2.6/shlex.py", line 172, in read_token > raise ValueError, "No closing quotation" > ValueError: No closing quotation > Also, copy-pasting the query interactively, the line never closes. > Strangely, the issue only seems to occur in presence of the WITH block. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7042) Unmatched quotes in comments confuse shell
[ https://issues.apache.org/jira/browse/IMPALA-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7042. -- Resolution: Not A Bug > Unmatched quotes in comments confuse shell > -- > > Key: IMPALA-7042 > URL: https://issues.apache.org/jira/browse/IMPALA-7042 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 2.12.0 >Reporter: Alan Jackoway >Priority: Major > > This jira is really similar to IMPALA-696 and IMPALA-2803 each of which claim > to be fixed, but this is still happening to me on 2.12.0. > Basically the issue is that this command requires you to close the quote in > impala-shell: > {code:sql} > > -- Alan's test query > > select 1 + 1; > > '; > {code} > Then it runs after closing the quote on the third line. > With a double quote, the behavior is even worse: > {code:sql} > > -- Alan"s test query > > select 1 + 1; > > "; > > "; > {code} > I haven't found any way to convince impala the quote is closed and run the > query with a double quote in the comment. Fortunately unmatched double quotes > in comments should be rare. Unmatched single quotes come up in our comments > fairly frequently, which is how I found this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7084) Partition doesn't exist after attempting to ALTER partition location to non-existing path
[ https://issues.apache.org/jira/browse/IMPALA-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7084. -- Resolution: Cannot Reproduce > Partition doesn't exist after attempting to ALTER partition location to > non-existing path > - > > Key: IMPALA-7084 > URL: https://issues.apache.org/jira/browse/IMPALA-7084 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 2.12.0 >Reporter: Gabor Kaszab >Priority: Major > Labels: correctness, ramp-up > > > > {code:java} > create table test (a int) partitioned by (b int); > insert into test partition (b=1) values (1); > // create another partition that points to a different location. > alter table test add partition (b=2) location > 'hdfs://localhost:20500/test-warehouse/test/b=2/'; > // setting the first partition to a non existing location. This fails as > expected. The error message is not exactly user friendly, though. Could have > said "Invalid location or such". > alter table test partition (b=1) set location > 'hdfs://localhost:20500/test-warehouse/test/b=5/'; > Query: alter table test partition (b=1) set location > 'hdfs://localhost:20500/test-warehouse/test/b=5/' > ERROR: TableLoadingException: Failed to load metadata for table: default.test > CAUSED BY: NullPointerException: null > // Setting the first partition to an existing location. This surprisingly > fails as the partitions doesn't exist. > alter table test partition (b=1) set location > 'hdfs://localhost:20500/test-warehouse/test/b=2/'; > Query: alter table test partition (b=1) set location > 'hdfs://localhost:20500/test-warehouse/test/b=1/' > ERROR: PartitionNotFoundException: Partition not found: > TPartitionKeyValue(name:b, value:1) > // However, show partition displays b=1 partition as well. > show partitions test; > Query: show partitions test > +---+---++--+--+---++---++ > | b | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | > Incremental stats | Location | > +---+---++--+--+---++---++ > | 1 | -1 | 1 | 2B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://localhost:20500/test-warehouse/test/b=1 | > | 2 | -1 | 1 | 2B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://localhost:20500/test-warehouse/test/b=2 | > | Total | -1 | 2 | 4B | 0B | | | | | > +---+---++--+--+---++---++ > // Invalidate metadata fixes this issue > invalidate metadata test; > // Now show partitions say that b=1 partition points to the non-existing > location. (This is not a problem as the location would be > // created one data is put into that partition again.) > show partitions test; > Query: show partitions test > +---+---++--+--+---++---+-- > --+ > | b | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | > Incremental stats | Location > | > +---+---++--+--+---++---+-- > --+ > | 1 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://localhost:2050 > 0/test-warehouse/test/b=5 | > | 2 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | > hdfs://localhost:2050 > 0/test-warehouse/test/b=2 | > | Total | -1 | 0 | 0B | 0B | | | | > | > +---+---++--+--+---++---+-- > --+ > // After the invalidate metadata the partition is again visilbe for ALTER > TABLE partition location commands. > alter table test partition (b=1) set location > 'hdfs://localhost:20500/test-warehouse/test/b=2/'; > Query: alter table test partition (b=1) set location > 'hdfs://localhost:20500/test-warehouse/test/b=2/' > ++ > | summary | > ++ > | New location has been set for the specified partition. | > ++ > // {code} > > My expectation here would be that in case the first alter partition location > command fails due to non-existing location then no state change should be > done in the background. In addition seeing an existing partition as a > non-existing one after a failed attempt to change some metada
[jira] [Created] (IMPALA-7106) When log trace is enabled, the rewritten SQL string isn't printed correctly
Fredy Wijaya created IMPALA-7106: Summary: When log trace is enabled, the rewritten SQL string isn't printed correctly Key: IMPALA-7106 URL: https://issues.apache.org/jira/browse/IMPALA-7106 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya The toSql() prints the the SQL that is close to the original SQL string which makes sense for the users. However when debugging, i.e. log level set to TRACE, it is useful to have the rewritten SQL printed out correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7156) Many authorization tests failed with "User 'jenkins' does not have privileges to access"
[ https://issues.apache.org/jira/browse/IMPALA-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7156. -- Resolution: Fixed Fix Version/s: Impala 2.13.0 > Many authorization tests failed with "User 'jenkins' does not have privileges > to access" > > > Key: IMPALA-7156 > URL: https://issues.apache.org/jira/browse/IMPALA-7156 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 2.13.0 >Reporter: Tim Armstrong >Assignee: Fredy Wijaya >Priority: Blocker > Labels: broken-build, flaky > Fix For: Impala 2.13.0 > > Attachments: 0001-Fix-IMPALA-7156-breakage-in-Sentry.patch, > jenkins-priv-logs.tar.gz > > > Failed tests: > {noformat} > > authorization.test_grant_revoke.TestGrantRevoke.test_role_privilege_case[exec_option: > {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, > 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, > 'exec_single_node_rows_threshold': 0} | table_format: text/none] > org.apache.impala.analysis.AuthorizationTest.TestServerLevelInsert[1] > org.apache.impala.analysis.AuthorizationTest.TestFunction[1] > org.apache.impala.analysis.AuthorizationTest.TestDropDatabase[1] > org.apache.impala.analysis.AuthorizationTest.TestServerLevelSelect[1] > org.apache.impala.analysis.AuthorizationTest.TestTruncate[1] > org.apache.impala.analysis.AuthorizationTest.TestShowCreateTable[1] > org.apache.impala.analysis.AuthorizationTest.TestShortUsernameUsed[1] > org.apache.impala.analysis.AuthorizationTest.TestComputeStatsTable[1] > org.apache.impala.analysis.AuthorizationTest.TestHs2GetColumns[1] > org.apache.impala.analysis.AuthorizationTest.TestLoad[1] > org.apache.impala.analysis.AuthorizationTest.TestCreateDatabase[1] > org.apache.impala.analysis.AuthorizationTest.TestInsert[1] > org.apache.impala.analysis.AuthorizationTest.TestSelect[1] > org.apache.impala.analysis.AuthorizationTest.TestCreateView[1] > org.apache.impala.analysis.AuthorizationTest.TestServerLevelAlter[1] > org.apache.impala.analysis.AuthorizationTest.TestResetMetadata[1] > org.apache.impala.analysis.AuthorizationTest.TestExplain[1] > org.apache.impala.analysis.AuthorizationTest.TestCommentOn[1] > org.apache.impala.analysis.AuthorizationTest.TestWithClause[1] > org.apache.impala.analysis.AuthorizationTest.TestAlterView[1] > org.apache.impala.analysis.AuthorizationTest.TestShowPermissions[1] > org.apache.impala.analysis.AuthorizationTest.TestServerLevelDrop[1] > org.apache.impala.analysis.AuthorizationTest.TestDropView[1] > org.apache.impala.analysis.AuthorizationTest.TestShowDbResultsFiltered[1] > > org.apache.impala.analysis.AuthorizationTest.TestShowTableResultsFiltered[1] > org.apache.impala.analysis.AuthorizationTest.TestUnion[1] > org.apache.impala.analysis.AuthorizationTest.TestUseDb[1] > org.apache.impala.analysis.AuthorizationTest.TestDropStats[1] > org.apache.impala.analysis.AuthorizationTest.TestDropTable[1] > org.apache.impala.analysis.AuthorizationTest.TestHs2GetSchema[1] > org.apache.impala.analysis.AuthorizationTest.TestHs2GetTables[1] > org.apache.impala.analysis.AuthorizationTest.TestDescribeDb[1] > org.apache.impala.analysis.AuthorizationTest.TestAlterTable[1] > org.apache.impala.analysis.AuthorizationTest.TestDescribe[1] > org.apache.impala.analysis.AuthorizationTest.TestServerLevelCreate[1] > org.apache.impala.analysis.AuthorizationTest.TestCreateTable[1] > {noformat} > The errors are all along the lines of: > {noformat} > org.apache.impala.catalog.AuthorizationException: User 'jenkins' does not > have privileges to execute 'INSERT' on: functional_parquet.alltypes > at > org.apache.impala.authorization.AuthorizationChecker.checkAccess(AuthorizationChecker.java:146) > at > org.apache.impala.analysis.AnalysisContext.authorizePrivilegeRequest(AnalysisContext.java:567) > at > org.apache.impala.analysis.AnalysisContext.authorize(AnalysisContext.java:530) > at > org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:409) > at > org.apache.impala.common.FrontendTestBase.parseAndAnalyze(FrontendTestBase.java:429) > at > org.apache.impala.analysis.AuthorizationTest.AuthzOk(AuthorizationTest.java:2841) > at > org.apache.impala.analysis.AuthorizationTest.AuthzOk(AuthorizationTest.java:2836) > at > org.apache.impala.analysis.AuthorizationTest.AuthzOk(AuthorizationTest.java:2832) > at > org.apache.impala.analysis.AuthorizationTest.TestTruncate(AuthorizationTest.java:1417) > {noformat} > {noformat
[jira] [Resolved] (IMPALA-7144) Reenable tests disabled by IMPALA-7143
[ https://issues.apache.org/jira/browse/IMPALA-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7144. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Reenable tests disabled by IMPALA-7143 > -- > > Key: IMPALA-7144 > URL: https://issues.apache.org/jira/browse/IMPALA-7144 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.13.0, Impala 3.1.0 >Reporter: Tim Armstrong >Assignee: Fredy Wijaya >Priority: Blocker > Fix For: Impala 2.13.0, Impala 3.1.0 > > > IMPALA-7143 should have unblocked tests but we need to get this test coverage > back. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-2751) quote in WITH block's comment breaks shell
[ https://issues.apache.org/jira/browse/IMPALA-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-2751. -- Resolution: Fixed > quote in WITH block's comment breaks shell > -- > > Key: IMPALA-2751 > URL: https://issues.apache.org/jira/browse/IMPALA-2751 > Project: IMPALA > Issue Type: Bug > Components: Clients >Affects Versions: Impala 2.2 > Environment: CDH5.4.8 >Reporter: Marcell Szabo >Assignee: Fredy Wijaya >Priority: Minor > Labels: impala-shell, shell, usability > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Steps to reproduce: > $ cat > test.sql > with a as ( > select 'a' > -- shouldn't matter > ) > select * from a; > $ impala-shell -f test.sql > /usr/bin/impala-shell: line 32: warning: setlocale: LC_CTYPE: cannot change > locale (UTF-8): No such file or directory > /usr/bin/impala-shell: line 32: warning: setlocale: LC_CTYPE: cannot change > locale (UTF-8): No such file or directory > Starting Impala Shell without Kerberos authentication > Connected to host:21000 > Server version: impalad version 2.2.0-cdh5 RELEASE (build > 1d0b017e2441dd8950924743d839f14b3995e259) > Traceback (most recent call last): > File "/usr/lib/impala-shell/impala_shell.py", line 1006, in > execute_queries_non_interactive_mode(options) > File "/usr/lib/impala-shell/impala_shell.py", line 922, in > execute_queries_non_interactive_mode > if shell.onecmd(query) is CmdStatus.ERROR: > File "/usr/lib64/python2.6/cmd.py", line 219, in onecmd > return func(arg) > File "/usr/lib/impala-shell/impala_shell.py", line 762, in do_with > tokens = list(lexer) > File "/usr/lib64/python2.6/shlex.py", line 269, in next > token = self.get_token() > File "/usr/lib64/python2.6/shlex.py", line 96, in get_token > raw = self.read_token() > File "/usr/lib64/python2.6/shlex.py", line 172, in read_token > raise ValueError, "No closing quotation" > ValueError: No closing quotation > Also, copy-pasting the query interactively, the line never closes. > Strangely, the issue only seems to occur in presence of the WITH block. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6917) Implement COMMENT ON TABLE/VIEW
[ https://issues.apache.org/jira/browse/IMPALA-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6917. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Implement COMMENT ON TABLE/VIEW > --- > > Key: IMPALA-6917 > URL: https://issues.apache.org/jira/browse/IMPALA-6917 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Syntax: > {noformat} > COMMENT ON TABLE my_db.my_table IS 'Employee Information'; > COMMENT ON VIEW my_db.my_view IS 'Employee Information';{noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7016) Statement to allow setting ownership for database
[ https://issues.apache.org/jira/browse/IMPALA-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7016. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Statement to allow setting ownership for database > - > > Key: IMPALA-7016 > URL: https://issues.apache.org/jira/browse/IMPALA-7016 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Affects Versions: Impala 3.0, Impala 2.13.0 >Reporter: Adam Holley >Assignee: Fredy Wijaya >Priority: Major > Labels: security > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Create statement to allow setting owner on database > {noformat} > ALTER DATABASE database_name SET OWNER [USER|ROLE] user_or_role;{noformat} > Examples: > {noformat} > ALTER DATABASE SET OWNER USER > ALTER DATABASE SET OWNER ROLE > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-2195) Improper handling of comments in queries
[ https://issues.apache.org/jira/browse/IMPALA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-2195. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Improper handling of comments in queries > > > Key: IMPALA-2195 > URL: https://issues.apache.org/jira/browse/IMPALA-2195 > Project: IMPALA > Issue Type: Bug > Components: Clients >Affects Versions: Impala 2.2, Impala 2.3.0 >Reporter: bharath v >Assignee: Fredy Wijaya >Priority: Minor > Labels: shell > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Shell doesn't handle comments properly while executing the queries. For > example, > {noformat} > [impalad:21000] > - random comment 1 > -### random comment 2 > insert overwrite test2 select * from test1; > Query: - random comment 1 > -### random comment 2 > insert overwrite test2 select * from test1 > Fetched 0 row(s) in 3.53s > {noformat} > Stderr for an insert query shows up as "Fetched 0 row(s)" instead of > "Inserted 1 row(s)" . This happens because the shell parser doesn't detect > this as an insert query due to the presence of comments. We don't strip the > comments before sending them to Cmd control, due which the default() method > is applied instead of do_insert(). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-5552) Proxy user list should support groups
[ https://issues.apache.org/jira/browse/IMPALA-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-5552. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Proxy user list should support groups > - > > Key: IMPALA-5552 > URL: https://issues.apache.org/jira/browse/IMPALA-5552 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Tristan Stevens >Assignee: Fredy Wijaya >Priority: Critical > Fix For: Impala 2.13.0, Impala 3.1.0 > > > The authorized_proxy_user_config takes a map of user->doAsUser* - i.e. user > is allowed to impersonate any users in the list of doAsUsers. > For enterprise deployments, this would be better specified as a list of > groups, rather than a a list of users: > user1->group* > When accepting a query, Impala will check that the doAs user is a member of > any of the list of groups specified for the connecting user. > HiveServer2 does this via Hadoop-level proxy user privileges (e.g. > {{ > hadoop.proxyuser.user1.hosts > doAsUser1,doAsUser2 > > > hadoop.proxyuser.user1.groups > doAsGroup1,doAsGroup2 > }} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6917) Implement COMMENT ON TABLE/VIEW
[ https://issues.apache.org/jira/browse/IMPALA-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6917. -- Resolution: Fixed > Implement COMMENT ON TABLE/VIEW > --- > > Key: IMPALA-6917 > URL: https://issues.apache.org/jira/browse/IMPALA-6917 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Syntax: > {noformat} > COMMENT ON TABLE my_db.my_table IS 'Employee Information'; > COMMENT ON VIEW my_db.my_view IS 'Employee Information';{noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7180) Pin Impala CDH dependencies
Fredy Wijaya created IMPALA-7180: Summary: Pin Impala CDH dependencies Key: IMPALA-7180 URL: https://issues.apache.org/jira/browse/IMPALA-7180 Project: IMPALA Issue Type: Improvement Components: Infrastructure Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya Pinning Impala CDH dependencies can make the build more stable and reproducible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7187) Skip test_group_impersonation when running inside Docker
Fredy Wijaya created IMPALA-7187: Summary: Skip test_group_impersonation when running inside Docker Key: IMPALA-7187 URL: https://issues.apache.org/jira/browse/IMPALA-7187 Project: IMPALA Issue Type: Bug Components: Infrastructure Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya test_group_impersonation runs fine inside a standard Docker container, but it fails when running with test-with-docker.py. There maybe something a bit special with the way test-with-docker.py runs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7187) Fix test_group_impersonation when running inside Docker
[ https://issues.apache.org/jira/browse/IMPALA-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7187. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Fix test_group_impersonation when running inside Docker > --- > > Key: IMPALA-7187 > URL: https://issues.apache.org/jira/browse/IMPALA-7187 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Labels: broken-build > Fix For: Impala 2.13.0, Impala 3.1.0 > > > test_group_impersonation runs fine inside a standard Docker container, but it > fails when running with test-with-docker.py. There maybe something a bit > special with the way test-with-docker.py runs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7180) Pin Impala CDH dependencies
[ https://issues.apache.org/jira/browse/IMPALA-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7180. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Pin Impala CDH dependencies > --- > > Key: IMPALA-7180 > URL: https://issues.apache.org/jira/browse/IMPALA-7180 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.1.0 > > > Pinning Impala CDH dependencies can make the build more stable and > reproducible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7217) UPDATE statement with SELECT privilege can reveal the existence of a table
Fredy Wijaya created IMPALA-7217: Summary: UPDATE statement with SELECT privilege can reveal the existence of a table Key: IMPALA-7217 URL: https://issues.apache.org/jira/browse/IMPALA-7217 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Fredy Wijaya Having a SELECT privilege when executing UPDATE can reveal the existence of a table, i.e. an AnalysisException is thrown instead of AuthorizationException. {noformat} [localhost:21000] default> grant select on server to role foo_role; +-+ | summary | +-+ | Privilege(s) have been granted. | +-+ Fetched 1 row(s) in 0.02s [localhost:21000] default> update doesntexist set a = 1; ERROR: AnalysisException: Could not resolve table reference: 'doesntexist' {noformat} Let's contrast this with UPSERT. {noformat} [localhost:21000] default> upsert into table doesntexist(id, name) values(1, 'a'); ERROR: AuthorizationException: User 'impdev' does not have privileges to access: default.doesntexist {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6802) Clean up tests in AuthorizationTest
[ https://issues.apache.org/jira/browse/IMPALA-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6802. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Clean up tests in AuthorizationTest > --- > > Key: IMPALA-6802 > URL: https://issues.apache.org/jira/browse/IMPALA-6802 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Labels: security > Fix For: Impala 3.1.0 > > > The tests in AuthorizationTest can be refactored to be more condensed. We > also need to increase the code coverage. It is also a good idea to refactor > these tests to provide flexibility in supporting tests for user-level > privileges in the future: https://issues.apache.org/jira/browse/IMPALA-6794 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7250) AnalysisException in select datediff() with group by
Fredy Wijaya created IMPALA-7250: Summary: AnalysisException in select datediff() with group by Key: IMPALA-7250 URL: https://issues.apache.org/jira/browse/IMPALA-7250 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Fredy Wijaya {noformat} [localhost:21000] default> select datediff(day,now()) from test_table where day>=(now() - interval 5 days) group by datediff(day,now()); Query: select datediff(day,now()) from test_table where day>=(now() - interval 5 days) group by datediff(day,now()) Query submitted at: 2018-06-05 21:11:23 (Coordinator: http://impala-dev:25000) ERROR: AnalysisException: select list expression not produced by aggregation output (missing from GROUP BY clause?): datediff(day, TIMESTAMP '2018-06-05 21:11:23.320564000') [localhost:21000] default> set ENABLE_EXPR_REWRITES=0; ENABLE_EXPR_REWRITES set to 0 [localhost:21000] default> select datediff(day,now()) from test_table where day>=(now() - interval 5 days) group by datediff(day,now()); Query: select datediff(day,now()) from test_table where day>=(now() - interval 5 days) group by datediff(day,now()) Query submitted at: 2018-06-05 21:11:31 (Coordinator: http://impala-dev:25000) Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=b64c8eabcedc58e5:da9edc86 Fetched 0 row(s) in 0.21s {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6918) Implement COMMENT ON COLUMN
[ https://issues.apache.org/jira/browse/IMPALA-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6918. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Implement COMMENT ON COLUMN > --- > > Key: IMPALA-6918 > URL: https://issues.apache.org/jira/browse/IMPALA-6918 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 3.1.0 > > > Syntax: > {noformat} > COMMENT ON COLUMN my_table.my_column IS 'Employee ID Number';{noformat} > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7264) Set owner type upon table creation
Fredy Wijaya created IMPALA-7264: Summary: Set owner type upon table creation Key: IMPALA-7264 URL: https://issues.apache.org/jira/browse/IMPALA-7264 Project: IMPALA Issue Type: Sub-task Components: Frontend Affects Versions: Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya Currently only owner name is set upon table creation. We need to also set owner type upon table creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6988) Statement to allow setting ownership for table/view
[ https://issues.apache.org/jira/browse/IMPALA-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-6988. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Statement to allow setting ownership for table/view > --- > > Key: IMPALA-6988 > URL: https://issues.apache.org/jira/browse/IMPALA-6988 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Affects Versions: Impala 3.0, Impala 2.13.0 >Reporter: Adam Holley >Assignee: Fredy Wijaya >Priority: Major > Labels: security > Fix For: Impala 3.1.0 > > > Create statement to allow setting owner. > {noformat} > ALTER TABLE database_name.table_name SET OWNER [USER|ROLE] user_or_role; > ALTER VIEW database_name.view_name SET OWNER [USER|ROLE] > user_or_role;{noformat} > Examples: > {noformat} > ALTER TABLE SET OWNER USER > ALTER TABLE SET OWNER ROLE > ALTER VIEW SET OWNER USER > ALTER VIEW SET OWNER ROLE > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7254) Inconsistent decimal behavior for the IN predicate
[ https://issues.apache.org/jira/browse/IMPALA-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7254. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Inconsistent decimal behavior for the IN predicate > -- > > Key: IMPALA-7254 > URL: https://issues.apache.org/jira/browse/IMPALA-7254 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.0 >Reporter: Taras Bobrovytsky >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.1.0 > > > The following query returns an error: > {code:java} > select cast(2 as decimal(38,37)) in (cast(3 as decimal(38,1)), cast(2 as > double)){code} > However, this query succeeds and returns true: > {code:java} > select cast(2 as decimal(38,37)) in (cast(2 as double), cast(3 as > decimal(38,1))){code} > The only difference is that the elements in the list are in a different > order. This behavior seems strange. Perhaps we should sort the elements > before analyzing? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7269) Single quote within double-quoted string not matching
[ https://issues.apache.org/jira/browse/IMPALA-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7269. -- Resolution: Cannot Reproduce > Single quote within double-quoted string not matching > - > > Key: IMPALA-7269 > URL: https://issues.apache.org/jira/browse/IMPALA-7269 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 2.10.0 >Reporter: Eric Karnowski >Priority: Major > > I'm using Impala to query data with a string value: > {{O'Reilly}} > When I query this using double quotes to handle the single quote, it accepts > the query but returns no rows. I have to escape the single quote with a > backslash for it to match. > Similarly, > {{SELECT "O'Reilly"}} > returns simply O > > I believe this is the same issue as > https://issues.apache.org/jira/browse/IMPALA-1081 and > https://issues.apache.org/jira/browse/IMPALA-3153 but these were closed as > "cannot reproduce". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7282) Sentry privilege disappears after a catalog refresh
Fredy Wijaya created IMPALA-7282: Summary: Sentry privilege disappears after a catalog refresh Key: IMPALA-7282 URL: https://issues.apache.org/jira/browse/IMPALA-7282 Project: IMPALA Issue Type: Bug Components: Catalog Affects Versions: Impala 3.0 Reporter: Fredy Wijaya {noformat} [localhost:21000] default> grant select on database functional to role foo_role; Query: grant select on database functional to role foo_role +-+ | summary | +-+ | Privilege(s) have been granted. | +-+ Fetched 1 row(s) in 0.05s [localhost:21000] default> grant all on database functional to role foo_role; Query: grant all on database functional to role foo_role +-+ | summary | +-+ | Privilege(s) have been granted. | +-+ Fetched 1 row(s) in 0.03s [localhost:21000] default> show grant role foo_role; Query: show grant role foo_role +--++---++-+---+--+-+ | scope| database | table | column | uri | privilege | grant_option | create_time | +--++---++-+---+--+-+ | database | functional | || | select| false| NULL| | database | functional | || | all | false| NULL| +--++---++-+---+--+-+ Fetched 2 row(s) in 0.02s [localhost:21000] default> show grant role foo_role; Query: show grant role foo_role +--++---++-+---+--+---+ | scope| database | table | column | uri | privilege | grant_option | create_time | +--++---++-+---+--+---+ | database | functional | || | all | false| Wed, Jul 11 2018 15:38:41.113 | +--++---++-+---+--+---+ Fetched 1 row(s) in 0.01s {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7295) Remove IMPALA_MINICLUSTER_PROFILE=2
Fredy Wijaya created IMPALA-7295: Summary: Remove IMPALA_MINICLUSTER_PROFILE=2 Key: IMPALA-7295 URL: https://issues.apache.org/jira/browse/IMPALA-7295 Project: IMPALA Issue Type: Improvement Components: Infrastructure Affects Versions: Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)