[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-17 Thread Qifan Chen (Code Review)
Qifan Chen has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 10:

(13 comments)

Added some comments on testing and the commit message.

http://gerrit.cloudera.org:8080/#/c/19383/10//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19383/10//COMMIT_MSG@30
PS10, Line 30: Kudu assign values for auto-incrementing column automatically
 : when inserting rows so insertion statements don't need to specify
 : values for auto-incrementing column.
Kudu recently enables the auto-incrementing column feature.  The feature works 
by appending a system generated column to the primary key columns to guarantee 
the uniqueness on primary key, when the primary key columns can be non-unique.

This auto column is of type big int and the assignment to it during insertion 
is automatic.


http://gerrit.cloudera.org:8080/#/c/19383/10/fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java:

http://gerrit.cloudera.org:8080/#/c/19383/10/fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java@2365
PS10, Line 2365: partition by hash
may add some more test cases with the following.

1. No partition by clause
2. Non unique primary key includes all selected columns

It seems even when primary key list is empty, the auto incrementing column can 
still  function as the primary key.  Do we allow such a case?


http://gerrit.cloudera.org:8080/#/c/19383/10/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java:

http://gerrit.cloudera.org:8080/#/c/19383/10/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@88
PS10, Line 88: AnalyzesOk("create table tab (x int non unique primary key) 
partition by hash(x) " +
 : "partitions 8 stored as kudu", isExternalPurgeTbl);
 : AnalyzesOk("create table tab (x int, non unique primary 
key(x)) " +
 : "partition by hash(x) partitions 8 stored as kudu", 
isExternalPurgeTbl);
 : AnalyzesOk("create table tab (x int, y int, non unique 
primary key (x, y)) " +
 : "partition by hash(x, y) partitions 8 stored as kudu", 
isExternalPurgeTbl);
 : AnalyzesOk("create table tab (x int, y int, non unique 
primary key (x)) " +
 : "partition by hash(x) partitions 8 stored as kudu", 
isExternalPurgeTbl);
 : AnalyzesOk("create table tab (x int, y int, non unique 
primary key(x, y)) " +
 : "partition by hash(y) partitions 8 stored as kudu", 
isExternalPurgeTbl);
 : AnalyzesOk("create table tab (x timestamp, y timestamp, non 
unique primary key(x)) "+
 : "partition by hash(x) partitions 8 stored as kudu", 
isExternalPurgeTbl);
 : // Promote all partition columns as non unique primary key 
columns if primary keys
 : // are not declared, but partition columns must be the first 
columns in the table.
 : AnalyzesOk("create table tab (x int, y int) partition by 
hash(x) partitions 8 " +
 : "stored as kudu", isExternalPurgeTbl);
 : AnalyzesOk("create table tab (x int, y int) partition by 
hash(x, y) partitions 8 " +
 : "stored as kudu", isExternalPurgeTbl);
 : AnalysisError("create table tab (x int, y int) partition by 
hash(y) partitions 8 " +
 : "stored as kudu", "Specify primary key or non unique 
primary key for the Kudu " +
 : "table, or create partitions with the beginning columns 
of the table.",
 : isExternalPurgeTbl);
In some of the new tests, may try partition by range and no partition to cover 
more cases.  See https://kudu.apache.org/docs/kudu_impala_integration.html.


http://gerrit.cloudera.org:8080/#/c/19383/10/testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
File testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test:

http://gerrit.cloudera.org:8080/#/c/19383/10/testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test@204
PS10, Line 204: order by id
Wonder if we can order DESC by auto_incrementing_id which can be useful to get 
the latest rows.


http://gerrit.cloudera.org:8080/#/c/19383/10/testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
File testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test:

http://gerrit.cloudera.org:8080/#/c/19383/10/testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test@709
PS10, Line 709: auto-incrementing column auto_incrementing_id
nit. The column name appeared twice.


http://gerrit.cloudera.org:

[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-16 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 10:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12176/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 10
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Reviewer: Qifan Chen 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Tue, 17 Jan 2023 06:34:16 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-16 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#10). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine recently adds support for non unique primary key. Kudu
server automatically adds an auto-incrementing column named
'auto_increment_id' in a Kudu table which has non unique primary key.
The non unique primary key and 'auto_increment_id' form the effective
unique composite primary key.

This patch integrated new version of Kudu which support non unique
primary key, added syntactic support for creating table with non unique
primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for auto-incrementing column automatically
when inserting rows so insertion statements don't need to specify
values for auto-incrementing column.
SELECT statement does not show auto-incrementing column unless the
column is explicitly specified in select list.
UPSERT operation is not supported now for Kudu table with auto
incrementing column due to limitation in Kudu engine.

When creating a Kudu table, specifying PRIMARY KEY is optional.
If there is no primary key attribute specified, the partition key
columes will be promoted as non unique primary key if those columns
are the beginning columns of the table.
New column "key_unique" is added to the output of 'describe' table
command for Kudu table.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - Added front end and end to end unit tests.
   Passed query_test/test_kudu.py and custom_cluster/test_kudu.py
   on local environment with new version of Kudu built on local
   machine.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_hms_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/custom_cluster/test_kudu.py
M tests/query_test/test_kudu.py
4

[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-16 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 9:

(16 comments)

Thanks Qifan for your review.

http://gerrit.cloudera.org:8080/#/c/19383/9//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19383/9//COMMIT_MSG@9
PS9, Line 9: Kudu engine adds support for non unique primary key. It adds
> nit recently
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/common/thrift/CatalogObjects.thrift
File common/thrift/CatalogObjects.thrift:

http://gerrit.cloudera.org:8080/#/c/19383/9/common/thrift/CatalogObjects.thrift@580
PS9, Line 580: add
> nit adds
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/common/thrift/CatalogObjects.thrift@581
PS9, Line 581: .
> nit. ", in this case, this field is set to false"
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
File fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java@99
PS9, Line 99: c.isPrimaryKeyUnique() ? "" : "non unique ", c.toString()
> This statement can be integrated into ColumnDef.toString().
use KuduUtil.getPrimaryKeyString()


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
File fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java@227
PS9, Line 227: createStmt_.isPrimaryKeyUnique(), 
createStmt_.getPrimaryKeyColumnDefs(),
> nit. It may be more logic to switch the two arguments, as the new argument
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/TableDef.java
File fe/src/main/java/org/apache/impala/analysis/TableDef.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/TableDef.java@87
PS9, Line 87: An auto-incrementing column will be added
:   // automatically by Kudu engine as key column if primary key is 
not unique.
> nit. If not, an auto-incrementing column will be added
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/TableDef.java@537
PS9, Line 537: only the columns, on which the
 :   // partitions are being created,
> nit. the partition columns have to be first in the primary key columns.
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/TableDef.java@566
PS9, Line 566:   throw new AnalysisException(String.format("Multiple 
%sprimary keys specified. " +
 :   "Composite %sprimary keys can be specified using the " 
+
 :   "%sPRIMARY KEY (col1, col2, ...) syntax at the end of 
the column definition.",
 :   isPrimaryKeyUnique() ? "" : "non unique ",
 :   isPrimaryKeyUnique() ? "" : "non unique ",
 :   isPrimaryKeyUnique() ? "" : "NON UNIQUE "));
> nit. This looks similar to the lines starting at 520.
use KuduUtil.getPrimaryKeyString()


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/Db.java
File fe/src/main/java/org/apache/impala/catalog/Db.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/Db.java@242
PS9, Line 242: boolean isPrimaryKeyUnique, List primaryKeyColumnDefs,
> As noted in another comment, these two arguments probably should be swapped
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
File fe/src/main/java/org/apache/impala/catalog/KuduColumn.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/KuduColumn.java@134
PS9, Line 134: position
> nit. 'position'.
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/KuduColumn.java@135
PS9, Line 135: if
> nit. when
Done


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/KuduColumn.java@144
PS9, Line 144: false
> One may call this function even when column.isIs_primary_key_unique() is tr
This is a static function and it is only called when the primary key of the 
table is not unique.


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java@133
PS9, Line 133:   if (isKey && !isKeyUnique) {
 :   csb.nonUniqueKey(true);
 : } else {

[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-15 Thread Qifan Chen (Code Review)
Qifan Chen has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 9:

(16 comments)

Looks good!

Will review the test part of the patch next.

http://gerrit.cloudera.org:8080/#/c/19383/9//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19383/9//COMMIT_MSG@9
PS9, Line 9: Kudu engine adds support for non unique primary key. It adds
nit recently


http://gerrit.cloudera.org:8080/#/c/19383/9/common/thrift/CatalogObjects.thrift
File common/thrift/CatalogObjects.thrift:

http://gerrit.cloudera.org:8080/#/c/19383/9/common/thrift/CatalogObjects.thrift@580
PS9, Line 580: add
nit adds


http://gerrit.cloudera.org:8080/#/c/19383/9/common/thrift/CatalogObjects.thrift@581
PS9, Line 581: .
nit. ", in this case, this field is set to false"


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
File fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java@99
PS9, Line 99: c.isPrimaryKeyUnique() ? "" : "non unique ", c.toString()
This statement can be integrated into ColumnDef.toString().


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
File fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java@227
PS9, Line 227: createStmt_.isPrimaryKeyUnique(), 
createStmt_.getPrimaryKeyColumnDefs(),
nit. It may be more logic to switch the two arguments, as the new argument 
further refines the primary key columns.


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/TableDef.java
File fe/src/main/java/org/apache/impala/analysis/TableDef.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/TableDef.java@87
PS9, Line 87: An auto-incrementing column will be added
:   // automatically by Kudu engine as key column if primary key is 
not unique.
nit. If not, an auto-incrementing column will be added
automatically by Kudu engine. This extra key column helps produce a unique 
composite primary key (primary keys + auto-incrementing construct).


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/TableDef.java@537
PS9, Line 537: only the columns, on which the
 :   // partitions are being created,
nit. the partition columns have to be first in the primary key columns.


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/analysis/TableDef.java@566
PS9, Line 566:   throw new AnalysisException(String.format("Multiple 
%sprimary keys specified. " +
 :   "Composite %sprimary keys can be specified using the " 
+
 :   "%sPRIMARY KEY (col1, col2, ...) syntax at the end of 
the column definition.",
 :   isPrimaryKeyUnique() ? "" : "non unique ",
 :   isPrimaryKeyUnique() ? "" : "non unique ",
 :   isPrimaryKeyUnique() ? "" : "NON UNIQUE "));
nit. This looks similar to the lines starting at 520.


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/Db.java
File fe/src/main/java/org/apache/impala/catalog/Db.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/Db.java@242
PS9, Line 242: boolean isPrimaryKeyUnique, List primaryKeyColumnDefs,
As noted in another comment, these two arguments probably should be swapped.


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
File fe/src/main/java/org/apache/impala/catalog/KuduColumn.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/KuduColumn.java@134
PS9, Line 134: position
nit. 'position'.


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/KuduColumn.java@135
PS9, Line 135: if
nit. when


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/catalog/KuduColumn.java@144
PS9, Line 144: false
One may call this function even when column.isIs_primary_key_unique() is true.  
Suggest to replace false with column.isIs_primary_key_unique().


http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java:

http://gerrit.cloudera.org:8080/#/c/19383/9/fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java@133
PS9, Line 133:   if (isKey && !isKeyUnique) {
 :   csb.nonUniqueKey(t

[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 9:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12145/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 9
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Wed, 11 Jan 2023 07:22:26 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-10 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#9). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
SELECT statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.
UPSERT operation is not supported now for Kudu table with auto
incrementing column due to limitation in Kudu engine.
When creating a Kudu table, specifying PRIMARY KEY is optional.
If there is no primary key attribute specified, the partition key
columes will be promoted as non unique primary key if those columns
are the beginning columns of the table.
New column "key_unique" is added to the output of 'describe' table
command for Kudu table.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - Added front end and end to end unit tests.
   Passed query_test/test_kudu.py and custom_cluster/test_kudu.py
   on local environment with new version of Kudu built on local
   machine.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_hms_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/custom_cluster/test_kudu.py
M tests/query_test/test_kudu.py
40 files changed, 1,105 insertions(+), 191 deletions(-)


 

[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-10 Thread Abhishek Chennaka (Code Review)
Abhishek Chennaka has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 8:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java:

http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@107
PS8, Line 107: A primary key is required for a Kudu table
Is this the right Error message for this case?
Additionally, if a user wants to create multiple partition levels, the columns 
on which the partitions are being created have to be specified first, right? In 
that case does the order matter as long as all the columns are the beginning of 
the table?


http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@715
PS8, Line 715: add
Do we need an equivalent test for dropping of the columns?


http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/ParserTest.java
File fe/src/test/java/org/apache/impala/analysis/ParserTest.java:

http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/ParserTest.java@2798
PS8, Line 2798: i INT PRIMARY KEY, NON UNIQUE PRIMARY KEY(i)
This looks a bit confusing for the end user where is declared as both PRIMARY 
KEY and NON UNIQUE PRIMARY KEY. Do we want to allow this?
Additionally what if we do something like below? What would the result be?
i INT NON UNIQUE PRIMARY KEY, PRIMARY KEY(i)


http://gerrit.cloudera.org:8080/#/c/19383/8/testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
File testdata/workloads/functional-query/queries/QueryTest/kudu_create.test:

http://gerrit.cloudera.org:8080/#/c/19383/8/testdata/workloads/functional-query/queries/QueryTest/kudu_create.test@535
PS8, Line 535: A primary key is required for a Kudu table
As pointed out before, maybe a more helpful message might help here?



--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 8
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Tue, 10 Jan 2023 19:25:02 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-09 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 8:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12132/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 8
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Tue, 10 Jan 2023 00:29:07 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-09 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#8). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
SELECT statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.
UPSERT operation is not supported now for Kudu table with auto
incrementing column due to limitation in Kudu engine.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - Added front end and end to end unit tests.
   Passed query_test/test_kudu.py and custom_cluster/test_kudu.py
   on local environment with new version of Kudu built on local
   machine.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_hms_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/custom_cluster/test_kudu.py
M tests/query_test/test_kudu.py
40 files changed, 1,101 insertions(+), 190 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/8
--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f32

[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-09 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 7:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12130/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 7
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Tue, 10 Jan 2023 00:12:17 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-09 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#7). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
SELECT statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.
UPSERT operation is not supported now for Kudu table with auto
incrementing column due to limitation in Kudu engine.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - Added front end and end to end unit tests.
   Passed query_test/test_kudu.py and custom_cluster/test_kudu.py
   on local environment with new version of Kudu built on local
   machine.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_hms_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/custom_cluster/test_kudu.py
M tests/query_test/test_kudu.py
40 files changed, 1,100 insertions(+), 189 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/7
--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f32

[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-09 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 7:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19383/7/tests/query_test/test_kudu.py
File tests/query_test/test_kudu.py:

http://gerrit.cloudera.org:8080/#/c/19383/7/tests/query_test/test_kudu.py@667
PS7, Line 667: \
flake8: E502 the backslash is redundant between brackets



--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 7
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Tue, 10 Jan 2023 00:01:44 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 6:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12125/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 6
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Sat, 07 Jan 2023 02:20:11 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-06 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#6). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
SELECT statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.
UPSERT operation is not supported now for Kudu table with auto
incrementing column due to limitation in Kudu engine.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
33 files changed, 747 insertions(+), 103 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/6
--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 6
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-05 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 5:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12116/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Fri, 06 Jan 2023 01:41:34 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2023-01-05 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
SELECT statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.
UPSERT operation is not supported now for Kudu table with auto
incrementing column due to limitation in Kudu engine.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
31 files changed, 735 insertions(+), 101 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/5
--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-21 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 4:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12057/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Wed, 21 Dec 2022 22:12:28 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-21 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#4). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
SELECT statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.
UPSERT operation is not supported now for Kudu table with auto
incrementing column due to limitation in Kudu engine.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
33 files changed, 733 insertions(+), 97 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/4
--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-21 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 3:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12055/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Wed, 21 Dec 2022 16:50:23 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-21 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#3). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
Select statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
33 files changed, 733 insertions(+), 97 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/3
--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-20 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 2:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12050/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Tue, 20 Dec 2022 22:32:11 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-20 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
Select statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.

Testing:
 - Integrated new version of Kudu built on local machine, ran
   manual test in impala-shell with queries to create tables
   with non unique primary key, and tested insert/update/delete
   operations for Kudu tables with non unique primary key.
 - TODO build toolchian with new version of Kudu, including
   the commits for KUDU-1945. Run core test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M be/src/exec/kudu/kudu-util.cc
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
34 files changed, 735 insertions(+), 97 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/2
--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-20 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 1:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/12049/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Kurt Deschler 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Tue, 20 Dec 2022 21:52:53 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-20 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/19383


Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..

WIP IMPALA-11809: Support non unique primary key for Kudu

Kudu engine adds support for non unique primary key. It adds
a column 'auto_increment_id' automatically in a table which
has non unique primary key. The non unique primary key and
'auto_increment_id' form unique composite primary key.

This patch integrated new version of Kudu which support non
unique primary key, added syntactic support for creating table
with non unique primary key.
Example:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) partitions 3
  STORED as KUDU;
  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) partitions 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

Kudu assign values for column 'auto_increment_id' automatically
when inserting rows so insertion statements don't need to specify
values for column 'auto_increment_id'.
Select statement does not show 'auto_increment_id' column unless
the column is explicitly specified in select list.

Testing:
 - Integrat new version of Kudu with the commits for KUDU-1945

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
---
M be/src/exec/kudu/kudu-util.cc
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/FeDb.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
34 files changed, 742 insertions(+), 97 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19383/1
--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Marton Greber 


[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu

2022-12-20 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19383 )

Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19383/1/fe/src/main/java/org/apache/impala/analysis/TableDef.java
File fe/src/main/java/org/apache/impala/analysis/TableDef.java:

http://gerrit.cloudera.org:8080/#/c/19383/1/fe/src/main/java/org/apache/impala/analysis/TableDef.java@513
PS1, Line 513:   throw new AnalysisException("Non unique primary key is 
only supported for Kudu.");
line too long (92 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Gerrit-Change-Number: 19383
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Abhishek Chennaka 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Marton Greber 
Gerrit-Comment-Date: Tue, 20 Dec 2022 21:44:15 +
Gerrit-HasComments: Yes