kamisama0002 opened a new issue, #14196:
URL: https://github.com/apache/gravitino/issues/14196
### Version
main branch (also present in v1.3.1)
### Describe what's wrong
Creating a Paimon table through the Spark connector fails when
`TBLPROPERTIES` contains `primary-key` and `bucket`.
The connector leaves these options in the property map and calls the
create-table overload without indexes. Gravitino rejects the reserved
properties before the request reaches Paimon. The same table definition can be
created through the native Paimon catalog.
### Error message and/or stacktrace
```text
Properties or property prefixes are reserved and cannot be set:
[primary-key, bucket]
```
### How to reproduce
With a configured Paimon filesystem catalog and an existing test schema,
execute:
```sql
CREATE TABLE paimon.default.pk_bucket_probe (
source_namespace STRING NOT NULL,
source_topic STRING NOT NULL,
source_partition INT NOT NULL,
source_offset BIGINT NOT NULL,
payload STRING,
dt STRING,
hr STRING
)
USING paimon
PARTITIONED BY (dt, hr)
TBLPROPERTIES (
'primary-key' =
'source_namespace,source_topic,source_partition,source_offset',
'bucket' = '-1',
'merge-engine' = 'deduplicate'
);
```
The table should have one composite primary key in the declared order,
`bucket=-1` without `bucket-key`, and identity partitions on `dt` and `hr`.
### Additional context
Related to #10367, which covers `CLUSTERED BY`. #10721 fixed bucket
conversion for Flink and added server-side support. The Spark connector still
needs to pass indexes and distribution metadata when creating the table.
Reserved-property validation should remain enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
