bigdata-spec commented on issue #8403:
URL: https://github.com/apache/hudi/issues/8403#issuecomment-1500082955
But if I create a partition hudi table ,delete from table where xxx is ok,
So, does mor table doen't support delete no partition table?
```
spark-sql> create table if not exists zone_hudi.hudi_spark_table0725_0111_mor
> (
> id string,
> brand_id int,
> brand_name string,
> vehicle_model_id int,
> vehicle_model_name string,
> etl_update_time string,
> dt string,
> hh string
> ) using hudi
> options (
> type = 'mor',
> primaryKey = 'brand_id,vehicle_model_id',
> preCombineField = 'etl_update_time',
> hoodie.cleaner.commits.retained = '2',
> hoodie.table.name='hudi_spark_table0725_0111_mor'
> )
> partitioned by (dt,hh);
64856 [main] WARN org.apache.hudi.common.config.DFSPropertiesConfiguration
- Cannot find HUDI_CONF_DIR, please set it as the dir of hudi-defaults.conf
64877 [main] WARN org.apache.hudi.common.config.DFSPropertiesConfiguration
- Properties file file:/etc/hudi/conf/hudi-defaults.conf not found. Ignoring to
load props file
67599 [main] WARN org.apache.hadoop.hive.ql.session.SessionState -
METASTORE_FILTER_HOOK will be ignored, since
hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
Time taken: 3.291 seconds
spark-sql> insert into zone_hudi.
> partition (dt,hh)
> select 'id',
> 100,
> 'rand_name',
> 1000,
> 'ehicle_model_name',
> CAST(current_timestamp AS string) as etl_update_time,
> '20220712',
> '09'
> ;
Error in query: Table or view 'partition' not found in database 'zone_hudi'
spark-sql> insert into zone_hudi.hudi_spark_table0725_0111_mor
> partition (dt,hh)
> select 'id',
> 100,
> 'rand_name',
> 1000,
> 'ehicle_model_name',
> CAST(current_timestamp AS string) as etl_update_time,
> '20220712',
> '09'
> ;
90584 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata -
Metadata table was not found at path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor/.hoodie/metadata
109989 [Executor task launch worker for task 0.0 in stage 6.0 (TID 6)] WARN
org.apache.hadoop.metrics2.impl.MetricsConfig - Cannot locate configuration:
tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
00:40 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
00:44 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
00:59 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
156792 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.vectorized.use.checked.expressions does not exist
156792 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.strict.checks.no.partition.filter does not exist
156792 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.internal.ss.authz.settings.applied.marker does not exist
156792 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.stats.jdbc.timeout does not exist
156792 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.strict.checks.orderby.no.limit does not exist
156792 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.vectorized.input.format.excludes does not exist
156793 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.stats.retries.wait does not exist
Time taken: 71.445 seconds
spark-sql> insert into zone_hudi.hudi_spark_table0725_0111_mor partition
(dt,hh)
> select 'id1',
> 100,
> 'rand_name',
> 1010,
> 'ehicle_model_name',
> CAST(current_timestamp AS string) as etl_update_time,
> '20220712',
> '09'
> ;
01:26 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
01:30 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
01:39 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
196295 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.vectorized.use.checked.expressions does not exist
196295 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.strict.checks.no.partition.filter does not exist
196295 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.internal.ss.authz.settings.applied.marker does not exist
196295 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.stats.jdbc.timeout does not exist
196295 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.strict.checks.orderby.no.limit does not exist
196295 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.vectorized.input.format.excludes does not exist
196295 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.stats.retries.wait does not exist
Time taken: 29.951 seconds
spark-sql> select * from zone_hudi.hudi_spark_table0725_0111_mor;
20230407163943734 20230407163943734_0_0
brand_id:100,vehicle_model_id:1000 dt=20220712/hh=09
65d6968a-1831-4d73-8cba-968ea6e2c55f-0_0-60-2432_20230407164100742.parquet
id 100 rand_name 1000 ehicle_model_name 2023-04-07
16:39:43.438 20220712 09
20230407164100742 20230407164100742_0_1
brand_id:100,vehicle_model_id:1010 dt=20220712/hh=09
65d6968a-1831-4d73-8cba-968ea6e2c55f-0_0-60-2432_20230407164100742.parquet
id1 100 rand_name 1010 ehicle_model_name 2023-04-07
16:41:00.702 20220712 09
Time taken: 0.661 seconds, Fetched 2 row(s)
spark-sql> delete from zone_hudi.hudi_spark_table0725_0111_mor where id='id';
02:35 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
02:39 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
02:50 WARN: Timeline-server-based markers are not supported for HDFS: base
path
hdfs://nameservice1/user/hive/warehouse/zone_hudi.db/hudi_spark_table0725_0111_mor.
Falling back to direct markers.
267474 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.vectorized.use.checked.expressions does not exist
267474 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.strict.checks.no.partition.filter does not exist
267474 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.internal.ss.authz.settings.applied.marker does not exist
267474 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.stats.jdbc.timeout does not exist
267474 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.strict.checks.orderby.no.limit does not exist
267474 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.vectorized.input.format.excludes does not exist
267474 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name
hive.stats.retries.wait does not exist
Time taken: 29.054 seconds
spark-sql> select * from zone_hudi.hudi_spark_table0725_0111_mor;
20230407164100742 20230407164100742_0_1
brand_id:100,vehicle_model_id:1010 dt=20220712/hh=09
65d6968a-1831-4d73-8cba-968ea6e2c55f-0_0-60-2432_20230407164100742.parquet
id1 100 rand_name 1010 ehicle_model_name 2023-04-07
16:41:00.702 20220712 09
Time taken: 1.104 seconds, Fetched 1 row(s)
spark-sql>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
