waywtdcc opened a new issue #4432: URL: https://github.com/apache/hudi/issues/4432
The parquet file size exceeds the configured value  hudi:0.10 flink:1.13.5 DDL: CREATE TABLE hudi.datagen_hudi2( id bigint , name1 VARCHAR(10), name2 VARCHAR(10), name3 VARCHAR(10), name4 VARCHAR(10), name5 VARCHAR(10), name6 VARCHAR(10), name7 VARCHAR(10), name8 VARCHAR(10), name9 VARCHAR(10), name10 VARCHAR(10), name11 VARCHAR(10), name12 VARCHAR(10), name13 VARCHAR(10), name14 VARCHAR(10), name15 VARCHAR(10), name16 VARCHAR(10), name17 VARCHAR(10), name18 VARCHAR(10), name19 VARCHAR(10), name20 VARCHAR(10), name21 VARCHAR(10), name22 VARCHAR(10), name23 VARCHAR(10), name24 VARCHAR(10), name25 VARCHAR(10), name26 VARCHAR(10), name27 VARCHAR(10), name28 VARCHAR(10), name29 VARCHAR(10), name30 VARCHAR(10), name31 VARCHAR(10), name32 VARCHAR(10), name33 VARCHAR(10), name34 VARCHAR(10), name35 VARCHAR(10), name36 VARCHAR(10), name37 VARCHAR(10), name38 VARCHAR(10), name39 VARCHAR(10), name40 VARCHAR(10), name41 VARCHAR(10), name42 VARCHAR(10), name43 VARCHAR(10), name44 VARCHAR(10), name45 VARCHAR(10), name46 VARCHAR(10), name47 VARCHAR(10), name48 VARCHAR(10), name49 VARCHAR(10), name50 VARCHAR(10), name VARCHAR(20), birthday TIMESTAMP(3), ts TIMESTAMP(3), `partition_str` VARCHAR(20), primary key(id) not enforced --必须指定uuid 主键 ) PARTITIONED BY (`partition_str`) with( 'connector'='hudi', 'path'= 'hdfs://test/user/hive/warehouse/hudi.db/datagen_hudi2' , 'hoodie.datasource.write.recordkey.field'= 'id'-- 主键 , 'write.tasks'= '1' , 'compaction.tasks'= '1' , 'table.type'= 'MERGE_ON_READ'-- 默认COPY_ON_WRITE,可选MERGE_ON_READ , 'compaction.async.enabled'= 'true'-- 是否开启异步压缩 , 'compaction.trigger.strategy'= 'num_or_time' , 'compaction.delta_commits'= '2', -- 默认为5 'compaction.delta_seconds' = '120', 'hive_sync.enable' = 'true', 'hive_sync.mode' = 'hms' , 'hive_sync.metastore.uris' = '*', 'hive_sync.table'='datagen_hudi2', 'hive_sync.db'='hudi' , 'index.global.enabled' = 'true', 'index.bootstrap.enabled' = 'true', 'write.log_block.size' = '61' ); -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
