你好, sink.partition-commit.trigger设置成process-time的话是不按照分区值来提取timestamp的,设置成partition-time才可以,但请注意partition-time需要通过watermark来触发。更详细的信息可以参考文档: https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connectors/filesystem.html#sink-partition-commit-trigger
另外如果怀疑是语法问题导致参数没有正确设置的话,也可以在hive里describe一下这张表验证一下。 On Thu, Apr 8, 2021 at 6:07 PM eriendeng <eriend...@tencent.com> wrote: > SET table.sql-dialect=hive; > CREATE TABLE hive_catalog.test_db.test_insert_tb ( > id BIGINT, > type INT, > user_id BIGINT, > title STRING, > ts TIMESTAMP > ) PARTITIONED BY (add_date STRING, add_hour STRING) STORED AS orc > TBLPROPERTIES ( > 'sink.partition-commit.trigger' = 'process-time', > 'sink.partition-commit.delay' = '0s', > 'sink.partition-commit.policy.kind' = 'metastore,success-file', > 'partition.time-extractor.kind' = 'custom', > 'partition.time-extractor.timestamp-pattern' = '$add_date $add_hour', > 'partition.time-extractor.class' = 'xxxx.YMDPartTimeExtractor' > ); > > > 如上在hive-dialect里建了一张表,启动任务后从kafka读然后写入,貌似TBLPROPERTIES里的参数一个都没生效,数据能从hdfs上看到,但是hive没有partition,分区也没有按照我class写的进行(我的class需要把分区弄成 > yyyyMMdd, HH这种日期格式,同时修复+8时区),TM > JM也没有任何exception,请问有见过这种情况的吗?是不是某个参数没搞对所以都没生效 > > > > -- > Sent from: http://apache-flink.147419.n8.nabble.com/ > -- Best regards! Rui Li