你好,

sink.partition-commit.trigger设置成process-time的话是不按照分区值来提取timestamp的,设置成partition-time才可以,但请注意partition-time需要通过watermark来触发。更详细的信息可以参考文档:
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connectors/filesystem.html#sink-partition-commit-trigger

另外如果怀疑是语法问题导致参数没有正确设置的话,也可以在hive里describe一下这张表验证一下。

On Thu, Apr 8, 2021 at 6:07 PM eriendeng <eriend...@tencent.com> wrote:

> SET table.sql-dialect=hive;
> CREATE TABLE hive_catalog.test_db.test_insert_tb (
>     id BIGINT,
>     type INT,
>     user_id BIGINT,
>     title STRING,
>     ts TIMESTAMP
> ) PARTITIONED BY (add_date STRING, add_hour STRING) STORED AS orc
> TBLPROPERTIES (
>     'sink.partition-commit.trigger' = 'process-time',
>     'sink.partition-commit.delay' = '0s',
>     'sink.partition-commit.policy.kind' = 'metastore,success-file',
>     'partition.time-extractor.kind' = 'custom',
>     'partition.time-extractor.timestamp-pattern' = '$add_date $add_hour',
>     'partition.time-extractor.class' = 'xxxx.YMDPartTimeExtractor'
> );
>
>
> 如上在hive-dialect里建了一张表,启动任务后从kafka读然后写入,貌似TBLPROPERTIES里的参数一个都没生效,数据能从hdfs上看到,但是hive没有partition,分区也没有按照我class写的进行(我的class需要把分区弄成
> yyyyMMdd, HH这种日期格式,同时修复+8时区),TM
> JM也没有任何exception,请问有见过这种情况的吗?是不是某个参数没搞对所以都没生效
>
>
>
> --
> Sent from: http://apache-flink.147419.n8.nabble.com/
>


-- 
Best regards!
Rui Li

回复