[
https://issues.apache.org/jira/browse/FLINK-18547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tinny cat updated FLINK-18547:
------------------------------
Description:
The options
‘partition.time-extractor.kind’，‘partition.time-extractor.class’，‘partition.time-extractor.timestamp-pattern’
，Configuration does not take effect，beacause the configurations are not
associated with TableBucketAssigner. Resulting in failure to get the correct
BucketId
{code:sql}
CREATE TABLE binlog_send_m (
binlog_tm BIGINT comment '数据库日志(binlog)写入时间',
extract_tm BIGINT comment '数据抽取时间',
db_operation STRING comment '数据库操作: INSERT,UPDATE,DELETE'
) PARTITIONED BY (extract_tm) WITH (
'connector'='filesystem',
'path' = '/path/wherver',
'format' = 'csv',
'partition.time-extractor.kind'='default',
'partition.time-extractor.timestamp-pattern'= '$extract_tm'
)
{code}
The values of bukcketId are as follows:
extract_tm = xxxxxxx
extract_tm = xxxxx
debuging in DefaultPartTimeExtractor, it is not be run.
was:
The options
‘partition.time-extractor.kind’，‘partition.time-extractor.class’，‘partition.time-extractor.timestamp-pattern’
，Configuration does not take effect，beacause the configurations are not
associated with TableBucketAssigner. Resulting in failure to get the correct
BucketId
{code:java}
public static String generatePartitionPath(LinkedHashMap<String, String>
partitionSpec) {
if (partitionSpec.isEmpty()) {
return "";
}
StringBuilder suffixBuf = new StringBuilder();
int i = 0;
for (Map.Entry<String, String> e : partitionSpec.entrySet()) {
if (i > 0) {
suffixBuf.append(Path.SEPARATOR);
}
suffixBuf.append(escapePathName(e.getKey()));
suffixBuf.append('=');
suffixBuf.append(escapePathName(e.getValue()));
i++;
}
suffixBuf.append(Path.SEPARATOR);
return suffixBuf.toString();
}
{code}
> the filesystem sql connector can't work well
> --------------------------------------------
>
> Key: FLINK-18547
> URL: https://issues.apache.org/jira/browse/FLINK-18547
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
> Affects Versions: 1.11.0
> Reporter: tinny cat
> Priority: Critical
>
> The options
> ‘partition.time-extractor.kind’，‘partition.time-extractor.class’，‘partition.time-extractor.timestamp-pattern’
> ，Configuration does not take effect，beacause the configurations are not
> associated with TableBucketAssigner. Resulting in failure to get the correct
> BucketId
> {code:sql}
> CREATE TABLE binlog_send_m (
> binlog_tm BIGINT comment '数据库日志(binlog)写入时间',
> extract_tm BIGINT comment '数据抽取时间',
> db_operation STRING comment '数据库操作: INSERT,UPDATE,DELETE'
> ) PARTITIONED BY (extract_tm) WITH (
> 'connector'='filesystem',
> 'path' = '/path/wherver',
> 'format' = 'csv',
> 'partition.time-extractor.kind'='default',
> 'partition.time-extractor.timestamp-pattern'= '$extract_tm'
> )
> {code}
> The values of bukcketId are as follows:
> extract_tm = xxxxxxx
> extract_tm = xxxxx
> debuging in DefaultPartTimeExtractor, it is not be run.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
