Timo Walther created FLINK-17158: ------------------------------------ Summary: Watermark strategy property cannot be expressed in YAML Key: FLINK-17158 URL: https://issues.apache.org/jira/browse/FLINK-17158 Project: Flink Issue Type: Bug Components: Table SQL / API, Table SQL / Ecosystem Reporter: Timo Walther
If the watermark strategy needs to be serialized into string-based properties for catalog storage, it is serialized into the following string-based properties: {code} DescriptorProperties properties = new DescriptorProperties(); properties.putTableSchema("schema", schema); Map<String, String> actual = properties.asMap(); Map<String, String> expected = new HashMap<>(); expected.put("schema.0.name", "f0"); expected.put("schema.0.data-type", "BIGINT"); expected.put("schema.1.name", "f1"); expected.put("schema.1.data-type", "ROW<`q1` VARCHAR(2147483647), `q2` TIMESTAMP(9)>"); expected.put("schema.2.name", "f2"); expected.put("schema.2.data-type", "VARCHAR(2147483647)"); expected.put("schema.3.name", "f3"); expected.put("schema.3.data-type", "BIGINT"); expected.put("schema.3.expr", "f0 + 1"); expected.put("schema.4.name", "f4"); expected.put("schema.4.data-type", "DECIMAL(10, 3)"); expected.put("schema.watermark.0.rowtime", "f1.q2"); expected.put("schema.watermark.0.strategy.expr", "`f1`.`q2` - INTERVAL '5' SECOND"); expected.put("schema.watermark.0.strategy.data-type", "TIMESTAMP(3)"); {code} However, this design does not follow a proper hierarchy as both an array and watermarks are stored under a common `schema.` key. Thus, it is impossible to express those properties in JSON or YAML file: {code} tables: - name: PROD_RATES_HISTORY type: source update-mode: append schema: - name: RS_TIMESTAMP data-type: TIMESTAMP(3) - name: RS_SYMBOL data-type: STRING - name: DOUBLE data-type: RS_RATE ???watermark: {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)