Ilya Soin created FLINK-39988:
---------------------------------
Summary: Support composite types in filter push-down for savepoint
Table API connector
Key: FLINK-39988
URL: https://issues.apache.org/jira/browse/FLINK-39988
Project: Flink
Issue Type: Improvement
Components: API / State Processor
Affects Versions: 2.4.0
Reporter: Ilya Soin
FLINK-39637 added predicate push-down when querying state using the _savepoint_
connector. However, it only works with primitive types, such as {_}string{_},
{_}long{_}, etc. It can be improved by adding support for composite types:
* POJOs
* Java records
* Flink Tuple
* Flink Row
* Avro records
The general idea is the following:
* user describes key column as _ROW(field1, field2, ..., fieldN)_ in the DDL +
provides {_}value-class{_}, if needed
* _SavepointFilterTranslator_ knows state key type and knows the filtering
which user is applying, e.g. _WHERE k =_ _ROW("Bob", 20)._ It can construct an
object _obj_ of key type, filling its fields with data supplied by user in the
{_}ROW("Bob", 20){_}{_}.{_} Then it can take _obj.hashcode()_ and derive the
exact _InputSplit_ of the key (same approach we do for simple types). We can
then scan only relevant splits.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
