[
https://issues.apache.org/jira/browse/FLINK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hans-Peter Grahsl updated FLINK-38792:
--------------------------------------
Description:
Flink CDC UDFs cannot properly support configuration parameters as part of the
pipeline definition. Some of the involved classes seem to be prepared for this
(e.g. `UserDefinedFunctionDescriptor` has overloaded constructors in place to
support potential parameters) while others are not aligned and can't make use
of any such configuration options.
The changes to make this work seem to be pretty straight-forward and affect
mostly the following classes which would need minor changes:
* `UdfDef`: add additional attribute of type `Map<String,String>` (see
ModelDef which does this already)
* `YamlPipelineDefinitionParser`: adapt toUdfDef(...) method accordingly
* `TransformTranslator`: adapt udfDefToUDFTuple(...) method accordingly
* write unit and integration tests for an example UDF which takes config
parameters
* add documentation which describes how to make use of config parameters for
UDFs
Since I needed this functionality I've already patched my fork based on the
above suggested code changes. Happy to provide a PR for this any time!
was:
Flink CDC UDFs cannot properly support configuration parameters as part of the
pipeline definition. Some of the involved classes seem to be prepared for this
(e.g. `UserDefinedFunctionDescriptor` has overloaded constructors in place to
support potential parameters) while others are not aligned and can't make use
of any such configuration options.
The changes to make this work seem to be pretty straight-forward and affect
mostly the following classes which would need minor changes:
* `UdfDef`: add additional attribute of type `Map<String,String>` (see
ModelDef which does this already)
* `YamlPipelineDefinitionParser`: adapt toUdfDef(...) method accordingly
* `TransformTranslator`: adapt udfDefToUDFTuple(...) method accordingly
* write unit and integration tests for an example UDF which takes config
parameters
* add documentation which describes how to make use of config parameters for
UDFs
Since I'needed this functionality I've already patched my fork based on the
above suggested code changes. Happy to provide a PR for this any time!
> Properly support configuration parameters for Flink CDC UDFs
> ------------------------------------------------------------
>
> Key: FLINK-38792
> URL: https://issues.apache.org/jira/browse/FLINK-38792
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Affects Versions: cdc-3.5.0
> Reporter: Hans-Peter Grahsl
> Priority: Major
>
> Flink CDC UDFs cannot properly support configuration parameters as part of
> the pipeline definition. Some of the involved classes seem to be prepared for
> this (e.g. `UserDefinedFunctionDescriptor` has overloaded constructors in
> place to support potential parameters) while others are not aligned and can't
> make use of any such configuration options.
> The changes to make this work seem to be pretty straight-forward and affect
> mostly the following classes which would need minor changes:
> * `UdfDef`: add additional attribute of type `Map<String,String>` (see
> ModelDef which does this already)
> * `YamlPipelineDefinitionParser`: adapt toUdfDef(...) method accordingly
> * `TransformTranslator`: adapt udfDefToUDFTuple(...) method accordingly
> * write unit and integration tests for an example UDF which takes config
> parameters
> * add documentation which describes how to make use of config parameters for
> UDFs
>
> Since I needed this functionality I've already patched my fork based on the
> above suggested code changes. Happy to provide a PR for this any time!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
