wind.wang created FLINK-40999:
---------------------------------
Summary: [cdc-composer] Factory JAR path resolution fails with
"URI is not hierarchical" for relative usrlib paths in application mode
Key: FLINK-40999
URL: https://issues.apache.org/jira/browse/FLINK-40999
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.6.0
Reporter: wind.wang
*Environment*: Apache Flink 1.20.5, Flink CDC 3.6.0 (pipeline connectors: MySQL
-> Paimon), Kubernetes application mode via Flink Kubernetes Operator,
connector JARs placed under {{/opt/flink/usrlib}}.
*Symptom*: submitting a CDC pipeline in application mode fails while the
composer resolves the JAR that contains a connector factory:
{noformat}
java.lang.IllegalArgumentException: URI is not hierarchical
{noformat}
*Root cause*: when connector JARs are picked up from {{usrlib}}, the class
loader reports the factory's {{CodeSource}} location as a *relative* file URL
such as {{file:usrlib/flink-cdc-pipeline-connector-mysql-3.6.0-1.20.jar}}. Such
a URI is opaque (non-hierarchical), so {{Paths.get(url.toURI())}} throws. The
same pattern exists in two places:
* {{org.apache.flink.cdc.common.factories.FactoryDiscoveryUtils}} (JAR path
lookup by factory identifier)
* {{org.apache.flink.cdc.composer.flink.FlinkPipelineComposer}} (collecting
JARs to ship with the job)
Hierarchical URIs (absolute paths, {{local://}} etc.) are unaffected, which is
why the problem only shows up in containerized application-mode deployments
that rely on relative {{usrlib}} class-path entries.
*Workaround we use in production*: a small custom entry JAR referenced with an
absolute {{local:///opt/flink/usrlib/...}} path so that the {{CodeSource}} is
always absolute.
*Proposed fix* (PR [#4542|https://github.com/apache/flink-cdc/pull/4542],
currently titled as a hotfix): extract URL-to-path resolution into one helper
that
# converts hierarchical URIs as before,
# resolves opaque {{file:<relative>}} URIs against the working directory and
returns an absolute path,
# logs a warning for locations that cannot be mapped instead of throwing,
and use it from both {{FactoryDiscoveryUtils}} and {{FlinkPipelineComposer}}.
Behaviour for existing hierarchical URIs is unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
