Arseniy Tashoyan created FLINK-26826:
----------------------------------------
Summary: env.readFile() fails if auto-generated UIDs disabled
Key: FLINK-26826
URL: https://issues.apache.org/jira/browse/FLINK-26826
Project: Flink
Issue Type: Bug
Affects Versions: 1.14.4, 1.12.7, 1.15.0
Environment: Flink 1.12.4
OpenJDK 8
Reporter: Arseniy Tashoyan
When the settings *pipeline.auto-generate-uids* is set to {*}false{*}, the
following functions from *StreamExecutionEnvironment* fail:
* readFile()
* readTextFile()
* createInput()
These functions call the private function {*}createFileInput(){*}, which does
the following:
{code:java}
SingleOutputStreamOperator<OUT> source =
addSource(monitoringFunction, sourceName, null, boundedness)
.transform("Split Reader: " + sourceName, typeInfo, factory);{code}
This code calls {*}addSource(){*}, but does not set UID for the added source
operator. Since the automated UID generation disabled, we get a failure:
{code:java}
java.lang.IllegalStateException: Auto generated UIDs have been disabled but
no UID or hash has been assigned to operator Custom File Source
at
org.apache.flink.streaming.util.graph.StreamGraphUtils.validateTransformationUid(StreamGraphUtils.java:47)
at
org.apache.flink.streaming.api.graph.SimpleTransformationTranslator.configure(SimpleTransformationTranslator.java:108)
at
org.apache.flink.streaming.api.graph.SimpleTransformationTranslator.translateForStreaming(SimpleTransformationTranslator.java:62)
at
org.apache.flink.streaming.api.graph.StreamGraphGenerator.translate(StreamGraphGenerator.java:671)
at
org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:403)
at
org.apache.flink.streaming.api.graph.StreamGraphGenerator.getParentInputIds(StreamGraphGenerator.java:692)
at
org.apache.flink.streaming.api.graph.StreamGraphGenerator.translate(StreamGraphGenerator.java:652)
at
org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:403)
at
org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:269)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:1958)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:1943)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1874)
at
org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.scala:799)
{code}
A possible fix is to pass an additional argument sourceUid to the function
{*}createFileInput(){*}, as well as to all public functions calling
{*}createFileInput(){*}. Same for Scala API.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
