Hi there,
I faced with issue in adding file to distributed cache in Flink.
My setup:
- Java 1.8
- Flink 1.8
- OS: Windows, Linux
Test scenario:
1. Create simple stream environment
2. Add to distributed cache local file
3. Add simple source function and sink
4. Execute job from CLI (Windows/Linux)
In order to restore job from savepoint or from checkpoint we need to run our
job from Flink CLI. And pipelines that have distributed cache fails their
execution.
Moreover it is different in Linux and Windows systems: in Windows we get
"java.nio.file.InvalidPathException: Illegal char <:> at index 4" and on Linux
we have our Flink freezing (it just stops and do not do anything, no any error
message or stacktrace).
My piece of code for windows environment:
public class CachePipeline {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment see =
StreamExecutionEnvironment.getExecutionEnvironment();
see.registerCachedFile("file:///D:/test.csv", "CacheFile");
see.addSource(new SourceFunction<Object>() {
@Override
public void run(SourceContext<Object> ctx) throws Exception {
ctx.collect(new Object());
}
@Override
public void cancel() {
}
}).print();
see.execute();
}
}
command for running job that I used for:
flink run -c test.CachePipeline D:\path\to\jar\cache-test.jar
Did anybody face with this?
