You can write to a single output file by setting parallelism == 1 So final ExecutionEnvironment env = ExecutionEnvironment. createLocalEnvironment().setParallelism(1);
The reason u see multiple output files is because, each worker is writing to a different file. On Fri, Nov 20, 2015 at 10:06 PM, jun aoki <[email protected]> wrote: > Hi Flink community > > I know I'm mistaken but could not find what I want. > > final ExecutionEnvironment env = > ExecutionEnvironment.createLocalEnvironment(); > DataSet<String> data = env.readTextFile("file:///text1.txt"); > FilterFunction<String> filter = new MyFilterFunction(); // looks for a > line starts with "[ERROR]" > DataSet<String> filteredData = data.filter(filter); > filteredData.writeAsText("file:///output1.txt"); > env.execute(); > > Then I expect to get a single file /output1.txt , but actually get > /output1.txt/1, /output1.txt/2, /output1.txt/3... > I assumed I was getting a single file because the method signature says > writeAsText(String filePath). <-- filePath instead of directoryPath > Also the Javadoc comment sounds like I assumed right. > > https://github.com/apache/flink/blob/master/flink-java/src/main/java/org/apache/flink/api/java/DataSet.java#L1354 > > Can anyone tell if the method signature and document should be fixed? or if > I am missing some configuration? > > -- > -jun >
