You can set the DOP of the data sink to 1 [1]. There is also a config parameter whether to create a directory or not in case of DOP=1. If I remember correctly, the default is to NOT create a folder for DOP=1.
[1] http://flink.incubator.apache.org/docs/0.7-incubating/programming_guide.html#parallel-execution Best, Fabian 2014-10-29 22:22 GMT+01:00 Flavio Pompermaier <[email protected]>: > Would it be that difficult to change the behaviour for file:/// and create > a single file?or is there a way to do that? > On Oct 29, 2014 9:52 PM, "Márton Balassi" <[email protected]> > wrote: > >> Dear Flavio, >> >> Yes, the writeAsText() merthod really creates a folder which contains a >> file for each execution thread, so your threads do not block each other and >> the execution can use multiple cores on your machine. You can see similar >> results if you try it with env.execute() from an IDE. >> >> There are filesystems, HDFS to mention the most prominent one which can >> transparently treat such folder structure as a single file and then it >> would behave as you expect. I hope this answers your question. >> >> Best, >> >> Marton >> >> On Wed, Oct 29, 2014 at 8:31 PM, Flavio Pompermaier <[email protected] >> > wrote: >> >>> Hi to all, >>> running the example at >>> http://flink.incubator.apache.org/docs/0.7-incubating/local_execution.html >>> I was thinking that the writeAsText on a local file was creating a text >>> file on my local filesystem..instead it creates something similar to a >>> sequence file (within a folder). >>> This is something misleading I think...or the API name is wrong or this >>> is a bug (IMHO). >>> Btw..how can I modify the following program to write results in a single >>> text file on my local filesystem? >>> >>> public static void main(String[] args) throws Exception { >>> ExecutionEnvironment env = ExecutionEnvironment.createLocalEnvironment(); >>> DataSet<String> data = env.readTextFile("file:///tmp/res.txt"); >>> data.filter(new FilterFunction<String>() { >>> public boolean filter(String value) { >>> return value.startsWith("http://";); >>> } >>> }).writeAsText("file:///tmp/res.txt"); >>> env.execute();} >>> >>> Best, >>> Flavio >>> >>> >>> >>
