Would it be that difficult to change the behaviour for file:/// and create a single file?or is there a way to do that? On Oct 29, 2014 9:52 PM, "Márton Balassi" <[email protected]> wrote:
> Dear Flavio, > > Yes, the writeAsText() merthod really creates a folder which contains a > file for each execution thread, so your threads do not block each other and > the execution can use multiple cores on your machine. You can see similar > results if you try it with env.execute() from an IDE. > > There are filesystems, HDFS to mention the most prominent one which can > transparently treat such folder structure as a single file and then it > would behave as you expect. I hope this answers your question. > > Best, > > Marton > > On Wed, Oct 29, 2014 at 8:31 PM, Flavio Pompermaier <[email protected]> > wrote: > >> Hi to all, >> running the example at >> http://flink.incubator.apache.org/docs/0.7-incubating/local_execution.html >> I was thinking that the writeAsText on a local file was creating a text >> file on my local filesystem..instead it creates something similar to a >> sequence file (within a folder). >> This is something misleading I think...or the API name is wrong or this >> is a bug (IMHO). >> Btw..how can I modify the following program to write results in a single >> text file on my local filesystem? >> >> public static void main(String[] args) throws Exception { >> ExecutionEnvironment env = ExecutionEnvironment.createLocalEnvironment(); >> DataSet<String> data = env.readTextFile("file:///tmp/res.txt"); >> data.filter(new FilterFunction<String>() { >> public boolean filter(String value) { >> return value.startsWith("http://";); >> } >> }).writeAsText("file:///tmp/res.txt"); >> env.execute();} >> >> Best, >> Flavio >> >> >> >
