Hi guys, sorry for late answer but I am still working to get this done but I don’t understand something
I do have my own writeRecord function, but that function is not able to open new output stream or anything else so I don’t understand how to do that at first I think I should at least partition my data according to the output key (each key to one file) then I need to name the file exactly with that key but I don’t know how to go on thanks michele Il giorno 30/lug/2015, alle ore 12:53, Radu Tudoran <[email protected]<mailto:[email protected]>> ha scritto: Re-hi, I have double –checked and actually there is an OutputFormat interface in flink which can be extended. I believe that for this kind of specific formats as mentioned by Michele, each can develop the appropriate format. On the other hand, having more outputformats I believe is something that could be contributed. We should identify a couple of common formats. The first one that comes in my mind is to have something for writing to memory (e.g. memory buffer) Dr. Radu Tudoran Research Engineer IT R&D Division <image001.png> HUAWEI TECHNOLOGIES Duesseldorf GmbH European Research Center Riesstrasse 25, 80992 München E-mail: [email protected]<mailto:[email protected]> Mobile: +49 15209084330 Telephone: +49 891588344173 HUAWEI TECHNOLOGIES Duesseldorf GmbH Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com<http://www.huawei.com/> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063, Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063, Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN From: Fabian Hueske [mailto:[email protected]] Sent: Thursday, July 30, 2015 11:34 AM To: [email protected]<mailto:[email protected]> Subject: Re: output writer Hi Michele, hi Radu Flink does not have such an OutputFormat, but I agree, it would be a valuable addition. Radu's approach looks like the way to go to implement this feature. @Radu, is there a way to contribute your OutputFormat to Flink? Cheers, Fabian 2015-07-30 10:24 GMT+02:00 Radu Tudoran <[email protected]<mailto:[email protected]>>: Hi, My 2 cents ... based on something similar that I have tried. I have created an own implementation for OutputFormat where you define your own logic for what happens in the "writerecord function". This logic would consist in making a distinction between the ids and write each to the appropriate file Might be that other solutions exist Dr. Radu Tudoran Research Engineer IT R&D Division HUAWEI TECHNOLOGIES Duesseldorf GmbH European Research Center Riesstrasse 25, 80992 München E-mail: [email protected]<mailto:[email protected]> Mobile: +49 15209084330<tel:%2B49%2015209084330> Telephone: +49 891588344173<tel:%2B49%20891588344173> HUAWEI TECHNOLOGIES Duesseldorf GmbH Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com<http://www.huawei.com/> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063, Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063, Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN -----Original Message----- From: Michele Bertoni [mailto:[email protected]<mailto:[email protected]>] Sent: Thursday, July 30, 2015 10:15 AM To: [email protected]<mailto:[email protected]> Subject: output writer Hi everybody, I have a question about the writer I have to save my dataset in different files according to a field of the tuples let’s assume I have a groupId in the tuple, I need to store each group in a different file, with a custom name: any idea on how i can do that? thanks! Michele
