Hi Chan,
if you feel up to implementing such an input format, then you can also
contribute it. You simply have to open a JIRA issue and take ownership of
it.
Cheers,
Till
On Wed, Jul 1, 2015 at 10:08 AM, chan fentes chanfen...@gmail.com wrote:
Thank you all for your help and for pointing out
How about allowing also a varArg of multiple file names for the input
format?
We'd then have the option of
- File or directory
- List of files or directories
- Base directory + regex that matches contained file paths
On Wed, Jul 1, 2015 at 10:13 AM, Flavio Pompermaier pomperma...@okkam.it
Hello,
I really looked in the documentation but unfortunately I could not find the
answer: how do you cancel your data SourceFunction from your “driver” code
(i.e., from a monitoring thread that can initiate a proper shutdown) ? Calling
“cancel()” on the object passed to the addSource() has no
Hi Ronny,
check out this answer on SO:
http://stackoverflow.com/questions/30599616/create-objects-from-input-files-in-apache-flink
It is a similar use case ... I guess you can get the metadata from the
input split as well.
On Wed, Jul 1, 2015 at 11:30 AM, Ronny Bräunlich r.braeunl...@gmail.com
Okay. We filter files starting with underscores because that is the same
behavior as Hadoop.
Hadoop is always creating some underscore files, so when reading results of
a MapReduce job, Flink would read these files.
On Wed, Jul 1, 2015 at 12:15 PM, Ronny Bräunlich r.braeunl...@gmail.com
wrote:
Do you also have the rest of the code. It would be helpful in order to find
out why it's not working.
Cheers,
TIll
On Wed, Jul 1, 2015 at 1:31 PM, Pa Rö paul.roewer1...@googlemail.com
wrote:
now i have implement a time logger in the open and close methods, it is
wrok fine, but i try to
I removed the FAQ from the main repository and merged it with the website's
version.
There is still the duplicate How to Contribute guide. It suffers from the
same sync problem.
On Tue, Jun 30, 2015 at 7:04 PM, Stephan Ewen se...@apache.org wrote:
+1
for moving the FAQ to the website.
On
Hi Robert,
thank you for your quick answer.
Just one additional question:
When I use the ExecutionEnvironment like this: DataSourceString files =
env.readTextFile(file:///Users/me/path/to/file/dir“);
Shouldn’t it read all the files in dir? I have three .json files there but when
I print the
On 01 Jul 2015, at 11:26, Maximilian Michels m...@apache.org wrote:
I removed the FAQ from the main repository and merged it with the website's
version.
There is still the duplicate How to Contribute guide. It suffers from the
same sync problem.
Just remove it as well. Don't need
Hello,
I want to read a file containing textfiles with Flink.
As I already found out I can simply point the environment to the directory and
it will read all the files.
What I couldn’t find out is if it’s possible to keep the file metadata somehow.
Concrete, I need the timestamp, the filename
Hi Robert,
just ignore my previous question.
My files started with underscore and I just found out that FileInputFormat does
filter for underscores in acceptFile().
Cheers,
Ronny
Am 01.07.2015 um 11:35 schrieb Robert Metzger rmetz...@apache.org:
Hi Ronny,
check out this answer on SO:
now i have implement a time logger in the open and close methods, it is
wrok fine, but i try to initial the flink class with a parameter (counter
of benchmark round),
but it will initial always with 0. but i get no exception. what i do wrong?
my benchmark class:
public class
Thank you all for your help and for pointing out different possibilities.
It would be nice to have an input format that takes a directory and a regex
pattern (for file names) to create one data source instead of 1500. This
would have helped me to avoid the problem. Maybe this can be included in
13 matches
Mail list logo
