this optimizer automatic mode or I determination it
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/how-Flink-Optimizer-work-and-what-is-process-do-it-tp1359p1363.html
Sent from the Apache Flink User Mailing List archive. mailing list
very thanks
what meaning Optimizer in flink?
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/how-Flink-Optimizer-work-and-what-is-process-do-it-tp1359p1361.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Thank you very much
Am 26.05.2015 um 11:26 schrieb Robert Metzger:
Hi,
you need to include dependencies not part of the main flink jar into
your usercode jar file.
Therefore, you have to build a fat jar.
The easiest way to do that is like this:
thanks for your message,
maybe you can give me a exsample for the GroupReduceFunction?
2015-05-22 23:29 GMT+02:00 Fabian Hueske fhue...@gmail.com:
There are two ways to do that:
1) You use a GroupReduceFunction, which gives you an iterator over all
points similar to Hadoop's ReduceFunction.
I usually put those connection params inside the hbase-site.xml that will
be included in the generated jar..
On Tue, May 26, 2015 at 2:07 PM, Hilmi Yildirim hilmi.yildi...@neofonie.de
wrote:
I want to add that it is strange that the client wants to establish a
connection to localhost but I
Where do you put the hbase-site.xml? In the resource folder of the
project or on the cluster?
Am 26.05.2015 um 14:12 schrieb Flavio Pompermaier:
I usually put those connection params inside the hbase-site.xml that
will be included in the generated jar..
On Tue, May 26, 2015 at 2:07 PM, Hilmi
I have 10 files..I debugged the code and it seems that there's a loop in
the FileInputFormat when files are nested far away from the root directory
of the scan
On Tue, May 26, 2015 at 2:14 PM, Robert Metzger rmetz...@apache.org wrote:
Hi Flavio,
how many files are in the directory?
You can
Yes, there is a loop to recursively search for files in directory but that
should be ok. The code fails when adding a new InputSplit to an ArrayList.
This is a standard operation.
Oh, I think I found a bug in `addNestedFiles`. It does not pick up the
length of the recursively found files in line
In the src/main/resources folder of the project
On Tue, May 26, 2015 at 2:42 PM, Hilmi Yildirim hilmi.yildi...@neofonie.de
wrote:
Where do you put the hbase-site.xml? In the resource folder of the
project or on the cluster?
Am 26.05.2015 um 14:12 schrieb Flavio Pompermaier:
I usually put
Hi to all,
In my program I'd like to infer from a mysql table the list of directory I
have to output on hdfs (status=0).
Once my job finish to clean each directory and update the status value of
my sql table.
How can I do that in Flink? Is there any callback on the dataset.output()
finish?
Best,
If you have this case, giving more memory is fighting a symptom, rather
than a cause.
If you really have that many duplicates in the data set (and you have not
just a bad implementation of hashCode()), then try the following:
1) Reverse hash join sides. Duplicates hurt only on the build-side,
Hi,
What can I do to give Flink more memory when running it from my IDE? I'm
getting the following exception:
Caused by: java.lang.RuntimeException: Hash join exceeded maximum number
of recursions, without reducing partitions enough to be memory resident.
Probably cause: Too many duplicate
Yeap, that definitively solves the problem! Could you make a PR to fix
that..?
Thank you in advance,
Flavio
On Tue, May 26, 2015 at 3:20 PM, Maximilian Michels m...@apache.org wrote:
Yes, there is a loop to recursively search for files in directory but that
should be ok. The code fails when
Definitely! Much better than using the String value.
2015-05-26 16:55 GMT+02:00 Flavio Pompermaier pomperma...@okkam.it:
Hi to all,
in my program I need to set recursive.file.enumeration to true and I
discovered that there's a constant for that variable in FileInputFormat
Done
2015-05-26 16:59 GMT+02:00 Flavio Pompermaier pomperma...@okkam.it:
Could you do that so I can avoid to make a PR just for that, please?
On Tue, May 26, 2015 at 4:58 PM, Fabian Hueske fhue...@gmail.com wrote:
Definitely! Much better than using the String value.
2015-05-26 16:55
hi community,
my k-means works fine now. thanks a lot for your help.
now i want test something, how is the best way in flink to cout
the iteration?
best regards,
paul
Thanks ;)
On Tue, May 26, 2015 at 5:08 PM, Fabian Hueske fhue...@gmail.com wrote:
Done
2015-05-26 16:59 GMT+02:00 Flavio Pompermaier pomperma...@okkam.it:
Could you do that so I can avoid to make a PR just for that, please?
On Tue, May 26, 2015 at 4:58 PM, Fabian Hueske fhue...@gmail.com
Pushed a fix to the master and will open a PR to programmatically fix this.
On Tue, May 26, 2015 at 4:22 PM, Flavio Pompermaier pomperma...@okkam.it
wrote:
Yeap, that definitively solves the problem! Could you make a PR to fix
that..?
Thank you in advance,
Flavio
On Tue, May 26, 2015 at
18 matches
Mail list logo
