Hi Jean,
I think it would be a nice to have feature to display some metrics on the
command line after a job has completed. We already have the run time and
the accumulator results available at the CLI and printing those would be
easy. What metrics in particular are you looking for?
Best,
Max
On
Hi Fabian,
I am trying to compare some examples on Hadoop, Spark and Flink. If
possible I would like to see the job statistics like the report given by
Hadoop. Since I am running these examples on a large cluster it would be
much better if I could obtain such data directly from the console.
@Ufuk, probably should. yes.
On Thu, 18 Jun 2015 at 16:18 Tamara Mendt tammyme...@gmail.com wrote:
Great, thanks!
On Thu, Jun 18, 2015 at 4:16 PM, Ufuk Celebi u...@apache.org wrote:
Should we add this to the Javadoc of the eagerly executed operations?
On 18 Jun 2015, at 16:11, Maximilian
Hi Maximilian,
The metrics am interested in are I/O, run time and communication. Could you
please provide an example of how to obtain such results?
Thank you!!
2015-06-18 10:45 GMT-03:00 Maximilian Michels m...@apache.org:
Hi Jean,
I think it would be a nice to have feature to display some
Hi Tamara!
Yes, there is. Since count/collect/print trigger an execution of the
ExecutionEnvironment, you can get the result afterwards using
env.getLastExecutionResult().
Best,
Max
On Thu, Jun 18, 2015 at 3:57 PM, Tamara Mendt tammyme...@gmail.com wrote:
Hey!
I am currently running a job
Hey!
I am currently running a job in which I wish to use collect to trigger my
job execution, but I also need to have access to the final accumulator
results. Up until now I have been accessing the accumulator results through
the JobExecutionResult that the function execute() returns.
Not
Hello Max,
I will try to do that! Do you know if I could obtain data about the I/O and
communication as well? From what I could understand I can get the runtime
and the accumulator results only. Is that right?
2015-06-18 11:37 GMT-03:00 Maximilian Michels m...@apache.org:
Hi Jean,
As I said,
Hi,
I tried to view directly from the web interface but I could not find any
other information about the completed jobs. I have the list, but when I
open it, no further information is provided. Is this correct?
2015-06-18 15:10 GMT-03:00 Jean Bez jeanluca...@gmail.com:
Hello Max,
I will try
when run progrm on big data customer 2.5GB orders 5GB disply error why
DataSource (at getCustomerDataSet(TPCHQuery3.java:252)
(org.apache.flink.api.java.io.CsvInputFormat)) (1/1) switched to FAILED
org.apache.flink.api.common.io.ParseException: Row too short:
Hi!
There are no I/O or record statistics collected at the moment. It is work
in progress. Also a new Web Frontend that visualizes those is in the works,
so this is going to improve soon, but for now, there is no easy way to grab
those numbers.
If you are interested in contributing, I could pull
Hi,
the CLI cannot show any job statistics. However, you can use the
JobManager web interface that is accessible at port 8081 from a browser.
-Matthias
On 06/17/2015 10:13 PM, Jean Bez wrote:
Hello,
Is it possible to view job statistics after it finished to execute
directly in the command
The reason for this restriction is that KeySelector keys (i.e., keys that
are extracted using a function) require special case handling at runtime.
If we allow combinations of KeySelector keys with other keys for grouping
and groupSorting, we have four different cases to cover compared to two. So
Hi Jean,
what kind of job execution stats are you interested in?
Cheers, Fabian
2015-06-18 9:01 GMT+02:00 Matthias J. Sax mj...@informatik.hu-berlin.de:
Hi,
the CLI cannot show any job statistics. However, you can use the
JobManager web interface that is accessible at port 8081 from a
13 matches
Mail list logo
