Yes, I'd also play around with the filters to get some output. Not sure what exactly you want to demonstrate, but processing a few MBs on a 10 node cluster might look a bit strange...
How about switching to another example that works on less specific data? There are a nice graph processing examples and also publicly available graph data set. Cheers, Fabian 2014-11-05 21:14 GMT+01:00 Kostas Tzoumas <[email protected]>: > How about changing the filtering conditions for your demo? > > On Wed, Nov 5, 2014 at 8:14 PM, Anirvan BASU <[email protected]> > wrote: > >> Thanks Fabian for your deep analysis. >> So that explains why none of the worker nodes have any data in the result >> files ? >> >> Now my question is - do you have any datasets that will yield non-zero >> result dataset ? >> I want to use (modify) for a demo at EIT-ICT labs , using Flink. >> >> Thanks in advance, >> Anirvan >> >> ------------------------------ >> >> *From: *"Fabian Hueske" <[email protected]> >> *To: *"Anirvan BASU" <[email protected]>, >> [email protected] >> *Sent: *Wednesday, November 5, 2014 8:11:08 PM >> >> *Subject: *Re: Looking for instructions & source for >> flink-java-examples-0.7.0-incubating-WebLogAnalysis.jar >> >> Hi Anirvan, >> >> just checked the data. >> The data you use and the WebLogAnalysis example program do not work well >> together and do not give you any results. >> All tuples are removed by filters or joins. >> >> Best, Fabian >> >> 2014-11-05 19:42 GMT+01:00 Fabian Hueske <[email protected]>: >> >>> Hi Anirvan, >>> >>> actually, the execution logs look good. It is possible, that the >>> provided data just does not "match" the code of the WebLogAnalysis example >>> program. Maybe some filters are too selective. I will check that and let >>> you know the result. >>> Have you tried to run any other job such as WordCount? >>> >>> To answer your questions: >>> - with ./log/, I refered to the log directory of your Flink setup. This >>> directory contains .out files into which the stdout of the JobManager and >>> the TaskManager processes is redirected. >>> - It is possible to change the files into which the stdout is >>> redirected. However, you have to manually adapt the bash start scripts for >>> that. >>> >>> 2014-11-05 15:26 GMT+01:00 Anirvan BASU <[email protected]>: >>> >>>> Hello Fabien and everyone, >>>> >>>> In my previous post, I missed some of your questions from your last >>>> email. >>>> >>>> Here are my replies: >>>> Have you checked the local file systems on all workers for output? >>>> Yes, I did (in the case of using "file:///address/to/local/file" using >>>> NFS). They were the same empty files. >>>> >>>> Did the job process any data at all? The jobs finishes within 1 second >>>> (which is still possible for very small input data). >>>> The data that was used was provided to me by robert Metzger. Please see >>>> the link here: >>>> https://github.com/rmetzger/scratch/tree/weblogdataexample/weblog >>>> Actually, the first lines in the "rank" file had some problem with the >>>> separators '|' It may be due to difference in coding between Linux machines >>>> ... the programme would end up with some error always. >>>> So I deleted the top few lines and then the programme finished with >>>> code FINISHED but empty files :-(( >>>> >>>> You can change the example program to write its output to the stdout by >>>> replacing the writeAsCSV() by print(). The stdout of all workers is >>>> redirected to the ./log/*.out files. >>>> Question to you: What is the location of this stdtout ./log/* ? I >>>> could not find it anywhere - neither in my local directories nor in the >>>> system root. >>>> Question to you: Is it possible to change the location of the stdout by >>>> changing the conf file flink-conf.yaml ? Which exact parameter should I >>>> change ? >>>> >>>> Thanks in advance for all your help, >>>> Anirvan >>>> >>>> >>>> ------------------------------ >>>> >>>> *From: *"Fabian Hueske" <[email protected]> >>>> *To: *[email protected] >>>> *Sent: *Tuesday, November 4, 2014 4:28:40 PM >>>> *Subject: *Re: Looking for instructions & source for >>>> flink-java-examples-0.7.0-incubating-WebLogAnalysis.jar >>>> >>>> Hi Anirvan, >>>> >>>> you specify input and output as files in the local file system >>>> (file:///). Each worker needs access to the all input files, which means >>>> that each worker needs (a copy of) these files in its local file system. >>>> The common setup to use Flink in a distributed cluster is to use a >>>> distributed data store such as HDFS (or a data store that can be accessed >>>> by each node). >>>> Using a shared file system (like NFS) that is mounted into each worker >>>> would work, but remember, that all nodes will concurrently read and write >>>> to the shared system. >>>> >>>> Have you checked the local file systems on all workers for output? >>>> Did the job process any data at all? The jobs finishes within 1 second >>>> (which is still possible for very small input data). >>>> >>>> You can change the example program to write its output to the stdout by >>>> replacing the writeAsCSV() by print(). The stdout of all workers is >>>> redirected to the ./log/*.out files. >>>> >>>> Best, Fabian >>>> >>>> 2014-11-04 16:08 GMT+01:00 Anirvan BASU <[email protected]>: >>>> >>>>> Hello Robert, Stephan et al, >>>>> >>>>> Hope you are doing fine in Berlin. >>>>> >>>>> I am getting back to you on my previous problem on the WebLogAnalysis >>>>> example, after a long time. >>>>> >>>>> We are currently using Flink 0.7.0 over a 10-node cluster in >>>>> Manager-Worker configuration. >>>>> >>>>> We ran the following command: >>>>> $ ./flink/bin/flink run >>>>> flink/examples/flink-java-examples-0.7.0-incubating-WebLogAnalysis.jar >>>>> file:///home/abasu/examples/Weblogs/documents >>>>> file:///home/abasu/examples/Weblogs/ranks >>>>> file:///home/abasu/examples/Weblogs/visits >>>>> file:///home/abasu/examples/Weblogs/result >>>>> >>>>> For the documents, rank and visits files, we used the data generated >>>>> by you from this link: >>>>> https://github.com/rmetzger/scratch/tree/weblogdataexample/weblog >>>>> >>>>> The program executed with the following output: >>>>> 11/04/2014 14:58:12: Job execution switched to status RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (1/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (1/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (2/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (2/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (3/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (3/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (4/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (4/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (5/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (5/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (6/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (6/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (7/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (7/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (8/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (8/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (9/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (9/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (1/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (1/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (2/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (2/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (3/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (3/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (4/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (4/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (5/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (5/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (6/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (6/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (7/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (7/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (8/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (8/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (9/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (9/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (1/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (1/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (2/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (2/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (3/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (3/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (4/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (4/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (5/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (5/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (6/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (1/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (6/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (7/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (7/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (8/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (8/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (9/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (9/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (2/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (5/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (6/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (7/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (8/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (2/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (9/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (5/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (6/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (8/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (7/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (3/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (9/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (1/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (1/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (2/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (2/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (8/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (2/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (3/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (2/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (3/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (3/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (1/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (1/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (1/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (3/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (1/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (4/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (1/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (4/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (5/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (6/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (4/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (7/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (9/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (3/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (1/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (4/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (4/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (6/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (6/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (5/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (5/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (6/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (5/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (2/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (2/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (4/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (2/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (7/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (7/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (8/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (8/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (7/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (3/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (3/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (8/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (9/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (9/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (3/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (9/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (4/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (4/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (4/9) switched to RUNNING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (5/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (5/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:12: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (5/9) switched to RUNNING >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (6/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (6/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (6/9) switched to RUNNING >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (7/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (7/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (7/9) switched to RUNNING >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (8/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (8/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (8/9) switched to RUNNING >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (9/9) switched to SCHEDULED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (9/9) switched to DEPLOYING >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (9/9) switched to RUNNING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (7/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (7/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (7/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (8/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (8/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (8/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (5/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (5/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (6/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (6/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (5/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (4/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (6/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (4/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (7/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (4/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (3/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (3/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (3/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (2/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (2/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (2/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (1/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (1/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (5/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (4/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (3/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (2/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (8/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (6/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (1/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (1/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (4/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (2/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (5/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (9/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (4/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (8/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (6/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (2/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (5/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (7/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (9/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (6/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (7/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (8/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: >>>>> Join(org.apache.flink.api.java.operators.JoinOperator$ProjectFlatJoinFunction) >>>>> (9/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (9/9) switched to >>>>> SCHEDULED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (9/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (9/9) switched to >>>>> DEPLOYING >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (4/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (2/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (7/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (6/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (5/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (8/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (1/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/documents) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterDocByKeyWords) >>>>> -> Map (Projection [0]) (3/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (1/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (1/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/visits) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterVisitsByDate) >>>>> -> Map (Projection [0]) (3/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CHAIN DataSource (CSV Input (|) >>>>> file:/home/abasu/examples/Weblogs/ranks) -> Filter >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$FilterByRank) >>>>> (3/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (9/9) switched to >>>>> RUNNING >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (7/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (7/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (8/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (5/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (8/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (6/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (5/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (6/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (4/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (4/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (1/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (1/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (3/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (3/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (2/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (2/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: DataSink(CsvOutputFormat (path: >>>>> file:/home/abasu/examples/Weblogs/result, delimiter: |)) (9/9) switched to >>>>> FINISHED >>>>> 11/04/2014 14:58:13: CoGroup >>>>> (org.apache.flink.examples.java.relational.WebLogAnalysis$AntiJoinVisits) >>>>> (9/9) switched to FINISHED >>>>> 11/04/2014 14:58:13: Job execution switched to status FINISHED >>>>> >>>>> The following directory was created: >>>>> /home/abasu/examples/Weblogs/result >>>>> with 9 files (named 1 to 9) >>>>> All these files are empty! >>>>> >>>>> Hence my naive question: Is this the expected output ? Or what should >>>>> be the expected output for an error-free run ? >>>>> >>>>> Please let me know where we are going wrong? >>>>> If possible do you have other data generated to try the WebLogAnalysis >>>>> example ? >>>>> >>>>> Thanks in advance for your advice and help, >>>>> Anirvan >>>>> >>>>> >>>>> >>>>>>> >>>>>>> >>>>>>>>>> Le 23/09/2014 17:22, rmetzger0 [via Apache Flink (Incubator) User >>>>>>>>>> Mailing List archive.] a écrit : >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> you have to use the "WebLogDataGenerator" found here: >>>>>>>>>> https://github.com/apache/incubator-flink/blob/master/flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/relational/util/WebLogDataGenerator.java >>>>>>>>>> >>>>>>>>>> It accepts two arguments, the number of documents and visits. >>>>>>>>>> The generated files are located in /tmp/documents /tmp/ranks and >>>>>>>>>> /tmp/visits. >>>>>>>>>> I've generated some sample data for you, located here: >>>>>>>>>> https://github.com/rmetzger/scratch/tree/weblogdataexample/weblog >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Robert >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Sep 23, 2014 at 4:05 PM, nirvanesque [via Apache Flink >>>>>>>>>> (Incubator) User Mailing List archive.] <[hidden email] >>>>>>>>>> <http://user/SendEmail.jtp?type=node&node=100&i=0>> wrote: >>>>>>>>>> >>>>>>>>>>> Hello Robert, >>>>>>>>>>> >>>>>>>>>>> Thanks as usual for all your help with the information. >>>>>>>>>>> >>>>>>>>>>> I'm trying in vain to create the different input files from the >>>>>>>>>>> program source code but running into difficulties. >>>>>>>>>>> >>>>>>>>>>> Could you (or anyone else) please post here samples of the 4 >>>>>>>>>>> inputs that are required to run this program ? >>>>>>>>>>> >>>>>>>>>>> Thanks in advance, >>>>>>>>>>> Anirvan >>>>>>>>>>> >>>>>>>>>>> Le 09/09/2014 23:54, rmetzger0 [via Apache Flink (Incubator) >>>>>>>>>>> User Mailing List archive.] a écrit : >>>>>>>>>>> >>>>>>>>>>> Hi Anirvan, >>>>>>>>>>> >>>>>>>>>>> sorry for the late response. You've posted the question to >>>>>>>>>>> Nabble, which is only a mirror of our actual mailing list at [hidden >>>>>>>>>>> email] <http://user/SendEmail.jtp?type=node&node=99&i=0>. >>>>>>>>>>> Sadly, the message is not automatically posted to the apache list >>>>>>>>>>> because >>>>>>>>>>> the apache server is rejecting the mails from nabble. >>>>>>>>>>> I've already asked and there is no way to change this behavior. >>>>>>>>>>> So I actually saw the two messages you posted here by accident. >>>>>>>>>>> >>>>>>>>>>> Regarding your actual question: >>>>>>>>>>> - The command line arguments for the WebLogAnalysis example are: >>>>>>>>>>> "WebLogAnalysis <documents path> <ranks path> <visits >>>>>>>>>>> path> <result path>" >>>>>>>>>>> >>>>>>>>>>> - Regarding the "info -d" command. I think its an artifact from >>>>>>>>>>> our old java API. I've filed an issue in JIRA: >>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-1095 Lets see how >>>>>>>>>>> we resolve it. Thanks for reporting this! >>>>>>>>>>> >>>>>>>>>>> You can find the source code of all of our examples in the >>>>>>>>>>> source release of Flink (in the flink-examples/flink-java-examples >>>>>>>>>>> project. >>>>>>>>>>> You can also access the source (and hence the examples) through >>>>>>>>>>> GitHub: >>>>>>>>>>> https://github.com/apache/incubator-flink/blob/master/flink-examples/flink-java-examples/src/main/java/org/apache/flink/example/java/relational/WebLogAnalysis.java. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> To build the examples, you can run: "mvn clean package >>>>>>>>>>> -DskipTests" in the "flink-examples/flink-java-examples" directory. >>>>>>>>>>> This >>>>>>>>>>> will re-build them. >>>>>>>>>>> >>>>>>>>>>> If you don't want to import the whole Flink project just for >>>>>>>>>>> playing around with the examples, you can also create an empty maven >>>>>>>>>>> project. This script: >>>>>>>>>>> curl >>>>>>>>>>> https://raw.githubusercontent.com/apache/incubator-flink/master/flink-quickstart/quickstart.sh >>>>>>>>>>> | >>>>>>>>>>> bash >>>>>>>>>>> >>>>>>>>>>> will automatically set everything up for you. Just import the >>>>>>>>>>> "quickstart" project into Eclipse or IntelliJ. It will download all >>>>>>>>>>> dependencies and package everything correctly. If you want to use an >>>>>>>>>>> example there, just copy the Java file into the "quickstart" >>>>>>>>>>> project. >>>>>>>>>>> >>>>>>>>>>> The examples are indeed a very good way to learn how to write >>>>>>>>>>> Flink jobs. >>>>>>>>>>> >>>>>>>>>>> Please continue asking if you have further questions! >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Robert >>>>>>>>>>> >>>>>>>>>>> >>>>> >>>>> >>>> >>>> >>> >> >> >
