Hi, this looks like a minor issue in your flink configuration. Can you have a look into your config (in the conf/flink-conf.yaml file) and tell me the value of the "parallelization.degree.default" property? If it is set to 0 or a negative value, thats the source of the issue. Set it to something higher (I recommend numOfNodes*numOfCoresPerNode).
But I have to admit that our error reporting here is not very helpful, we'll improve that... Cheers, Robert On Wed, Sep 24, 2014 at 4:23 PM, nirvanesque [via Apache Flink (Incubator) User Mailing List archive.] <[email protected]> wrote: > Hello robert, > > Apologies for getting back to you on this one :-( > Es tut mir wirklich Leid! > > I've been trying today several times in vain this programme, and following > is the error output : > > [QUOTE] > abasu@edel-4:~$ ./flink/bin/flink run -v > flink/examples/flink-java-examples-0.6-incubating-WebLogAnalysis.jar > file:///home/abasu/examples/Weblogs/documents > file:///home/abasu/examples/Weblogs/ranks > file:///home/abasu/examples/Weblogs/visits > file:///home/abasu/examples/Weblogs/results > Error: The main method caused an error. > org.apache.flink.client.program.ProgramInvocationException: The main > method caused an error. > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:404) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:307) > at org.apache.flink.client.program.Client.run(Client.java:244) > at > org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:332) > at org.apache.flink.client.CliFrontend.run(CliFrontend.java:319) > at > org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:930) > at org.apache.flink.client.CliFrontend.main(CliFrontend.java:954) > Caused by: java.lang.IllegalArgumentException > at > org.apache.flink.compiler.dag.OptimizerNode.setDegreeOfParallelism(OptimizerNode.java:414) > at > org.apache.flink.compiler.PactCompiler$GraphCreatingVisitor.preVisit(PactCompiler.java:772) > at > org.apache.flink.compiler.PactCompiler$GraphCreatingVisitor.preVisit(PactCompiler.java:616) > at > org.apache.flink.api.common.operators.base.GenericDataSinkBase.accept(GenericDataSinkBase.java:287) > at org.apache.flink.api.common.Plan.accept(Plan.java:281) > at > org.apache.flink.compiler.PactCompiler.compile(PactCompiler.java:511) > at > org.apache.flink.compiler.PactCompiler.compile(PactCompiler.java:460) > at > org.apache.flink.client.program.Client.getOptimizedPlan(Client.java:196) > at > org.apache.flink.client.program.Client.getOptimizedPlan(Client.java:209) > at org.apache.flink.client.program.Client.run(Client.java:285) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:54) > at > org.apache.flink.example.java.relational.WebLogAnalysis.main(WebLogAnalysis.java:148) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:389) > ... 6 more > abasu@edel-4:~$ > > [/QUOTE] > > > I'm using Flink version 0.6 on a cluster of 10 nodes. > Job launched from Job manager node. > > As per your instructions, I've used the set for <<documents>> <<ranks>> > <<visits>> <<results>> > that I downloaded from your scratch tree branch. > > It seems that the main function fails because of other reasons (even > before entering the programme logic). > I'm trying to break my head on the exact reason .... > > If you could help me in this direction ... ? > > Thanks a million in advance, > Anirvan > > > Le 23/09/2014 17:22, rmetzger0 [via Apache Flink (Incubator) User Mailing > List archive.] a écrit : > > Hi, > > you have to use the "WebLogDataGenerator" found here: > https://github.com/apache/incubator-flink/blob/master/flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/relational/util/WebLogDataGenerator.java > > It accepts two arguments, the number of documents and visits. > The generated files are located in /tmp/documents /tmp/ranks and > /tmp/visits. > I've generated some sample data for you, located here: > https://github.com/rmetzger/scratch/tree/weblogdataexample/weblog > > > Best, > Robert > > > On Tue, Sep 23, 2014 at 4:05 PM, nirvanesque [via Apache Flink (Incubator) > User Mailing List archive.] <[hidden email] > <http://user/SendEmail.jtp?type=node&node=100&i=0>> wrote: > >> Hello Robert, >> >> Thanks as usual for all your help with the information. >> >> I'm trying in vain to create the different input files from the program >> source code but running into difficulties. >> >> Could you (or anyone else) please post here samples of the 4 inputs that >> are required to run this program ? >> >> Thanks in advance, >> Anirvan >> >> Le 09/09/2014 23:54, rmetzger0 [via Apache Flink (Incubator) User Mailing >> List archive.] a écrit : >> >> Hi Anirvan, >> >> sorry for the late response. You've posted the question to Nabble, which >> is only a mirror of our actual mailing list at [hidden email] >> <http://user/SendEmail.jtp?type=node&node=99&i=0>. Sadly, the message is >> not automatically posted to the apache list because the apache server is >> rejecting the mails from nabble. >> I've already asked and there is no way to change this behavior. >> So I actually saw the two messages you posted here by accident. >> >> Regarding your actual question: >> - The command line arguments for the WebLogAnalysis example are: >> "WebLogAnalysis <documents path> <ranks path> <visits path> <result >> path>" >> >> - Regarding the "info -d" command. I think its an artifact from our old >> java API. I've filed an issue in JIRA: >> https://issues.apache.org/jira/browse/FLINK-1095 Lets see how we resolve >> it. Thanks for reporting this! >> >> You can find the source code of all of our examples in the source release >> of Flink (in the flink-examples/flink-java-examples project. You can also >> access the source (and hence the examples) through GitHub: >> https://github.com/apache/incubator-flink/blob/master/flink-examples/flink-java-examples/src/main/java/org/apache/flink/example/java/relational/WebLogAnalysis.java. >> >> >> To build the examples, you can run: "mvn clean package -DskipTests" in >> the "flink-examples/flink-java-examples" directory. This will re-build >> them. >> >> If you don't want to import the whole Flink project just for playing >> around with the examples, you can also create an empty maven project. This >> script: >> curl >> https://raw.githubusercontent.com/apache/incubator-flink/master/flink-quickstart/quickstart.sh >> | >> bash >> >> will automatically set everything up for you. Just import the >> "quickstart" project into Eclipse or IntelliJ. It will download all >> dependencies and package everything correctly. If you want to use an >> example there, just copy the Java file into the "quickstart" project. >> >> The examples are indeed a very good way to learn how to write Flink jobs. >> >> Please continue asking if you have further questions! >> >> Best, >> Robert >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the >> discussion below: >> >> http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Looking-for-instructions-source-for-flink-java-examples-0-6-incubating-WebLogAnalysis-jar-tp66p67.html >> To unsubscribe from Looking for instructions & source for >> flink-java-examples-0.6-incubating-WebLogAnalysis.jar, click here. >> NAML >> <http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> >> >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the >> discussion below: >> >> http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Looking-for-instructions-source-for-flink-java-examples-0-6-incubating-WebLogAnalysis-jar-tp66p99.html >> To unsubscribe from Looking for instructions & source for >> flink-java-examples-0.6-incubating-WebLogAnalysis.jar, click here. >> NAML >> <http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Looking-for-instructions-source-for-flink-java-examples-0-6-incubating-WebLogAnalysis-jar-tp66p100.html > To unsubscribe from Looking for instructions & source for > flink-java-examples-0.6-incubating-WebLogAnalysis.jar, click here. > NAML > <http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Looking-for-instructions-source-for-flink-java-examples-0-6-incubating-WebLogAnalysis-jar-tp66p102.html > To unsubscribe from Looking for instructions & source for > flink-java-examples-0.6-incubating-WebLogAnalysis.jar, click here > <http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=66&code=cm1ldHpnZXJAYXBhY2hlLm9yZ3w2NnwtMTU5OTExMTY4NA==> > . > NAML > <http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Looking-for-instructions-source-for-flink-java-examples-0-6-incubating-WebLogAnalysis-jar-tp66p103.html Sent from the Apache Flink (Incubator) User Mailing List archive. mailing list archive at Nabble.com.
