Great tutorial! Thanks a lot ^^ On 4 November 2015 at 17:12, Leonard Wolters <leon...@sagent.io> wrote:
> Indeed very nice! Thanks > On Nov 4, 2015 5:04 PM, "Till Rohrmann" <trohrm...@apache.org> wrote: > >> Really cool tutorial Trevor :-) >> >> On Wed, Nov 4, 2015 at 3:26 PM, Robert Metzger <rmetz...@apache.org> >> wrote: >> >>> For those interested, Trevor wrote a blog post describing how to setup >>> Spark, Flink and Zeppelin, both locally and on clusters: >>> http://trevorgrant.org/2015/11/03/apache-casserole-a-delicious-big-data-recipe-for-the-whole-family/ >>> Thanks Trevor for the great tutorial! >>> >>> On Thu, Oct 22, 2015 at 4:23 PM, Till Rohrmann <trohrm...@apache.org> >>> wrote: >>> >>>> Hi Trevor, >>>> >>>> that’s actually my bad since I only tested my branch against a remote >>>> cluster. I fixed the problem (not properly starting the >>>> LocalFlinkMiniCluster) so that you can now use Zeppelin also in local >>>> mode. Just check out my branch again. >>>> >>>> Cheers, >>>> Till >>>> >>>> >>>> On Wed, Oct 21, 2015 at 10:00 PM, Trevor Grant < >>>> trevor.d.gr...@gmail.com> wrote: >>>> >>>>> Hey Till, >>>>> >>>>> I cloned your branch of Zeplin and while it will compile, it fails >>>>> tests on timeout, which consequently was the same issue I was having when >>>>> trying to use Zeppelin. >>>>> >>>>> Ideas? >>>>> >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Test set: org.apache.zeppelin.flink.FlinkInterpreterTest >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: >>>>> 100.347 sec <<< FAILURE! - in >>>>> org.apache.zeppelin.flink.FlinkInterpreterTest >>>>> org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed: 100.347 >>>>> sec <<< ERROR! >>>>> java.util.concurrent.TimeoutException: Futures timed out after [100000 >>>>> milliseconds] >>>>> at >>>>> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) >>>>> at >>>>> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) >>>>> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107) >>>>> at >>>>> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) >>>>> at scala.concurrent.Await$.result(package.scala:107) >>>>> at >>>>> org.apache.flink.runtime.minicluster.FlinkMiniCluster.getLeaderIndex(FlinkMiniCluster.scala:171) >>>>> at >>>>> org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.getLeaderRPCPort(LocalFlinkMiniCluster.scala:132) >>>>> at >>>>> org.apache.zeppelin.flink.FlinkInterpreter.getPort(FlinkInterpreter.java:136) >>>>> at >>>>> org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:98) >>>>> at >>>>> org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:42) >>>>> >>>>> org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed: 100.347 >>>>> sec <<< ERROR! >>>>> java.lang.NullPointerException: null >>>>> at >>>>> org.apache.zeppelin.flink.FlinkInterpreter.close(FlinkInterpreter.java:221) >>>>> at >>>>> org.apache.zeppelin.flink.FlinkInterpreterTest.tearDown(FlinkInterpreterTest.java:48) >>>>> >>>>> >>>>> >>>>> Trevor Grant >>>>> Data Scientist >>>>> https://github.com/rawkintrevo >>>>> http://stackexchange.com/users/3002022/rawkintrevo >>>>> >>>>> *"Fortunate is he, who is able to know the causes of things." -Virgil* >>>>> >>>>> >>>>> On Wed, Oct 21, 2015 at 11:57 AM, Till Rohrmann <trohrm...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi Trevor, >>>>>> >>>>>> in order to use Zeppelin with a different Flink version in local >>>>>> mode, meaning that Zeppelin starts a LocalFlinkMiniCluster when >>>>>> executing your jobs, you have to build Zeppelin and change the >>>>>> flink.version property in the zeppelin/flink/pom.xml file to the >>>>>> version you want to use. >>>>>> >>>>>> If you want to let Zeppelin submit jobs to a remote cluster, you >>>>>> should build Zeppelin with the version of your cluster. That’s because >>>>>> internally Zeppelin will use this version to construct a JobGraph >>>>>> which is then submitted to the cluster. In order to configure the remote >>>>>> cluster, you have to go the *Interpreter* page and scroll down to >>>>>> the *flink* section. There you have to specify the address of your >>>>>> cluster under *host* and the port under *port*. This should then be >>>>>> used to submit jobs to the Flink cluster. >>>>>> >>>>>> I hope this answers your question. >>>>>> >>>>>> Btw: If you want to use Zeppelin with the latest Flink 0.10-SNAPSHOT >>>>>> version, you should checkout my branch >>>>>> https://github.com/tillrohrmann/incubator-zeppelin/tree/flink-0.10-SNAPSHOT >>>>>> where I’ve made the necessary changes. >>>>>> >>>>>> Cheers, >>>>>> Till >>>>>> >>>>>> >>>>>> On Wed, Oct 21, 2015 at 5:00 PM, Trevor Grant < >>>>>> trevor.d.gr...@gmail.com> wrote: >>>>>> >>>>>>> I'm setting up some Flink/Spark/Zeppelin at work. Spark+Zeppelin >>>>>>> seems to be relatively well supported and configurable but the Flink is >>>>>>> not >>>>>>> so much. >>>>>>> >>>>>>> I want Zeppelin to run against my 0.10 build instead of the 0.6 >>>>>>> build that ships with Zeppelin. My best guess at the moment on how to >>>>>>> accomplish this is to create a symbolic link from the /opt/zepplin/flink >>>>>>> folder to /opt/flink-0.10, but this feels dirty and wrong. >>>>>>> >>>>>>> Does anyone out there have any experience connecting Zeppelin to a >>>>>>> non-prepackaged Flink build? >>>>>>> >>>>>>> I feel like there is a great opporutnity for a HOWTO write up if non >>>>>>> currently exists. >>>>>>> >>>>>>> I'm asking on the Zeppelin user mailing list too as soon as I am >>>>>>> added. >>>>>>> >>>>>>> Thanks for any help >>>>>>> >>>>>>> tg >>>>>>> >>>>>>> >>>>>>> Trevor Grant >>>>>>> Data Scientist >>>>>>> https://github.com/rawkintrevo >>>>>>> http://stackexchange.com/users/3002022/rawkintrevo >>>>>>> >>>>>>> *"Fortunate is he, who is able to know the causes of things." >>>>>>> -Virgil* >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>
