Really cool tutorial Trevor :-) On Wed, Nov 4, 2015 at 3:26 PM, Robert Metzger <rmetz...@apache.org> wrote:
> For those interested, Trevor wrote a blog post describing how to setup > Spark, Flink and Zeppelin, both locally and on clusters: > http://trevorgrant.org/2015/11/03/apache-casserole-a-delicious-big-data-recipe-for-the-whole-family/ > Thanks Trevor for the great tutorial! > > On Thu, Oct 22, 2015 at 4:23 PM, Till Rohrmann <trohrm...@apache.org> > wrote: > >> Hi Trevor, >> >> that’s actually my bad since I only tested my branch against a remote >> cluster. I fixed the problem (not properly starting the >> LocalFlinkMiniCluster) so that you can now use Zeppelin also in local >> mode. Just check out my branch again. >> >> Cheers, >> Till >> >> >> On Wed, Oct 21, 2015 at 10:00 PM, Trevor Grant <trevor.d.gr...@gmail.com> >> wrote: >> >>> Hey Till, >>> >>> I cloned your branch of Zeplin and while it will compile, it fails tests >>> on timeout, which consequently was the same issue I was having when trying >>> to use Zeppelin. >>> >>> Ideas? >>> >>> >>> ------------------------------------------------------------------------------- >>> Test set: org.apache.zeppelin.flink.FlinkInterpreterTest >>> >>> ------------------------------------------------------------------------------- >>> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 100.347 >>> sec <<< FAILURE! - in org.apache.zeppelin.flink.FlinkInterpreterTest >>> org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed: 100.347 >>> sec <<< ERROR! >>> java.util.concurrent.TimeoutException: Futures timed out after [100000 >>> milliseconds] >>> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) >>> at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) >>> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107) >>> at >>> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) >>> at scala.concurrent.Await$.result(package.scala:107) >>> at >>> org.apache.flink.runtime.minicluster.FlinkMiniCluster.getLeaderIndex(FlinkMiniCluster.scala:171) >>> at >>> org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.getLeaderRPCPort(LocalFlinkMiniCluster.scala:132) >>> at >>> org.apache.zeppelin.flink.FlinkInterpreter.getPort(FlinkInterpreter.java:136) >>> at >>> org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:98) >>> at >>> org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:42) >>> >>> org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed: 100.347 >>> sec <<< ERROR! >>> java.lang.NullPointerException: null >>> at >>> org.apache.zeppelin.flink.FlinkInterpreter.close(FlinkInterpreter.java:221) >>> at >>> org.apache.zeppelin.flink.FlinkInterpreterTest.tearDown(FlinkInterpreterTest.java:48) >>> >>> >>> >>> Trevor Grant >>> Data Scientist >>> https://github.com/rawkintrevo >>> http://stackexchange.com/users/3002022/rawkintrevo >>> >>> *"Fortunate is he, who is able to know the causes of things." -Virgil* >>> >>> >>> On Wed, Oct 21, 2015 at 11:57 AM, Till Rohrmann <trohrm...@apache.org> >>> wrote: >>> >>>> Hi Trevor, >>>> >>>> in order to use Zeppelin with a different Flink version in local mode, >>>> meaning that Zeppelin starts a LocalFlinkMiniCluster when executing >>>> your jobs, you have to build Zeppelin and change the flink.version >>>> property in the zeppelin/flink/pom.xml file to the version you want to >>>> use. >>>> >>>> If you want to let Zeppelin submit jobs to a remote cluster, you should >>>> build Zeppelin with the version of your cluster. That’s because internally >>>> Zeppelin will use this version to construct a JobGraph which is then >>>> submitted to the cluster. In order to configure the remote cluster, you >>>> have to go the *Interpreter* page and scroll down to the *flink* >>>> section. There you have to specify the address of your cluster under >>>> *host* and the port under *port*. This should then be used to submit >>>> jobs to the Flink cluster. >>>> >>>> I hope this answers your question. >>>> >>>> Btw: If you want to use Zeppelin with the latest Flink 0.10-SNAPSHOT >>>> version, you should checkout my branch >>>> https://github.com/tillrohrmann/incubator-zeppelin/tree/flink-0.10-SNAPSHOT >>>> where I’ve made the necessary changes. >>>> >>>> Cheers, >>>> Till >>>> >>>> >>>> On Wed, Oct 21, 2015 at 5:00 PM, Trevor Grant <trevor.d.gr...@gmail.com >>>> > wrote: >>>> >>>>> I'm setting up some Flink/Spark/Zeppelin at work. Spark+Zeppelin >>>>> seems to be relatively well supported and configurable but the Flink is >>>>> not >>>>> so much. >>>>> >>>>> I want Zeppelin to run against my 0.10 build instead of the 0.6 build >>>>> that ships with Zeppelin. My best guess at the moment on how to >>>>> accomplish >>>>> this is to create a symbolic link from the /opt/zepplin/flink folder to >>>>> /opt/flink-0.10, but this feels dirty and wrong. >>>>> >>>>> Does anyone out there have any experience connecting Zeppelin to a >>>>> non-prepackaged Flink build? >>>>> >>>>> I feel like there is a great opporutnity for a HOWTO write up if non >>>>> currently exists. >>>>> >>>>> I'm asking on the Zeppelin user mailing list too as soon as I am >>>>> added. >>>>> >>>>> Thanks for any help >>>>> >>>>> tg >>>>> >>>>> >>>>> Trevor Grant >>>>> Data Scientist >>>>> https://github.com/rawkintrevo >>>>> http://stackexchange.com/users/3002022/rawkintrevo >>>>> >>>>> *"Fortunate is he, who is able to know the causes of things." -Virgil* >>>>> >>>>> >>>> >>> >> >
