yes. with this example (examples/streaming/WordCount.jar) my cluster worked.
the file log/*out from the master is still empty and the file log/*out from the slave node has my result. The dashboard also shows that the job is completed. So, like you said there are some external dependencies that I didn`t include in my deploy. Do you have any clue? I am following the original quickstart ( https://ci.apache.org/projects/flink/flink-docs-master/quickstart/setup_quickstart.html ) Kind Regards, Felipe *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Mon, Aug 6, 2018 at 5:01 PM Gary Yao <g...@data-artisans.com> wrote: > Hi, > > nc exits after the first connection is closed. Are you re-running the nc > command every time the job finishes? > > The stacktrace you copied does not indicate that a TaskManager cannot > connect > to the JobManager. I can only see that the SocketTextStreamFunction (from > the > SocketWindowWordCount job?) cannot open the connection to the address that > you > specified. > > Can you try to run examples/streaming/WordCount.jar. It is a simpler job > which > does not rely on external dependencies. > > If all the above fails, can you tell us how you submit the job? Can you > post > the full command? Can you also post the full JobManager & TaskManager logs? > > Best, > Gary > > > > On Mon, Aug 6, 2018 at 4:10 PM, Felipe Gutierrez < > felipe.o.gutier...@gmail.com> wrote: > >> do you mean "nc -l 9000"? If so, I did start before. >> the task manager running on the master can connect to the job manager. >> but the task manager on the slave node cannot. The second time that I start >> the WordCount task it recognizes only one task manager (from the master) >> and runs my task. But the task manager from the slave does not process >> anything and it is started. >> >> here is the error stack trace from the slave node: >> >> 2017-05-30 05:10:39,853 INFO >> org.apache.flink.runtime.state.heap.HeapKeyedStateBackend - >> Initializing heap keyed state backend with stream factory. >> 2017-05-30 05:10:39,977 INFO org.apache.flink.runtime.taskmanager.Task >> - Source: Socket Stream -> Flat Map (1/1) >> (d5e3d87395995d3977d2f472de896e23) switched from RUNNING to FAILED. >> java.net.ConnectException: Connection refused >> at java.net.PlainSocketImpl.socketConnect(Native Method) >> at java.net >> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) >> at java.net >> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) >> at java.net >> .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) >> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >> at java.net.Socket.connect(Socket.java:589) >> at >> org.apache.flink.streaming.api.functions.source.SocketTextStreamFunction.run(SocketTextStreamFunction.java:96) >> at >> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87) >> at >> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56) >> at >> org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99) >> at >> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306) >> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) >> at java.lang.Thread.run(Thread.java:745) >> 2017-05-30 05:10:40,016 INFO org.apache.flink.runtime.taskmanager.Task >> - Freeing task resources for Source: Socket Stream -> >> Flat Map (1/1) (d5e3d87395995d3977d2f472de896e23). >> >> >> *--* >> *-- Felipe Gutierrez* >> >> *-- skype: felipe.o.gutierrez* >> *--* *https://felipeogutierrez.blogspot.com >> <https://felipeogutierrez.blogspot.com>* >> >> >> On Mon, Aug 6, 2018 at 2:17 PM vino yang <yanghua1...@gmail.com> wrote: >> >>> Hi Felipe, >>> >>> From the exception information, it seems that you did not start the >>> socket server, the socket source needs to connect to the socket server. >>> >>> Please make sure the socket server has started and is available. >>> >>> Thanks, vino. >>> >>> 2018-08-06 18:45 GMT+08:00 Felipe Gutierrez < >>> felipe.o.gutier...@gmail.com>: >>> >>>> yes. >>>> >>>> when I execute the jps command on the master node I >>>> see TaskManagerRunner and StandaloneSessionClusterEntrypoint (which I >>>> believe it is the jobManager). On the slave nodes I see TaskManagerRunner >>>> when I run jps command >>>> >>>> >>>> *--* >>>> *-- Felipe Gutierrez* >>>> >>>> *-- skype: felipe.o.gutierrez* >>>> *--* *https://felipeogutierrez.blogspot.com >>>> <https://felipeogutierrez.blogspot.com>* >>>> >>>> >>>> On Mon, Aug 6, 2018 at 12:13 PM miki haiat <miko5...@gmail.com> wrote: >>>> >>>>> Did you start job manager and task manager on the same resbery pi ? >>>>> >>>>> On Mon, 6 Aug 2018, 12:01 Felipe Gutierrez, < >>>>> felipe.o.gutier...@gmail.com> wrote: >>>>> >>>>>> Hello everyone, >>>>>> >>>>>> I am trying to run Flink on Raspberry Pis. My first test for word >>>>>> count in a single node worked. I just have to decrease the Heap memory of >>>>>> the jobmanager.heap.mb and taskmanager.heap.mb to 512. >>>>>> My second test is to add 2 slave nodes I got the error: "Java >>>>>> HotSpot(TM) Client VM warning: G1 GC is disabled in this release." at the >>>>>> file log/flink-root-taskexecutor-0-*.out. >>>>>> >>>>>> This link ( >>>>>> https://blog.sflow.com/2016/06/raspberry-pi-real-time-network-analytics.html) >>>>>> says that in order to Raspberry Pi ARM architecture works with JVM it is >>>>>> necessary to configure the JVM as: >>>>>> -Xms600M >>>>>> -Xmx600M >>>>>> -XX:+UseParNewGC >>>>>> -XX:+UseConcMarkSweepGC >>>>>> -XX:+CMSIncrementalMode >>>>>> >>>>>> then I set this variables on the path inside the file flink-conf.yaml >>>>>> env.java.opts: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC >>>>>> -XX:+CMSIncrementalMode" >>>>>> env.java.opts.jobmanager: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC >>>>>> -XX:+CMSIncrementalMode" >>>>>> env.java.opts.taskmanager: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC >>>>>> -XX:+CMSIncrementalMode" >>>>>> >>>>>> and the error "Java HotSpot(TM) Client VM warning: G1 GC is disabled >>>>>> in this release." is not showing anymore. However, the connection from >>>>>> the >>>>>> master node to the slave node is still not possible. Does anybody know >>>>>> how >>>>>> I must configure flink to deal with that? >>>>>> >>>>>> This is the error stack trace: >>>>>> >>>>>> 2017-05-25 12:40:26,421 INFO >>>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: >>>>>> Socket Stream -> Flat Map (1/1) (b81b6492fc0860367be422d0b0bf4358) >>>>>> switched >>>>>> from DEPLOYING to RUNNING. >>>>>> 2017-05-25 12:40:26,891 INFO >>>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: >>>>>> Socket Stream -> Flat Map (1/1) (b81b6492fc0860367be422d0b0bf4358) >>>>>> switched >>>>>> from RUNNING to FAILED. >>>>>> java.net.ConnectException: Connection refused >>>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>>> at java.net >>>>>> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) >>>>>> at java.net >>>>>> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) >>>>>> at java.net >>>>>> .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) >>>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>>> at java.net.Socket.connect(Socket.java:589) >>>>>> at >>>>>> org.apache.flink.streaming.api.functions.source.SocketTextStreamFunction.run(SocketTextStreamFunction.java:96) >>>>>> at >>>>>> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87) >>>>>> at >>>>>> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56) >>>>>> at >>>>>> org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99) >>>>>> at >>>>>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306) >>>>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) >>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>> 2017-05-25 12:40:26,898 INFO >>>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph - Job >>>>>> Socket >>>>>> Window WordCount (71c6d7796eccf6587d9d1deda0490e09) switched from state >>>>>> RUNNING to FAILING. >>>>>> java.net.ConnectException: Connection refused >>>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>>> at java.net >>>>>> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) >>>>>> at java.net >>>>>> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) >>>>>> at java.net >>>>>> .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) >>>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>>> at java.net.Socket.connect(Socket.java:589) >>>>>> at >>>>>> org.apache.flink.streaming.api.functions.source.SocketTextStreamFunction.run(SocketTextStreamFunction.java:96) >>>>>> at >>>>>> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87) >>>>>> at >>>>>> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56) >>>>>> at >>>>>> org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99) >>>>>> at >>>>>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306) >>>>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) >>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>> 2017-05-25 12:40:26,921 INFO >>>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph - >>>>>> Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger, >>>>>> ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out >>>>>> (1/1) (aa1a0e7ee3a1d3ad8f99b2608bd64c5b) switched from RUNNING to >>>>>> CANCELING. >>>>>> 2017-05-25 12:40:26,975 INFO >>>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph - >>>>>> Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger, >>>>>> ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out >>>>>> (1/1) (aa1a0e7ee3a1d3ad8f99b2608bd64c5b) switched from CANCELING to >>>>>> CANCELED. >>>>>> >>>>>> >>>>>> >>>>>> Thanks, Felipe >>>>>> *--* >>>>>> *-- Felipe Gutierrez* >>>>>> >>>>>> *-- skype: felipe.o.gutierrez* >>>>>> *--* *https://felipeogutierrez.blogspot.com >>>>>> <https://felipeogutierrez.blogspot.com>* >>>>>> >>>>> >>> >
