Hi,
Thanks for the update, I am glad to hear your good news.
By the way, the question you asked, "why the default port (9090) is the
same as the port used by ThriftServer", I didn't think it is a seroius issue. I
guess head node is suitable for deploment of ThriftServer, and worker node is
suitable for deployment of Streaming Receiver. These two instances have little
chance to be located at same node.
--
Best wishes to you !
From ：Xiaoxiang Yu
At 2020-07-27 23:53:24, "Kirill Bogdanov" <kirill...@gmail.com> wrote:
>Hi Xiaoxiang,
>
>Just to give an update, I was able to launch stream consumer after changing
>the port in kylin.stream.node and the rest of functionality worked great.
>However, I still don't understand why the default port (9090) is the same
>as the port used by ThriftServer.
>
>Thanks & Best Regards,
>Kirill
>
>On Mon, 27 Jul 2020 at 11:21, Kirill Bogdanov <kirill...@gmail.com> wrote:
>
>> Hi Xiaoxiang,
>>
>> Thanks for your reply. You are right, I did not correctly identified that
>> ThriftServer is already using port 9090, thank you for your commands.
>> However, could you clarify how I can start Kylin streaming now? Should I
>> change streaming port or should I change ThriftServer port from 9090 to
>> something else? What is the correct configuration in this case?
>>
>> [hadoop@ip-172-31-0-218 ~]$ sudo netstat -anp | grep LISTEN | grep 9090
>> tcp 0 0 :::9090 :::*
>> LISTEN 4013/java
>> [hadoop@ip-172-31-0-218 ~]$ sudo jps -mlv | grep 4013
>> 4013 org.apache.hadoop.hbase.thrift.ThriftServer start -Dproc_thrift
>> -XX:OnOutOfMemoryError=kill -9 %p -Xmx1024m -XX:+UseConcMarkSweepGC
>> -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=70
>> -Dnetworkaddress.cache.ttl=5 -Dhbase.log.dir=/var/log/hbase
>> -Dhbase.log.file=hbase-hbase-thrift-ip-172-31-0-218.log
>> -Dhbase.home.dir=/usr/lib/hbase -Dhbase.id.str=hbase
>> -Dhbase.root.logger=INFO,DRFA
>> -Djava.library.path=:/usr/lib/hadoop-lzo/lib/native:/usr/lib/hadoop/lib/native
>> -Dhbase.security.logger=INFO,DRFAS
>>
>> I will write my next questions to the user mailing list.
>>
>> Best regards,
>> Kirill
>>
>> On Mon, 27 Jul 2020 at 03:28, Xiaoxiang Yu <x...@apache.org> wrote:
>>
>>> Did you confirm that 9090 is not occpied?
>>>
>>> For example, using following command:
>>> ```
>>> [root@cdh-master ~]# netstat -anp | grep LISTEN | grep 9090
>>> tcp 0 0 0.0.0.0:9090 0.0.0.0:*
>>> LISTEN 32274/java
>>> [root@cdh-master ~]#
>>> [root@cdh-master ~]# jps -mlv | grep 32274
>>> 32274 org.apache.hadoop.hbase.thrift.ThriftServer start --port 9090
>>> -threadpool --bind 0.0.0.0 -Dproc_thrift -XX:OnOutOfMemoryError=kill -9 %p
>>> -Djava.net.preferIPv4Stack=true -Xms1073741824 -Xmx1073741824
>>> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
>>> -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled
>>> -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
>>> -Dhbase.log.dir=/var/log/hbase
>>> -Dhbase.log.file=hbase-cmf-hbase-HBASETHRIFTSERVER-cdh-master.log.out
>>> -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase
>>> -Dhbase.id.str= -Dhbase.root.logger=INFO,RFA
>>> -Djava.library.path=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase/lib/native/Linux-amd64-64
>>> -Dhbase.security.logger=INFO,NullAppender
>>> ```
>>>
>>> Besides, I guess using user mailing maybe more suitable.
>>>
>>>
>>>
>>>
>>>
>>> --
>>> *Best wishes to you ! *
>>> *From ：**Xiaoxiang Yu*
>>>
>>>
>>>
>>> At 2020-07-25 00:11:48, "Kirill Bogdanov" <kirill...@gmail.com> wrote:
>>> >Hi,
>>> >
>>> >Apologies if this is a wrong thread to post to, but I have a Kylin question
>>> >that I couldn't resolve myself yet.
>>> >
>>> >I am deploying Kylin on AWS EMR by following the official tutorial
>>> >http://kylin.apache.org/docs/install/kylin_aws_emr.html
>>> >I am interested in running Near RT Streaming using Kafka. My Kafka is
>>> >deployed on AWS MSK. I managed to deploy Kylin, connect it with Kafka
>>> >stream and configure model and the cube, however, I have a problem enabling
>>> >the cube to run OLAP.
>>> >
>>> >When I execute: ./bin/kylin.sh streaming start I am getting this error
>>> >message in the streaming logs:
>>> >
>>> >2020-07-24T15:58:12,763 ERROR [main]
>>> >org.apache.kylin.stream.server.StreamingReceiver - streaming receiver start
>>> >fail
>>> >java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:9090
>>> > at
>>> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at org.eclipse.jetty.server.Server.doStart(Server.java:385)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.apache.kylin.stream.server.StreamingReceiver.startHttpServer(StreamingReceiver.java:94)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.apache.kylin.stream.server.StreamingReceiver.start(StreamingReceiver.java:63)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.apache.kylin.stream.server.StreamingReceiver.main(StreamingReceiver.java:51)
>>> >[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> >Caused by: java.net.BindException: Address already in use
>>> > at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_252]
>>> > at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_252]
>>> > at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_252]
>>> > at
>>> >sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
>>> >~[?:1.8.0_252]
>>> > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
>>> >~[?:1.8.0_252]
>>> > at
>>> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > ... 9 more
>>> >
>>> >This is strange because there is no receiver running and no process is
>>> >using port 9090. What else should I check? Did I miss some configuration?
>>> >
>>> >Thanks & best regards,
>>> >Kirill
>>>
>>>
