Hi Xiaoxiang, Just to give an update, I was able to launch stream consumer after changing the port in kylin.stream.node and the rest of functionality worked great. However, I still don't understand why the default port (9090) is the same as the port used by ThriftServer.
Thanks & Best Regards, Kirill On Mon, 27 Jul 2020 at 11:21, Kirill Bogdanov <kirill...@gmail.com> wrote: > Hi Xiaoxiang, > > Thanks for your reply. You are right, I did not correctly identified that > ThriftServer is already using port 9090, thank you for your commands. > However, could you clarify how I can start Kylin streaming now? Should I > change streaming port or should I change ThriftServer port from 9090 to > something else? What is the correct configuration in this case? > > [hadoop@ip-172-31-0-218 ~]$ sudo netstat -anp | grep LISTEN | grep 9090 > tcp 0 0 :::9090 :::* > LISTEN 4013/java > [hadoop@ip-172-31-0-218 ~]$ sudo jps -mlv | grep 4013 > 4013 org.apache.hadoop.hbase.thrift.ThriftServer start -Dproc_thrift > -XX:OnOutOfMemoryError=kill -9 %p -Xmx1024m -XX:+UseConcMarkSweepGC > -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=70 > -Dnetworkaddress.cache.ttl=5 -Dhbase.log.dir=/var/log/hbase > -Dhbase.log.file=hbase-hbase-thrift-ip-172-31-0-218.log > -Dhbase.home.dir=/usr/lib/hbase -Dhbase.id.str=hbase > -Dhbase.root.logger=INFO,DRFA > -Djava.library.path=:/usr/lib/hadoop-lzo/lib/native:/usr/lib/hadoop/lib/native > -Dhbase.security.logger=INFO,DRFAS > > I will write my next questions to the user mailing list. > > Best regards, > Kirill > > On Mon, 27 Jul 2020 at 03:28, Xiaoxiang Yu <x...@apache.org> wrote: > >> Did you confirm that 9090 is not occpied? >> >> For example, using following command: >> ``` >> [root@cdh-master ~]# netstat -anp | grep LISTEN | grep 9090 >> tcp 0 0 0.0.0.0:9090 0.0.0.0:* >> LISTEN 32274/java >> [root@cdh-master ~]# >> [root@cdh-master ~]# jps -mlv | grep 32274 >> 32274 org.apache.hadoop.hbase.thrift.ThriftServer start --port 9090 >> -threadpool --bind 0.0.0.0 -Dproc_thrift -XX:OnOutOfMemoryError=kill -9 %p >> -Djava.net.preferIPv4Stack=true -Xms1073741824 -Xmx1073741824 >> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC >> -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled >> -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh >> -Dhbase.log.dir=/var/log/hbase >> -Dhbase.log.file=hbase-cmf-hbase-HBASETHRIFTSERVER-cdh-master.log.out >> -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase >> -Dhbase.id.str= -Dhbase.root.logger=INFO,RFA >> -Djava.library.path=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase/lib/native/Linux-amd64-64 >> -Dhbase.security.logger=INFO,NullAppender >> ``` >> >> Besides, I guess using user mailing maybe more suitable. >> >> >> >> >> >> -- >> *Best wishes to you ! * >> *From ：**Xiaoxiang Yu* >> >> >> >> At 2020-07-25 00:11:48, "Kirill Bogdanov" <kirill...@gmail.com> wrote: >> >Hi, >> > >> >Apologies if this is a wrong thread to post to, but I have a Kylin question >> >that I couldn't resolve myself yet. >> > >> >I am deploying Kylin on AWS EMR by following the official tutorial >> >http://kylin.apache.org/docs/install/kylin_aws_emr.html >> >I am interested in running Near RT Streaming using Kafka. My Kafka is >> >deployed on AWS MSK. I managed to deploy Kylin, connect it with Kafka >> >stream and configure model and the cube, however, I have a problem enabling >> >the cube to run OLAP. >> > >> >When I execute: ./bin/kylin.sh streaming start I am getting this error >> >message in the streaming logs: >> > >> >2020-07-24T15:58:12,763 ERROR [main] >> >org.apache.kylin.stream.server.StreamingReceiver - streaming receiver start >> >fail >> >java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:9090 >> > at >> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at >> >org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at >> >org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at >> >org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at >> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at org.eclipse.jetty.server.Server.doStart(Server.java:385) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at >> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at >> >org.apache.kylin.stream.server.StreamingReceiver.startHttpServer(StreamingReceiver.java:94) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at >> >org.apache.kylin.stream.server.StreamingReceiver.start(StreamingReceiver.java:63) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > at >> >org.apache.kylin.stream.server.StreamingReceiver.main(StreamingReceiver.java:51) >> >[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> >Caused by: java.net.BindException: Address already in use >> > at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_252] >> > at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_252] >> > at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_252] >> > at >> >sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220) >> >~[?:1.8.0_252] >> > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85) >> >~[?:1.8.0_252] >> > at >> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342) >> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0] >> > ... 9 more >> > >> >This is strange because there is no receiver running and no process is >> >using port 9090. What else should I check? Did I miss some configuration? >> > >> >Thanks & best regards, >> >Kirill >> >>
