hit-lacus commented on pull request #1351:
URL: https://github.com/apache/kylin/pull/1351#issuecomment-671065519
1. Go to web page and re-enable Cube, you will see some failure, please
don't be in a panic.
```sh
2020-08-09 23:23:10,437 INFO [qtp193625667-35]
controller.AdminController:58 : receive assign
request:AssignRequest{cubeName='UserAnalysisCube', partitions=[Partition
[partitionId=0, partitionInfo=null], Partition [partitionId=5,
partitionInfo=null], Partition [partitionId=10, partitionInfo=null], Partition
[partitionId=15, partitionInfo=null], Partition [partitionId=13,
partitionInfo=null], Partition [partitionId=2, partitionInfo=null], Partition
[partitionId=8, partitionInfo=null], Partition [partitionId=12,
partitionInfo=null], Partition [partitionId=14, partitionInfo=null], Partition
[partitionId=9, partitionInfo=null], Partition [partitionId=11,
partitionInfo=null], Partition [partitionId=1, partitionInfo=null], Partition
[partitionId=4, partitionInfo=null], Partition [partitionId=6,
partitionInfo=null], Partition [partitionId=7, partitionInfo=null], Partition
[partitionId=3, partitionInfo=null]], startConsumers=true}
2020-08-09 23:23:10,437 INFO [qtp193625667-35] server.StreamingServer:351 :
create and start new consumer for cube:UserAnalysisCube
2020-08-09 23:23:10,468 INFO [qtp193625667-35] kafka.KafkaSource:141 :
Create StreamingConnector for Cube UserAnalysisCube, assignedPartitions
[Partition [partitionId=0, partitionInfo=null], Partition [partitionId=5,
partitionInfo=null], Partition [partitionId=10, partitionInfo=null], Partition
[partitionId=15, partitionInfo=null], Partition [partitionId=13,
partitionInfo=null], Partition [partitionId=2, partitionInfo=null], Partition
[partitionId=8, partitionInfo=null], Partition [partitionId=12,
partitionInfo=null], Partition [partitionId=14, partitionInfo=null], Partition
[partitionId=9, partitionInfo=null], Partition [partitionId=11,
partitionInfo=null], Partition [partitionId=1, partitionInfo=null], Partition
[partitionId=4, partitionInfo=null], Partition [partitionId=6,
partitionInfo=null], Partition [partitionId=7, partitionInfo=null], Partition
[partitionId=3, partitionInfo=null]], startProtocol null
2020-08-09 23:23:10,470 INFO [qtp193625667-35]
kafka.TimedJsonStreamParser:98 : Using parser field mapping by
{content_list=content_list, act_type=act_type, event_ts_2=event_ts_2,
event_ts=event_ts, devide_type=user_detail.devide_type,
location_city=user_detail.location.city, network_type=user_detail.network_type,
video_id=video_id, event_date_2=event_date_2, str_minute=str_minute,
video_type=video_type, play_times=play_times, pageview_id=pageview_id,
active_minutes=active_minutes, device_brand=device_brand,
str_minute_second=str_minute_second, play_duration=play_duration,
event_date=event_date, page_id=page_id, str_second=str_second, uid=uid}
2020-08-09 23:23:10,470 INFO [qtp193625667-35]
kafka.TimedJsonStreamParser:120 : TimedJsonStreamParser with formatTs false
tsColName event_date
2020-08-09 23:23:10,470 INFO [qtp193625667-35] consumer.ConsumerConfig:279
: ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = latest
bootstrap.servers = [cdh-master:9092, cdh-worker-1:9092,
cdh-worker-2:9092]
check.crcs = true
client.dns.lookup = default
client.id =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id =
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = true
isolation.level = read_uncommitted
key.deserializer = class
org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 500
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class
org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 20000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class
org.apache.kafka.common.serialization.ByteArrayDeserializer
2020-08-09 23:23:10,473 INFO [qtp193625667-35] utils.AppInfoParser:109 :
Kafka version : 2.1.1
2020-08-09 23:23:10,473 INFO [qtp193625667-35] utils.AppInfoParser:110 :
Kafka commitId : 21234bee31165527
2020-08-09 23:23:10,475 INFO [qtp193625667-35] kafka.KafkaSource:192 :
localConsumeStats from local checkpoint null, remoteConsumeStats from remote
checkpoint
{"0":363171,"1":363198,"2":363249,"3":363171,"4":363199,"5":363250,"6":363170,"7":363196,"8":363250,"9":363170}
2020-08-09 23:23:10,475 INFO [qtp193625667-35] kafka.KafkaSource:239 :
start kafka connector from specified
position:KafkaPosition{partitionOffsetMap={0=363171, 1=363198, 2=363249,
3=363171, 4=363199, 5=363250, 6=363170, 7=363196, 8=363250, 9=363170}}
2020-08-09 23:23:10,476 ERROR [qtp193625667-35] server.StreamingServer:357 :
consumer start fail for cube:UserAnalysisCube
java.lang.NullPointerException
at
org.apache.kylin.stream.source.kafka.consumer.KafkaConnector.open(KafkaConnector.java:86)
at
org.apache.kylin.stream.core.consumer.StreamingConsumerChannel.start(StreamingConsumerChannel.java:85)
at
org.apache.kylin.stream.server.StreamingServer.startConsumer(StreamingServer.java:355)
at
org.apache.kylin.stream.server.StreamingServer.startConsumers(StreamingServer.java:327)
at
org.apache.kylin.stream.server.rest.controller.AdminController.assign(AdminController.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:849)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:760)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:500)
at
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.lang.Thread.run(Thread.java:748)
```
2. Let try doing a reassignment.
<img width="1517" alt="image"
src="https://user-images.githubusercontent.com/14030549/89735825-bd5b1000-da97-11ea-9944-ae54331002aa.png";>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
![https://user-images.githubusercontent.com/14030549/89735825-bd5b1000-da97-11ea-9944-ae54331002aa.png"](https://user-images.githubusercontent.com/14030549/89735825-bd5b1000-da97-11ea-9944-ae54331002aa.png"){kind=link}