[
https://issues.apache.org/jira/browse/KYLIN-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173893#comment-17173893
]
ASF GitHub Bot commented on KYLIN-4683:
---------------------------------------
hit-lacus edited a comment on pull request #1351:
URL: https://github.com/apache/kylin/pull/1351#issuecomment-671064075
1. I will kill the producer.
2. check latest offset via Python
3. Resize topic partition.
```sh
[root@cdh-client xiaoxiang.yu]# kafka-topics --alter --topic useraction_xxyu
--zookeeper cdh-master:2181 --partitions 16
20/08/09 23:13:33 INFO utils.Log4jControllerRegistration$: Registered
kafka:type=kafka.Log4jController MBean
20/08/09 23:13:34 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient]
Initializing a new session to cdh-master:2181.
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.5-cdh5.14.2--1, built on 03/27/2018 20:39 GMT
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:host.name=cdh-client
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.8.0_171
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Oracle Corporation
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:java.home=/usr/java/jdk1.8.0_171/jre
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:os.version=2.6.32-754.17.1.el6.x86_64
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client environment:user.name=root
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:user.home=/root
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/root/xiaoxiang.yu
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=cdh-master:2181 sessionTimeout=30000
watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@6302bbb1
20/08/09 23:13:34 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Waiting
until connected.
20/08/09 23:13:34 INFO zookeeper.ClientCnxn: Opening socket connection to
server cdh-master/10.1.3.90:2181. Will not attempt to authenticate using SASL
(unknown error)
20/08/09 23:13:34 INFO zookeeper.ClientCnxn: Socket connection established,
initiating session, client: /10.1.3.91:54122, server: cdh-master/10.1.3.90:2181
20/08/09 23:13:34 INFO zookeeper.ClientCnxn: Session establishment complete
on server cdh-master/10.1.3.90:2181, sessionid = 0x172bdedb56fcc33, negotiated
timeout = 30000
20/08/09 23:13:34 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient]
Connected.
WARNING: If partitions are increased for a topic that has a key, the
partition logic or ordering of the messages will be affected
20/08/09 23:13:34 INFO zk.AdminZkClient: Creating 6 partitions for
'useraction_xxyu' with the following replica assignment: Map(11 ->
ArrayBuffer(115), 14 -> ArrayBuffer(115), 13 -> ArrayBuffer(114), 10 ->
ArrayBuffer(114), 12 -> ArrayBuffer(116), 15 -> ArrayBuffer(116)).
20/08/09 23:13:34 INFO zk.AdminZkClient: Topic update Map(useraction_xxyu-8
-> Vector(115), useraction_xxyu-6 -> Vector(116), useraction_xxyu-0 ->
Vector(116), useraction_xxyu-13 -> ArrayBuffer(114), useraction_xxyu-1 ->
Vector(114), useraction_xxyu-9 -> Vector(116), useraction_xxyu-15 ->
ArrayBuffer(116), useraction_xxyu-11 -> ArrayBuffer(115), useraction_xxyu-5 ->
Vector(115), useraction_xxyu-2 -> Vector(115), useraction_xxyu-10 ->
ArrayBuffer(114), useraction_xxyu-14 -> ArrayBuffer(115), useraction_xxyu-7 ->
Vector(114), useraction_xxyu-12 -> ArrayBuffer(116), useraction_xxyu-4 ->
Vector(114), useraction_xxyu-3 -> Vector(116))
Adding partitions succeeded!
20/08/09 23:13:34 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closing.
20/08/09 23:13:34 INFO zookeeper.ZooKeeper: Session: 0x172bdedb56fcc33 closed
20/08/09 23:13:34 INFO zookeeper.ClientCnxn: EventThread shut down
20/08/09 23:13:34 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closed.
```
4. check partition again
<img width="801" alt="image"
src="https://user-images.githubusercontent.com/14030549/89735618-77517c80-da96-11ea-950b-69fdcdf59cf7.png";>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Fail to consume kafka when partition number get larger
> ------------------------------------------------------
>
> Key: KYLIN-4683
> URL: https://issues.apache.org/jira/browse/KYLIN-4683
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v3.0.2
> Reporter: tianhui
> Priority: Major
> Attachments: image-2020-08-05-17-20-37-270.png
>
>
> I run a testing streaming cube with kafka. At first, the topic has 3
> partitions, and the cube running smoothly. But after I alter kafka topic to 7
> partitions, all receivers stop consuming. !image-2020-08-05-17-20-37-270.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
![https://user-images.githubusercontent.com/14030549/89735618-77517c80-da96-11ea-950b-69fdcdf59cf7.png"](https://user-images.githubusercontent.com/14030549/89735618-77517c80-da96-11ea-950b-69fdcdf59cf7.png"){kind=link}