[
https://issues.apache.org/jira/browse/KYLIN-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI closed KYLIN-2281.
-------------------------------
Resolution: Pending Closed
> Pending on the first step(save data from kafka) when building streaming cube
> ----------------------------------------------------------------------------
>
> Key: KYLIN-2281
> URL: https://issues.apache.org/jira/browse/KYLIN-2281
> Project: Kylin
> Issue Type: Test
> Components: Client - CLI
> Affects Versions: v1.6.0
> Environment: CDH5.8 + Kylin1.6 + Kafka0.10
> Reporter: QiLiFei
> Priority: Critical
>
> Hi engineer,
> 1.
> I tried several times to build the streaming cube based on kafka source .
> However , it will always pending on the first step(already take about 160
> mins ) to save data from kafka. I checked the data from kafka . There are
> only about one thousand data . Which is confused for me . Anyone who can help
> me ? I'm not sure what to do next . Is there configuration for kafka or kylin
> need to reset or not ?
> 2.Command
> curl -X PUT --user ADMIN:KYLIN -H "Content-Type:
> application/json;charset=utf-8" -d '{ "sourceOffsetStart": 0,
> "sourceOffsetEnd": 9223372036854775807, "buildType": "BUILD"}'
> http://172.31.18.62:7070/kylin/api/cubes/StreamingCube27/build2
> 3. config and log
> I set the available mb for kylin 16G of 20G. I can see the continously log
> info by command tail -f kylin.log as below . Currenty I've stop the kafka
> producer for this topic.
> 2016-12-15 11:19:41,457 DEBUG [pool-9-thread-1] dao.ExecutableDao:210 :
> updating job output, id: dfb72486-50b8-44c1-bee5-4ad0393ef66e-00
> 2016-12-15 11:19:41,462 DEBUG [pool-9-thread-1] hbase.HBaseResourceStore:262
> : Update row /execute_output/dfb72486-50b8-44c1-bee5-4ad0393ef66e-00 from
> oldTs: 1481771971443, to newTs: 1481771981457, operation result: true
> 2016-12-15 11:19:51,505 DEBUG [pool-9-thread-1] dao.ExecutableDao:210 :
> updating job output, id: dfb72486-50b8-44c1-bee5-4ad0393ef66e-00
> 2016-12-15 11:19:51,510 DEBUG [pool-9-thread-1] hbase.HBaseResourceStore:262
> : Update row /execute_output/dfb72486-50b8-44c1-bee5-4ad0393ef66e-00 from
> oldTs: 1481771981457, to newTs: 1481771991505, operation result: true
> 4.Kafka souce data count （sourceOffset & endOffset）
> phadoopdn1:/opt/kafka_2.11-0.10.1.0 # bin/kafka-run-class.sh
> kafka.tools.GetOffsetShell --broker-list 172.31.18.62:9092 --topic
> kylin_demo30 --time -1
> kylin_demo30: 0:70572
> phadoopdn1:/opt/kafka_2.11-0.10.1.0 #
> Anyone who can help me to figure out this issue ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
