[
https://issues.apache.org/jira/browse/KYLIN-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344267#comment-17344267
]
ASF subversion and git services commented on KYLIN-4964:
--------------------------------------------------------
Commit 8497add0b8d04647ea8cf0ff2265af46104a88a0 in kylin's branch
refs/heads/master from dixingxing
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8497add ]
[KYLIN-4964] Receiver consumer thread should be stoped while encounting
unrecoverable error (#1622)
* Improve error handle for streaming receiver, stop consumer thread while
encounting unrecoverable error
* KYLIN-4964 Receiver consumer thread should be stoped while encounting
unrecoverable error
Co-authored-by: dixingxing <dixingx...@autohome.com.cn>
> Receiver consumer thread should be stoped while encounting unrecoverable error
> ------------------------------------------------------------------------------
>
> Key: KYLIN-4964
> URL: https://issues.apache.org/jira/browse/KYLIN-4964
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
> Affects Versions: v3.1.1
> Reporter: Xingxing Di
> Assignee: Xingxing Di
> Priority: Major
>
> org.apache.kylin.stream.core.storage.columnar.ColumnarMemoryStorePersister#persist
> will catch exception and just log it, this behavior will cause critical
> problem, like our case: there is no space left on device, `persist` method
> failed for many times, we lost several hours of data.
> Here is our solution which already tested:
> # Throw IllegalStorageException while we cannot do the persist
> # Then stop the consumer thread
> # Add `consumer_thread_alive` in ConsumerStats for monitoring
> # Also fix an another issue which will cause wrong result after restore from
> checkpoint
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
