[
https://issues.apache.org/jira/browse/DRILL-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610148#comment-16610148
]
ASF GitHub Bot commented on DRILL-6625:
---------------------------------------
aravi5 opened a new pull request #1463: DRILL-6625: Intermittent failures in
Kafka unit tests
URL: https://github.com/apache/drill/pull/1463
### Fix producer errors
- Increase the value of `ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG` to
accomadate slower systems.
- Increase the value of producer `ProducerConfig.RETRIES_CONFIG` to 3 (from
0).
- Prevent producer to send duplicate messages due to retries by setting
`ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG` to `true`.
### Fix consumer errors
- Increase `ExecConstants.KAFKA_POLL_TIMEOUT` (from 200 ms).
### Stabilize KafkaFilterPushdownTest
- Catch the issue with setup early by checking if expected number of records
exist before running tests in `KafkaFilterPushdownTest` class.
Please refer to
[DRILL-6625](https://issues.apache.org/jira/browse/DRILL-6625) for more details
about the fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Intermittent failures in Kafka unit tests
> -----------------------------------------
>
> Key: DRILL-6625
> URL: https://issues.apache.org/jira/browse/DRILL-6625
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Other
> Affects Versions: 1.13.0
> Reporter: Boaz Ben-Zvi
> Assignee: Abhishek Ravi
> Priority: Major
> Fix For: 1.15.0
>
>
> The following failures have been seen (consistently on my Mac, or
> occasionally on Jenkins) when running the unit tests, in the Kafka test suit.
> After the failure, maven hangs for a long time.
> Cost was 0.0 (instead of 26.0) :
> {code:java}
> Running org.apache.drill.exec.store.kafka.KafkaFilterPushdownTest
> 16:46:57.748 [main] ERROR org.apache.drill.TestReporter - Test Failed (d:
> -65.3 KiB(73.6 KiB), h: -573.5 MiB(379.5 MiB), nh: 1.2 MiB(117.1 MiB)):
> testPushdownWithOr(org.apache.drill.exec.store.kafka.KafkaFilterPushdownTest)
> java.lang.AssertionError: Unable to find expected string "kafkaScanSpec"
> : {
> "topicName" : "drill-pushdown-topic"
> },
> "cost" : 26.0 in plan: {
> "head" : {
> "version" : 1,
> "generator" : {
> "type" : "ExplainHandler",
> "info" : ""
> },
> "type" : "APACHE_DRILL_PHYSICAL",
> "options" : [ {
> "kind" : "STRING",
> "accessibleScopes" : "ALL",
> "name" : "store.kafka.record.reader",
> "string_val" :
> "org.apache.drill.exec.store.kafka.decoders.JsonMessageReader",
> "scope" : "SESSION"
> }, {
> "kind" : "LONG",
> "accessibleScopes" : "ALL",
> "name" : "planner.width.max_per_node",
> "num_val" : 2,
> "scope" : "SESSION"
> }, {
> "kind" : "BOOLEAN",
> "accessibleScopes" : "ALL",
> "name" : "exec.errors.verbose",
> "bool_val" : true,
> "scope" : "SESSION"
> }, {
> "kind" : "LONG",
> "accessibleScopes" : "ALL",
> "name" : "store.kafka.poll.timeout",
> "num_val" : 200,
> "scope" : "SESSION"
> } ],
> "queue" : 0,
> "hasResourcePlan" : false,
> "resultMode" : "EXEC"
> },
> "graph" : [ {
> "pop" : "kafka-scan",
> "@id" : 6,
> "userName" : "",
> "kafkaStoragePluginConfig" : {
> "type" : "kafka",
> "kafkaConsumerProps" : {
> "bootstrap.servers" : "127.0.0.1:63751",
> "group.id" : "drill-test-consumer"
> },
> "enabled" : true
> },
> "columns" : [ "`**`" ],
> "kafkaScanSpec" : {
> "topicName" : "drill-pushdown-topic"
> },
> "cost" : 0.0
> }, {
> {code}
> Or occasionally:
> {code}
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> 11:52:57.571 [main] ERROR o.a.d.e.s.k.KafkaMessageGenerator -
> org.apache.kafka.common.errors.NetworkException: The server disconnected
> before a response was received.
> java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.NetworkException: The server disconnected
> before a response was received.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
