Hi,

By looking at the stack trace, the issue seems to not originate from Drill,
instead it is a Kafka issue, did you try to increase the limit on the OS to
avoid the Too many files open error ?
What version of Apache Drill, Kafka and OS are you trying this on ?

Caused by: org.apache.kafka.common.KafkaException:
java.io.IOException: Too many open files
    at org.apache.kafka.common.network.Selector.<init>(Selector.java:129)
~[kafka-clients-0.11.0.1.jar:na]
    at org.apache.kafka.common.network.Selector.<init>(Selector.java:156)
~[kafka-clients-0.11.0.1.jar:na]
    at org.apache.kafka.common.network.Selector.<init>(Selector.java:160)
~[kafka-clients-0.11.0.1.jar:na]
    at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:701)
~[kafka-clients-0.11.0.1.jar:na]
    ... 25 common frames omitted
Caused by: java.io.IOException: Too many open files
    at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
~[na:1.8.0_181]
    at sun.nio.ch.EPollArrayWrapper.<init>(EPollArrayWrapper.java:130)
~[na:1.8.0_181]
    at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:69)
~[na:1.8.0_181]
    at sun.nio.ch
.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
~[na:1.8.0_181]
    at java.nio.channels.Selector.open(Selector.java:227) ~[na:1.8.0_181]
    at org.apache.kafka.common.network.Selector.<init>(Selector.java:127)
~[kafka-clients-0.11.0.1.jar:na]

Thanks,
Khurram

On Tue, Jun 11, 2019 at 2:38 PM Aravind Voruganti <
aravind.vorugant...@gmail.com> wrote:

> Hello Apache Drill support,
>
> I am Aravind Voruganti a Java developer and a big fan of Apache Drill.
> Below is the issue I was facing with my new application when using Drill
> over Kafka for querying. I have also detailed the issue in Stackoverflow
> which can found at:
>
>
> https://stackoverflow.com/questions/56549783/failed-to-construct-kafka-consumer-using-apache-drill
>
> I am using the Apache Drill (1.14) JDBC driver in my application which
> consumes the data from the Kafka. The application works just fine for some
> time and after few iterations it fails to execute due to the following Too
> many files open issue. I made sure there are no file handle leaks in my
> code but still nor sure why this issue is happening?
>
> It looks like the issue is happening from with-in the Apache drill
> libraries when constructing the Kafka consumer. Can any one please guide me
> help this problem fixed?
>
> The problem perishes when I restart my Apache drillbit but very soon it
> happens again. I did check the file descriptor count on my unix machine
> using ulimit -a | wc -l & lsof -a -p <PID> | wc -l before and after the
> drill process restart and it seems the drill process is considerably taking
> a lot of file descriptors. I tried increasing the file descriptor count on
> the system but still no luck.
>
> I have followed the Apache Drill storage plugin documentation in
> configuring the Kafka plugin into Apache Drill at
> https://drill.apache.org/docs/kafka-storage-plugin/
>
> Any help on this issue is highly appreciated. Thanks.
>
> JDBC URL: jdbc:drill:drillbit=localhost:31010;schema=kafka
>
> NOTE: I am pushing down the filters in my query SELECT * FROM myKafkaTopic
> WHERE kafkaMsgTimestamp > 1560210931626
>
> org.apache.drill.common.exceptions.UserException: DATA_READ ERROR:
> Failed to fetch start/end offsets of the topic  myKafkaTopic
>
> Failed to construct kafka consumer
>
> [Error Id: 73f896a7-09d4-425b-8cd5-f269c3a6e69a ]
>     at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.store.kafka.KafkaGroupScan.init(KafkaGroupScan.java:198)
> [drill-storage-kafka-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.store.kafka.KafkaGroupScan.<init>(KafkaGroupScan.java:98)
> [drill-storage-kafka-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.store.kafka.KafkaStoragePlugin.getPhysicalScan(KafkaStoragePlugin.java:83)
> [drill-storage-kafka-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan(AbstractStoragePlugin.java:111)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.logical.DrillTable.getGroupScan(DrillTable.java:99)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:89)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:69)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:62)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.logical.DrillScanRule.onMatch(DrillScanRule.java:38)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
> [calcite-core-1.16.0-drill-r6.jar:1.16.0-drill-r6]
>     at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:652)
> [calcite-core-1.16.0-drill-r6.jar:1.16.0-drill-r6]
>     at
> org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:368)
> [calcite-core-1.16.0-drill-r6.jar:1.16.0-drill-r6]
>     at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:429)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:369)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:255)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:318)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:180)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:145)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:83)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:567)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:266)
> [drill-java-exec-1.14.0.jar:1.14.0]
>     at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_181]
>     at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_181]
>     at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
> Caused by: org.apache.kafka.common.KafkaException: Failed to construct
> kafka consumer
>     at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:765)
> ~[kafka-clients-0.11.0.1.jar:na]
>     at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:633)
> ~[kafka-clients-0.11.0.1.jar:na]
>     at
> org.apache.drill.exec.store.kafka.KafkaGroupScan.init(KafkaGroupScan.java:168)
> [drill-storage-kafka-1.14.0.jar:1.14.0]
>     ... 23 common frames omitted
> Caused by: org.apache.kafka.common.KafkaException:
> java.io.IOException: Too many open files
>     at org.apache.kafka.common.network.Selector.<init>(Selector.java:129)
> ~[kafka-clients-0.11.0.1.jar:na]
>     at org.apache.kafka.common.network.Selector.<init>(Selector.java:156)
> ~[kafka-clients-0.11.0.1.jar:na]
>     at org.apache.kafka.common.network.Selector.<init>(Selector.java:160)
> ~[kafka-clients-0.11.0.1.jar:na]
>     at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:701)
> ~[kafka-clients-0.11.0.1.jar:na]
>     ... 25 common frames omitted
> Caused by: java.io.IOException: Too many open files
>     at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
> ~[na:1.8.0_181]
>     at sun.nio.ch.EPollArrayWrapper.<init>(EPollArrayWrapper.java:130)
> ~[na:1.8.0_181]
>     at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:69)
> ~[na:1.8.0_181]
>     at sun.nio.ch
> .EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
> ~[na:1.8.0_181]
>     at java.nio.channels.Selector.open(Selector.java:227) ~[na:1.8.0_181]
>     at org.apache.kafka.common.network.Selector.<init>(Selector.java:127)
> ~[kafka-clients-0.11.0.1.jar:na]```
>
> I am struck in my application because of this issue and your help in
> solving this issue means a lot to me.
>
> Thanks
> Aravind
>

Reply via email to