To increase your open file limits on your system, please refer to the content on below link. Let us know if that resolved it for you. https://easyengine.io/tutorials/linux/increase-open-files-limit/
Thanks, Khurram On Tue, Jun 11, 2019 at 10:45 PM Khurram Faraaz <kfar...@mapr.com> wrote: > Hi, > > By looking at the stack trace, the issue seems to not originate from > Drill, instead it is a Kafka issue, did you try to increase the limit on > the OS to avoid the Too many files open error ? > What version of Apache Drill, Kafka and OS are you trying this on ? > > Caused by: org.apache.kafka.common.KafkaException: > java.io.IOException: Too many open files > at org.apache.kafka.common.network.Selector.<init>(Selector.java:129) > ~[kafka-clients-0.11.0.1.jar:na] > at org.apache.kafka.common.network.Selector.<init>(Selector.java:156) > ~[kafka-clients-0.11.0.1.jar:na] > at org.apache.kafka.common.network.Selector.<init>(Selector.java:160) > ~[kafka-clients-0.11.0.1.jar:na] > at > org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:701) > ~[kafka-clients-0.11.0.1.jar:na] > ... 25 common frames omitted > Caused by: java.io.IOException: Too many open files > at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method) > ~[na:1.8.0_181] > at sun.nio.ch.EPollArrayWrapper.<init>(EPollArrayWrapper.java:130) > ~[na:1.8.0_181] > at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:69) > ~[na:1.8.0_181] > at sun.nio.ch > .EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36) > ~[na:1.8.0_181] > at java.nio.channels.Selector.open(Selector.java:227) ~[na:1.8.0_181] > at org.apache.kafka.common.network.Selector.<init>(Selector.java:127) > ~[kafka-clients-0.11.0.1.jar:na] > > Thanks, > Khurram > > On Tue, Jun 11, 2019 at 2:38 PM Aravind Voruganti < > aravind.vorugant...@gmail.com> wrote: > >> Hello Apache Drill support, >> >> I am Aravind Voruganti a Java developer and a big fan of Apache Drill. >> Below is the issue I was facing with my new application when using Drill >> over Kafka for querying. I have also detailed the issue in Stackoverflow >> which can found at: >> >> >> https://stackoverflow.com/questions/56549783/failed-to-construct-kafka-consumer-using-apache-drill >> >> I am using the Apache Drill (1.14) JDBC driver in my application which >> consumes the data from the Kafka. The application works just fine for some >> time and after few iterations it fails to execute due to the following Too >> many files open issue. I made sure there are no file handle leaks in my >> code but still nor sure why this issue is happening? >> >> It looks like the issue is happening from with-in the Apache drill >> libraries when constructing the Kafka consumer. Can any one please guide >> me >> help this problem fixed? >> >> The problem perishes when I restart my Apache drillbit but very soon it >> happens again. I did check the file descriptor count on my unix machine >> using ulimit -a | wc -l & lsof -a -p <PID> | wc -l before and after the >> drill process restart and it seems the drill process is considerably >> taking >> a lot of file descriptors. I tried increasing the file descriptor count on >> the system but still no luck. >> >> I have followed the Apache Drill storage plugin documentation in >> configuring the Kafka plugin into Apache Drill at >> https://drill.apache.org/docs/kafka-storage-plugin/ >> >> Any help on this issue is highly appreciated. Thanks. >> >> JDBC URL: jdbc:drill:drillbit=localhost:31010;schema=kafka >> >> NOTE: I am pushing down the filters in my query SELECT * FROM myKafkaTopic >> WHERE kafkaMsgTimestamp > 1560210931626 >> >> org.apache.drill.common.exceptions.UserException: DATA_READ ERROR: >> Failed to fetch start/end offsets of the topic myKafkaTopic >> >> Failed to construct kafka consumer >> >> [Error Id: 73f896a7-09d4-425b-8cd5-f269c3a6e69a ] >> at >> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) >> ~[drill-common-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.store.kafka.KafkaGroupScan.init(KafkaGroupScan.java:198) >> [drill-storage-kafka-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.store.kafka.KafkaGroupScan.<init>(KafkaGroupScan.java:98) >> [drill-storage-kafka-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.store.kafka.KafkaStoragePlugin.getPhysicalScan(KafkaStoragePlugin.java:83) >> [drill-storage-kafka-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.store.AbstractStoragePlugin.getPhysicalScan(AbstractStoragePlugin.java:111) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.logical.DrillTable.getGroupScan(DrillTable.java:99) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:89) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:69) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.logical.DrillScanRel.<init>(DrillScanRel.java:62) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.logical.DrillScanRule.onMatch(DrillScanRule.java:38) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212) >> [calcite-core-1.16.0-drill-r6.jar:1.16.0-drill-r6] >> at >> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:652) >> [calcite-core-1.16.0-drill-r6.jar:1.16.0-drill-r6] >> at >> org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:368) >> [calcite-core-1.16.0-drill-r6.jar:1.16.0-drill-r6] >> at >> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:429) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:369) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:255) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:318) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:180) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:145) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:83) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at org.apache.drill.exec.work >> .foreman.Foreman.runSQL(Foreman.java:567) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:266) >> [drill-java-exec-1.14.0.jar:1.14.0] >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> [na:1.8.0_181] >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> [na:1.8.0_181] >> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181] >> Caused by: org.apache.kafka.common.KafkaException: Failed to construct >> kafka consumer >> at >> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:765) >> ~[kafka-clients-0.11.0.1.jar:na] >> at >> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:633) >> ~[kafka-clients-0.11.0.1.jar:na] >> at >> org.apache.drill.exec.store.kafka.KafkaGroupScan.init(KafkaGroupScan.java:168) >> [drill-storage-kafka-1.14.0.jar:1.14.0] >> ... 23 common frames omitted >> Caused by: org.apache.kafka.common.KafkaException: >> java.io.IOException: Too many open files >> at org.apache.kafka.common.network.Selector.<init>(Selector.java:129) >> ~[kafka-clients-0.11.0.1.jar:na] >> at org.apache.kafka.common.network.Selector.<init>(Selector.java:156) >> ~[kafka-clients-0.11.0.1.jar:na] >> at org.apache.kafka.common.network.Selector.<init>(Selector.java:160) >> ~[kafka-clients-0.11.0.1.jar:na] >> at >> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:701) >> ~[kafka-clients-0.11.0.1.jar:na] >> ... 25 common frames omitted >> Caused by: java.io.IOException: Too many open files >> at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method) >> ~[na:1.8.0_181] >> at sun.nio.ch.EPollArrayWrapper.<init>(EPollArrayWrapper.java:130) >> ~[na:1.8.0_181] >> at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:69) >> ~[na:1.8.0_181] >> at sun.nio.ch >> .EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36) >> ~[na:1.8.0_181] >> at java.nio.channels.Selector.open(Selector.java:227) ~[na:1.8.0_181] >> at org.apache.kafka.common.network.Selector.<init>(Selector.java:127) >> ~[kafka-clients-0.11.0.1.jar:na]``` >> >> I am struck in my application because of this issue and your help in >> solving this issue means a lot to me. >> >> Thanks >> Aravind >> >
