[jira] [Updated] (DRILL-5299) Query queues can be split by different users or different businesses
[ https://issues.apache.org/jira/browse/DRILL-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5299: Description: So far we can have 2 query queues that are split based on the cost model. For serving multiple users and multiple businesses, Drill cluster maintainers would always like to control the query concurrency that are allocated for a concrete user or a concrete businesses. Which means, each user or business can have its own "query pool" defined by a specific "pool size". (was: So far we can have 2 query queues that are split based on the cost model. For serving multiple users and multiple businesses, Drill cluster maintainers always like to control the query concurrency that are allocated for a concrete user or a concrete businesses. Which means, each user or business can have its own "query pool" defined by a specific "pool size".) > Query queues can be split by different users or different businesses > > > Key: DRILL-5299 > URL: https://issues.apache.org/jira/browse/DRILL-5299 > Project: Apache Drill > Issue Type: Improvement > Components: Query Planning & Optimization >Affects Versions: Future > Environment: RH Linux / OpenJDK 8 >Reporter: Hongze Zhang > > So far we can have 2 query queues that are split based on the cost model. For > serving multiple users and multiple businesses, Drill cluster maintainers > would always like to control the query concurrency that are allocated for a > concrete user or a concrete businesses. Which means, each user or business > can have its own "query pool" defined by a specific "pool size". -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (DRILL-5299) Query queues can be split by different users or different businesses
Hongze Zhang created DRILL-5299: --- Summary: Query queues can be split by different users or different businesses Key: DRILL-5299 URL: https://issues.apache.org/jira/browse/DRILL-5299 Project: Apache Drill Issue Type: Improvement Components: Query Planning & Optimization Affects Versions: Future Environment: RH Linux / OpenJDK 8 Reporter: Hongze Zhang So far we can have 2 query queues that are split based on the cost model. For serving multiple users and multiple businesses, Drill cluster maintainers always like to control the query concurrency that are allocated for a concrete user or a concrete businesses. Which means, each user or business can have its own "query pool" defined by a specific "pool size". -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816911#comment-15816911 ] Hongze Zhang edited comment on DRILL-5068 at 1/11/17 7:14 AM: -- Basically, It is a way for user to jump from system table result to detailed profile page directly on the drill Web UI. The hyperlink can be clickable in the Freemarker generated view, and will not be working on JDBC client. Without given a link, users have to copy and paste the queryID to the tail of profile page url to open the detail page. Do you think JDBC users may be confused about this column? It is indeed not pretty printed on console (http flags will be shown there). was (Author: zbdzzg): Basically, It is a way for user to jump from system table result to detailed profile page directly on the drill Web UI. The hyperlink can be clickable in the Freemarker generated view, and will not be working on JDBC client. Without given a link, users have to copy and paste the queryID to the tail of profile page url to open the detail page. Do you think JDBC users may be confused about this column? It is indeed not pretty printed on console (http flags will be show there). > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816911#comment-15816911 ] Hongze Zhang commented on DRILL-5068: - Basically, It is a way for user to jump from system table result to detailed profile page directly on the drill Web UI. The hyperlink can be clickable in the Freemarker generated view, and will not be working on JDBC client. Without given a link, users have to copy and paste the queryID to the tail of profile page url to open the detail page. Do you think JDBC users may be confused about this column? It is indeed not pretty printed on console (http flags will be show there). > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804372#comment-15804372 ] Hongze Zhang commented on DRILL-5068: - The mail has been sent to dev. And could you please review this? > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5068: Reviewer: Khurram Faraaz > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5068: Reviewer: (was: Khurram Faraaz) > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769982#comment-15769982 ] Hongze Zhang commented on DRILL-5068: - Hi Khurram, Thank you for your guide! I will mail to the dev list to ask the question. > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769033#comment-15769033 ] Hongze Zhang commented on DRILL-5068: - [~khfaraaz] Hi, Is this thing useful for next version of Drill ? Thanks! > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-5028) Opening profiles page from web ui gets very slow when a lot of history files have been stored in HDFS or Local FS.
[ https://issues.apache.org/jira/browse/DRILL-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang closed DRILL-5028. --- Resolution: Later > Opening profiles page from web ui gets very slow when a lot of history files > have been stored in HDFS or Local FS. > -- > > Key: DRILL-5028 > URL: https://issues.apache.org/jira/browse/DRILL-5028 > Project: Apache Drill > Issue Type: Improvement > Components: Functions - Drill >Affects Versions: 1.8.0 >Reporter: Hongze Zhang >Priority: Minor > Fix For: Future > > > We have a Drill cluster with 20+ Nodes and we store all history profiles in > hdfs. Without doing periodically cleans for hdfs, the profiles page gets > slower while serving more queries. > Code from LocalPersistentStore.java uses fs.list(false, basePath) for > fetching the latest 100 history profiles by default, I guess this operation > blocks the page loading (Millions small files can be stored in the basePath), > maybe we can try some other ways to reach the same goal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-5054) Provide jquery-ui and jquery-dataTables locally
[ https://issues.apache.org/jira/browse/DRILL-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang closed DRILL-5054. --- Resolution: Later > Provide jquery-ui and jquery-dataTables locally > --- > > Key: DRILL-5054 > URL: https://issues.apache.org/jira/browse/DRILL-5054 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 / FireFox 50.0 >Reporter: Hongze Zhang >Priority: Minor > Fix For: Future > > > Hi, > Currently Drill uses CDN for serving source files of jquery-ui and > jquery-dataTables. This is OK for most cases, but not working in an isolated > environment. > This is a patch adding these files so that Drill will work fine in intranet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang reassigned DRILL-5068: --- Assignee: Hongze Zhang > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5068: Assignee: (was: Sudheesh Katkam) Reviewer: Khurram Faraaz > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5068: Assignee: Sudheesh Katkam Component/s: (was: Metadata) Storage - Information Schema > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Information Schema >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang >Assignee: Sudheesh Katkam > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5068: Description: Hi, Currently the profile page on UI is still not detailed enough for some complicated uses (eg. show all failed queries during these three days), we can only access latest 100 query profiles on this page. We may sometimes need a specific system table for querying completed profiles. was: Hi, Currently the profile page on UI is still not detailed enough for some uses more complicated (eg. show all failed queries during these three days), we can only access latest 100 query profiles on this page. We may sometimes need a specific system table for querying completed profiles. > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Metadata >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some > complicated uses (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5068: Flags: Patch Environment: Fedora 25 OpenJDK 8 Firefox 50.0 Description: Hi, Currently the profile page on UI is still not detailed enough for some uses more complicated (eg. show all failed queries during these three days), we can only access latest 100 query profiles on this page. We may sometimes need a specific system table for querying completed profiles. was: Hi, Currently the profile page on UI is still not very detailed, we can only access latest 100 query profiles on this page. We may need a specific system table to query completed profiles. > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Metadata >Affects Versions: 1.8.0 > Environment: Fedora 25 > OpenJDK 8 > Firefox 50.0 >Reporter: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not detailed enough for some uses > more complicated (eg. show all failed queries during these three days), we > can only access latest 100 query profiles on this page. > We may sometimes need a specific system table for querying completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5051) DRILL-5051: Fix incorrect result returned in nest query with offset specified
[ https://issues.apache.org/jira/browse/DRILL-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5051: Summary: DRILL-5051: Fix incorrect result returned in nest query with offset specified (was: Fix incorrect computation of 'fetch' in LimitRecordBatch when 'offset' is specified) > DRILL-5051: Fix incorrect result returned in nest query with offset specified > - > > Key: DRILL-5051 > URL: https://issues.apache.org/jira/browse/DRILL-5051 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Labels: ready-to-commit > Fix For: Future > > > My SQl: > select count(1) from (select id from (select id from > cp.`tpch/lineitem.parquet` LIMIT 2) limit 1 offset 1) > This SQL returns nothing. > Something goes wrong in LimitRecordBatch.java, and the reason is different > with [DRILL-4884|https://issues.apache.org/jira/browse/DRILL-4884?filter=-2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5051) Fix incorrect computation of 'fetch' in LimitRecordBatch when 'offset' is specified
[ https://issues.apache.org/jira/browse/DRILL-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5051: Summary: Fix incorrect computation of 'fetch' in LimitRecordBatch when 'offset' is specified (was: DRILL-5051: Fix incorrect computation of 'fetch' in LimitRecordBatch when 'offset' is specified) > Fix incorrect computation of 'fetch' in LimitRecordBatch when 'offset' is > specified > --- > > Key: DRILL-5051 > URL: https://issues.apache.org/jira/browse/DRILL-5051 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Labels: ready-to-commit > Fix For: Future > > > My SQl: > select count(1) from (select id from (select id from > cp.`tpch/lineitem.parquet` LIMIT 2) limit 1 offset 1) > This SQL returns nothing. > Something goes wrong in LimitRecordBatch.java, and the reason is different > with [DRILL-4884|https://issues.apache.org/jira/browse/DRILL-4884?filter=-2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5051) DRILL-5051: Fix incorrect computation of 'fetch' in LimitRecordBatch when 'offset' is specified
[ https://issues.apache.org/jira/browse/DRILL-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5051: Summary: DRILL-5051: Fix incorrect computation of 'fetch' in LimitRecordBatch when 'offset' is specified (was: Returning incorrect number of rows while querying using both nested select and offset) > DRILL-5051: Fix incorrect computation of 'fetch' in LimitRecordBatch when > 'offset' is specified > --- > > Key: DRILL-5051 > URL: https://issues.apache.org/jira/browse/DRILL-5051 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 >Reporter: Hongze Zhang >Assignee: Hongze Zhang > Labels: ready-to-commit > Fix For: Future > > > My SQl: > select count(1) from (select id from (select id from > cp.`tpch/lineitem.parquet` LIMIT 2) limit 1 offset 1) > This SQL returns nothing. > Something goes wrong in LimitRecordBatch.java, and the reason is different > with [DRILL-4884|https://issues.apache.org/jira/browse/DRILL-4884?filter=-2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5068: Issue Type: Improvement (was: New Feature) > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: Improvement > Components: Metadata >Affects Versions: 1.8.0 >Reporter: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not very detailed, we can only > access latest 100 query profiles on this page. > We may need a specific system table to query completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5054) Provide jquery-ui and jquery-dataTables locally
[ https://issues.apache.org/jira/browse/DRILL-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5054: Priority: Minor (was: Major) > Provide jquery-ui and jquery-dataTables locally > --- > > Key: DRILL-5054 > URL: https://issues.apache.org/jira/browse/DRILL-5054 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 / FireFox 50.0 >Reporter: Hongze Zhang >Priority: Minor > Fix For: Future > > > Hi, > Currently Drill uses CDN for serving source files of jquery-ui and > jquery-dataTables. This is OK for most cases, but not working in an isolated > environment. > This is a patch adding these files so that Drill will work fine in intranet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5068) Add a new system table for completed profiles
[ https://issues.apache.org/jira/browse/DRILL-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5068: Description: Hi, Currently the profile page on UI is still not very detailed, we can only access latest 100 query profiles on this page. We may need a specific system table to query completed profiles. was: Hi, Currently the profile page on UI is still not very detailed, we can only access latest 100 query profiles on this page. For more complicated use case, like checking the failure queries number, the more earlier queries before latest 100, queries from a specific user, we can add SQL support to the history profiles. This will benefit Drill cluster maintenance. > Add a new system table for completed profiles > - > > Key: DRILL-5068 > URL: https://issues.apache.org/jira/browse/DRILL-5068 > Project: Apache Drill > Issue Type: New Feature > Components: Metadata >Affects Versions: 1.8.0 >Reporter: Hongze Zhang > Fix For: Future > > > Hi, > Currently the profile page on UI is still not very detailed, we can only > access latest 100 query profiles on this page. > We may need a specific system table to query completed profiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-5068) Add a new system table for completed profiles
Hongze Zhang created DRILL-5068: --- Summary: Add a new system table for completed profiles Key: DRILL-5068 URL: https://issues.apache.org/jira/browse/DRILL-5068 Project: Apache Drill Issue Type: New Feature Components: Metadata Affects Versions: 1.8.0 Reporter: Hongze Zhang Fix For: Future Hi, Currently the profile page on UI is still not very detailed, we can only access latest 100 query profiles on this page. For more complicated use case, like checking the failure queries number, the more earlier queries before latest 100, queries from a specific user, we can add SQL support to the history profiles. This will benefit Drill cluster maintenance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5028) Opening profiles page from web ui gets very slow when a lot of history files have been stored in HDFS or Local FS.
[ https://issues.apache.org/jira/browse/DRILL-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5028: Flags: (was: Patch) > Opening profiles page from web ui gets very slow when a lot of history files > have been stored in HDFS or Local FS. > -- > > Key: DRILL-5028 > URL: https://issues.apache.org/jira/browse/DRILL-5028 > Project: Apache Drill > Issue Type: Improvement > Components: Functions - Drill >Affects Versions: 1.8.0 >Reporter: Hongze Zhang > Fix For: Future > > > We have a Drill cluster with 20+ Nodes and we store all history profiles in > hdfs. Without doing periodically cleans for hdfs, the profiles page gets > slower while serving more queries. > Code from LocalPersistentStore.java uses fs.list(false, basePath) for > fetching the latest 100 history profiles by default, I guess this operation > blocks the page loading (Millions small files can be stored in the basePath), > maybe we can try some other ways to reach the same goal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5054) Provide jquery-ui and jquery-dataTables locally
[ https://issues.apache.org/jira/browse/DRILL-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5054: Description: Hi, Currently Drill uses CDN for serving source files of jquery-ui and jquery-dataTables. This is OK for most cases, but not working in an isolated environment. This is a patch adding these files so that Drill will work fine in intranet. was: Hi, Currently Drill uses CDN for serving source files of jquery-ui and jquery-dataTables. This is OK for most cases, but not working in an isolated environment. Adding these files so that Drill will work fine in intranet. > Provide jquery-ui and jquery-dataTables locally > --- > > Key: DRILL-5054 > URL: https://issues.apache.org/jira/browse/DRILL-5054 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 / FireFox 50.0 >Reporter: Hongze Zhang > Fix For: Future > > > Hi, > Currently Drill uses CDN for serving source files of jquery-ui and > jquery-dataTables. This is OK for most cases, but not working in an isolated > environment. > This is a patch adding these files so that Drill will work fine in intranet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-5054) Provide jquery-ui and jquery-dataTables locally
Hongze Zhang created DRILL-5054: --- Summary: Provide jquery-ui and jquery-dataTables locally Key: DRILL-5054 URL: https://issues.apache.org/jira/browse/DRILL-5054 Project: Apache Drill Issue Type: Improvement Components: Web Server Affects Versions: 1.8.0 Environment: Fedora 24 / OpenJDK 8 / FireFox 50.0 Reporter: Hongze Zhang Fix For: Future Hi, Currently Drill uses CDN for serving source files of jquery-ui and jquery-dataTables. This is OK for most cases, but not working in an isolated environment. Adding these files so that Drill will work fine in intranet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (DRILL-5051) Returning incorrect number of rows while querying using both nested select and offset
[ https://issues.apache.org/jira/browse/DRILL-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676648#comment-15676648 ] Hongze Zhang edited comment on DRILL-5051 at 11/18/16 4:28 PM: --- Hi Khurram, I personally guess that the method *void limitWithSV(int recordCount)* has a little problem, although I still don't understand why the author made different behaviour in two limit methods (limitWithSV and limitWithNoSV). I did a quick fix in [https://github.com/apache/drill/pull/658] including merging two methods. You can help check if the fix is appropriate. was (Author: zbdzzg): Hi Khurram, I personally guess that the method *void limitWithSV(int recordCount)* has a little problem, although I still don't understand why the author made different behavior in two limit methods (limitWithSV and limitWithNoSV). I did a quick fix in [https://github.com/apache/drill/pull/658] including merging two methods. You can help check if the fix is appropriate. > Returning incorrect number of rows while querying using both nested select > and offset > - > > Key: DRILL-5051 > URL: https://issues.apache.org/jira/browse/DRILL-5051 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 >Reporter: Hongze Zhang >Assignee: Sudheesh Katkam > Fix For: Future > > > My SQl: > select count(1) from (select id from (select id from > cp.`tpch/lineitem.parquet` LIMIT 2) limit 1 offset 1) > This SQL returns nothing. > Something goes wrong in LimitRecordBatch.java, and the reason is different > with [DRILL-4884|https://issues.apache.org/jira/browse/DRILL-4884?filter=-2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (DRILL-5051) Returning incorrect number of rows while querying using both nested select and offset
[ https://issues.apache.org/jira/browse/DRILL-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676648#comment-15676648 ] Hongze Zhang edited comment on DRILL-5051 at 11/18/16 4:27 PM: --- Hi Khurram, I personally guess that the method *void limitWithSV(int recordCount)* has a little problem, although I still don't understand why the author made different behavior in two limit methods (limitWithSV and limitWithNoSV). I did a quick fix in [https://github.com/apache/drill/pull/658] including merging two methods. You can help check if the fix is appropriate. was (Author: zbdzzg): Hi Khurram, I personally guess that the method *void limitWithSV(int recordCount)* has a little problem, although I still don't understand why the author made different behaviors in two limit methods (limitWithSV and limitWithNoSV). I did a quick fix in [https://github.com/apache/drill/pull/658] including merging two methods. You can help check if the fix is appropriate. > Returning incorrect number of rows while querying using both nested select > and offset > - > > Key: DRILL-5051 > URL: https://issues.apache.org/jira/browse/DRILL-5051 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 >Reporter: Hongze Zhang >Assignee: Sudheesh Katkam > Fix For: Future > > > My SQl: > select count(1) from (select id from (select id from > cp.`tpch/lineitem.parquet` LIMIT 2) limit 1 offset 1) > This SQL returns nothing. > Something goes wrong in LimitRecordBatch.java, and the reason is different > with [DRILL-4884|https://issues.apache.org/jira/browse/DRILL-4884?filter=-2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (DRILL-5051) Returning incorrect number of rows while querying using both nested select and offset
[ https://issues.apache.org/jira/browse/DRILL-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676648#comment-15676648 ] Hongze Zhang edited comment on DRILL-5051 at 11/18/16 4:27 PM: --- Hi Khurram, I personally guess that the method *void limitWithSV(int recordCount)* has a little problem, although I still don't understand why the author made different behaviors in two limit methods (limitWithSV and limitWithNoSV). I did a quick fix in [https://github.com/apache/drill/pull/658] including merging two methods. You can help check if the fix is appropriate. was (Author: zbdzzg): Hi Khurram, I personally guess that the method *void limitWithSV(int recordCount)* has a little problem, although I still don't understand why the author made different behavior in two limit methods (limitWithSV and limitWithNoSV). I did a quick fix in [https://github.com/apache/drill/pull/658] including merging two methods. You can help check if the fix is appropriate. > Returning incorrect number of rows while querying using both nested select > and offset > - > > Key: DRILL-5051 > URL: https://issues.apache.org/jira/browse/DRILL-5051 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 >Reporter: Hongze Zhang >Assignee: Sudheesh Katkam > Fix For: Future > > > My SQl: > select count(1) from (select id from (select id from > cp.`tpch/lineitem.parquet` LIMIT 2) limit 1 offset 1) > This SQL returns nothing. > Something goes wrong in LimitRecordBatch.java, and the reason is different > with [DRILL-4884|https://issues.apache.org/jira/browse/DRILL-4884?filter=-2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-5051) Returning incorrect number of rows while querying using both nested select and offset
[ https://issues.apache.org/jira/browse/DRILL-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676648#comment-15676648 ] Hongze Zhang commented on DRILL-5051: - Hi Khurram, I personally guess that the method *void limitWithSV(int recordCount)* has a little problem, although I still don't understand why the author made different behavior in two limit methods (limitWithSV and limitWithNoSV). I did a quick fix in [https://github.com/apache/drill/pull/658] including merging two methods. You can help check if the fix is appropriate. > Returning incorrect number of rows while querying using both nested select > and offset > - > > Key: DRILL-5051 > URL: https://issues.apache.org/jira/browse/DRILL-5051 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.8.0 > Environment: Fedora 24 / OpenJDK 8 >Reporter: Hongze Zhang > Fix For: Future > > > My SQl: > select count(1) from (select id from (select id from > cp.`tpch/lineitem.parquet` LIMIT 2) limit 1 offset 1) > This SQL returns nothing. > Something goes wrong in LimitRecordBatch.java, and the reason is different > with [DRILL-4884|https://issues.apache.org/jira/browse/DRILL-4884?filter=-2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-5051) Returning incorrect number of rows while querying using both nested select and offset
Hongze Zhang created DRILL-5051: --- Summary: Returning incorrect number of rows while querying using both nested select and offset Key: DRILL-5051 URL: https://issues.apache.org/jira/browse/DRILL-5051 Project: Apache Drill Issue Type: Bug Components: Query Planning & Optimization Affects Versions: 1.8.0 Environment: Fedora 24 / OpenJDK 8 Reporter: Hongze Zhang Fix For: Future My SQl: select count(1) from (select id from (select id from cp.`tpch/lineitem.parquet` LIMIT 2) limit 1 offset 1) This SQL returns nothing. Something goes wrong in LimitRecordBatch.java, and the reason is different with [DRILL-4884|https://issues.apache.org/jira/browse/DRILL-4884?filter=-2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-5028) Opening profiles page from web ui gets very slow when a lot of history files have been stored in HDFS or Local FS.
[ https://issues.apache.org/jira/browse/DRILL-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated DRILL-5028: Description: We have a Drill cluster with 20+ Nodes and we store all history profiles in hdfs. Without doing periodically cleans for hdfs, the profiles page gets slower while serving more queries. Code from LocalPersistentStore.java uses fs.list(false, basePath) for fetching the latest 100 history profiles by default, I guess this operation blocks the page loading (Millions small files can be stored in the basePath), maybe we can try some other ways to reach the same goal. was: We have a Drill cluster with 20+ Nodes and we store all history profiles in hdfs. Without doing periodically cleans for hdfs, the profiles page gets slower while serving more queries. Code from LocalPersistentStore.java uses fs.list(false, basePath) for fetching the latest 100 history profiles by default, I guess this operation blocks that page (Millions small files can be stored in the basePath), maybe we can try some other ways to reach the same goal. > Opening profiles page from web ui gets very slow when a lot of history files > have been stored in HDFS or Local FS. > -- > > Key: DRILL-5028 > URL: https://issues.apache.org/jira/browse/DRILL-5028 > Project: Apache Drill > Issue Type: Improvement > Components: Functions - Drill >Affects Versions: 1.8.0 >Reporter: Hongze Zhang > Fix For: Future > > > We have a Drill cluster with 20+ Nodes and we store all history profiles in > hdfs. Without doing periodically cleans for hdfs, the profiles page gets > slower while serving more queries. > Code from LocalPersistentStore.java uses fs.list(false, basePath) for > fetching the latest 100 history profiles by default, I guess this operation > blocks the page loading (Millions small files can be stored in the basePath), > maybe we can try some other ways to reach the same goal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-5028) Opening profiles page from web ui gets very slow when a lot of history files have been stored in HDFS or Local FS.
Hongze Zhang created DRILL-5028: --- Summary: Opening profiles page from web ui gets very slow when a lot of history files have been stored in HDFS or Local FS. Key: DRILL-5028 URL: https://issues.apache.org/jira/browse/DRILL-5028 Project: Apache Drill Issue Type: Improvement Components: Functions - Drill Affects Versions: 1.8.0 Reporter: Hongze Zhang Fix For: Future We have a Drill cluster with 20+ Nodes and we store all history profiles in hdfs. Without doing periodically cleans for hdfs, the profiles page gets slower while serving more queries. Code from LocalPersistentStore.java uses fs.list(false, basePath) for fetching the latest 100 history profiles by default, I guess this operation blocks that page (Millions small files can be stored in the basePath), maybe we can try some other ways to reach the same goal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4884) Fix IOB exception in limit n query when n is beyond 65535.
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Summary: Fix IOB exception in limit n query when n is beyond 65535. (was:
Drill produced IOB exception while querying data of 65536 limitation using non
batched reader)
> Fix IOB exception in limit n query when n is beyond 65535.
> --
>
> Key: DRILL-4884
> URL: https://issues.apache.org/jira/browse/DRILL-4884
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.8.0
> Environment: CentOS 6.5 / JAVA 8
>Reporter: Hongze Zhang
> Fix For: 1.9.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Drill produces IOB while using a non batched scanner and limiting SQL by
> 65536.
> SQL:
> {noformat}
> select id from xx limit 1 offset 65535
> {noformat}
> Result:
> {noformat}
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [classes/:na]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [classes/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
> (expected: range(0, 131072))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
> ~[classes/:4.0.27.Final]
> at
> org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[classes/:na]
> at
>
[jira] [Updated] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Component/s: (was: Functions - Drill)
Query Planning & Optimization
> Drill produced IOB exception while querying data of 65536 limitation using
> non batched reader
> -
>
> Key: DRILL-4884
> URL: https://issues.apache.org/jira/browse/DRILL-4884
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.8.0
> Environment: CentOS 6.5 / JAVA 8
>Reporter: Hongze Zhang
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Drill produces IOB while using a non batched scanner and limiting SQL by
> 65536.
> SQL:
> {noformat}
> select id from xx limit 1 offset 65535
> {noformat}
> Result:
> {noformat}
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [classes/:na]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [classes/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
> (expected: range(0, 131072))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
> ~[classes/:4.0.27.Final]
> at
> org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
> ~[classes/:na]
>
[jira] [Updated] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Reviewer: Daniel Barclay
> Drill produced IOB exception while querying data of 65536 limitation using
> non batched reader
> -
>
> Key: DRILL-4884
> URL: https://issues.apache.org/jira/browse/DRILL-4884
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.8.0
> Environment: CentOS 6.5 / JAVA 8
>Reporter: Hongze Zhang
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Drill produces IOB while using a non batched scanner and limiting SQL by
> 65536.
> SQL:
> {noformat}
> select id from xx limit 1 offset 65535
> {noformat}
> Result:
> {noformat}
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [classes/:na]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [classes/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
> (expected: range(0, 131072))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
> ~[classes/:4.0.27.Final]
> at
> org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
> ~[classes/:na]
> at
>
[jira] [Updated] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Description:
Drill produces IOB while using a non batched scanner and limiting SQL by 65536.
SQL:
{noformat}
select id from xx limit 1 offset 65535
{noformat}
Result:
{noformat}
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
[classes/:na]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[classes/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
(expected: range(0, 131072))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
~[classes/:4.0.27.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
~[classes/:4.0.27.Final]
at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
~[classes/:4.0.27.Final]
at
org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250)
~[classes/:na]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_101]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_101]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
~[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:250)
[classes/:na]
... 4 common frames omitted
{noformat}
Code from
[jira] [Updated] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Description:
Drill produces IOB while using a non batched scanner and limiting SQL by 65536.
SQL:
{noformat}
select id from xx limit 1 offset 65535
{noformat}
Result:
{noformat}
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
[classes/:na]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[classes/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
(expected: range(0, 131072))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
~[classes/:4.0.27.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
~[classes/:4.0.27.Final]
at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
~[classes/:4.0.27.Final]
at
org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250)
~[classes/:na]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_101]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_101]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
~[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:250)
[classes/:na]
... 4 common frames omitted
{noformat}
Code from
[jira] [Updated] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Description:
Drill produces IOB while using a non batched scanner and limiting SQL by 65536.
SQL:
{noformat}
select id from isearch.tmall_auction_cluster limit 1 offset 65535
{noformat}
Result:
{noformat}
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
[classes/:na]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[classes/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
(expected: range(0, 131072))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
~[classes/:4.0.27.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
~[classes/:4.0.27.Final]
at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
~[classes/:4.0.27.Final]
at
org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250)
~[classes/:na]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_101]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_101]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
~[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:250)
[classes/:na]
... 4 common frames omitted
[jira] [Updated] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
[
https://issues.apache.org/jira/browse/DRILL-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang updated DRILL-4884:
Remaining Estimate: 168h (was: 2h)
Original Estimate: 168h (was: 2h)
> Drill produced IOB exception while querying data of 65536 limitation using
> non batched reader
> -
>
> Key: DRILL-4884
> URL: https://issues.apache.org/jira/browse/DRILL-4884
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.8.0
> Environment: CentOS 6.5 / JAVA 8
>Reporter: Hongze Zhang
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Drill produces IOB while using a non batched scanner and limiting SQL by
> 65536.
> SQL:
> {noformat}
> select id from isearch.tmall_auction_cluster limit 1 offset 65535
> {noformat}
> Result:
> {noformat}
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [classes/:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [classes/:na]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [classes/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
> (expected: range(0, 131072))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
> ~[classes/:4.0.27.Final]
> at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
> ~[classes/:4.0.27.Final]
> at
> org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[classes/:na]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[classes/:na]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
> ~[classes/:na]
[jira] [Created] (DRILL-4884) Drill produced IOB exception while querying data of 65536 limitation using non batched reader
Hongze Zhang created DRILL-4884:
---
Summary: Drill produced IOB exception while querying data of 65536
limitation using non batched reader
Key: DRILL-4884
URL: https://issues.apache.org/jira/browse/DRILL-4884
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 1.8.0
Environment: CentOS 6.5 / JAVA 8
Reporter: Hongze Zhang
Drill produces IOB while using a non batched scanner and limiting SQL by 65536.
SQL:
{noformat}
select id from isearch.tmall_auction_cluster limit 1 offset 65535
{noformat}
Result:
{noformat}
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:324)
[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
[classes/:na]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[classes/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Caused by: java.lang.IndexOutOfBoundsException: index: 131072, length: 2
(expected: range(0, 131072))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:175)
~[classes/:4.0.27.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:197)
~[classes/:4.0.27.Final]
at io.netty.buffer.DrillBuf.setChar(DrillBuf.java:517)
~[classes/:4.0.27.Final]
at
org.apache.drill.exec.record.selection.SelectionVector2.setIndex(SelectionVector2.java:79)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.limitWithNoSV(LimitRecordBatch.java:167)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.doWork(LimitRecordBatch.java:145)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:115)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
~[classes/:na]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
~[classes/:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250)
~[classes/:na]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_101]
at javax.security.auth.Subject.doAs(Subject.java:422)
