[jira] [Commented] (DRILL-7020) big varchar doesn't work with extractHeader=true
[
https://issues.apache.org/jira/browse/DRILL-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821571#comment-16821571
]
Paul Rogers commented on DRILL-7020:
The "write" part of the message means "write to the value vector", so read this
as "tried to write too large a value to the column's value vector."
The request here seems to be to modify the "compliant" text reader to avoid the
use of the fixed-size buffer for column values in order to allow column values
larger than 64K.
The simple way to do this is to use a Java string for the value, or to
reallocate the column buffer as needed for ever larger values.
A more elegant way, now that the compliant reader is on the row set framework,
is to implement an "append" operation which will take a buffer and append it to
the value (if any) already in the column. This will allow reading large values
without having to allocate large intermediate buffers.
> big varchar doesn't work with extractHeader=true
>
>
> Key: DRILL-7020
> URL: https://issues.apache.org/jira/browse/DRILL-7020
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Text CSV
>Affects Versions: 1.15.0
>Reporter: benj
>Priority: Major
>
> with a TEST file of csv type like
> {code:java}
> col1,col2
> w,x
> ...y...,z
> {code}
> where ...y... is > 65536 characters string (let say 66000 for example)
> SELECT with +*extractHeader=false*+ are OK
> {code:java}
> SELECT * FROM TABLE(tmp.`TEST`(type => 'text', fieldDelimiter => ',',
> extractHeader => false));
> col1 | col2
> +-+--
> | w | x
> | ...y... | z
> {code}
> But SELECT with +*extractHeader=true*+ gives an error
> {code:java}
> SELECT * FROM TABLE(tmp.`TEST`(type => 'text', fieldDelimiter => ',',
> extractHeader => true));
> Error: UNSUPPORTED_OPERATION ERROR: Trying to write something big in a column
> columnIndex 1
> Limit 65536
> Fragment 0:0
> {code}
> Note that is possible to use extractHeader=false with skipFirstLine=true but
> in this case it's not possible to automatically get columns names.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6879) Indicate a warning in the WebUI when a query makes little to no progress for a while
[
https://issues.apache.org/jira/browse/DRILL-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821517#comment-16821517
]
Kunal Khatua edited comment on DRILL-6879 at 4/18/19 9:53 PM:
--
Hi Bridget,
The content looks good, but the table in
[https://drill.apache.org/docs/query-profiles/#query-profile-warnings] is
unusually squeezed for the *Icon* column.
Can you see if something like suffixing {{ }} to the *Icon* column header helps?
(Ref: https://stackoverflow.com/a/44555965)
was (Author: kkhatua):
Hi Bridget,
The content looks good, but the table in
[https://drill.apache.org/docs/query-profiles/#query-profile-warnings] is
unusually squeezed for the *Icon* column.
Can you see if something like suffixing {{}} to the *Icon* column header
helps?
> Indicate a warning in the WebUI when a query makes little to no progress for
> a while
>
>
> Key: DRILL-6879
> URL: https://issues.apache.org/jira/browse/DRILL-6879
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Monitoring, Web Server
>Affects Versions: 1.14.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Major
> Labels: doc-complete, ready-to-commit
> Fix For: 1.16.0
>
> Attachments: image-2018-12-04-11-54-54-247.png,
> image-2018-12-06-11-19-00-339.png, image-2018-12-06-11-27-14-719.png
>
>
> When running a very large query on a cluster with limited resource, we
> noticed that one of the node's VM thread freezes the fragment threads as it
> tries to do some work (GC perhaps?). This is a clear indication that the
> query is stuck in a weird state where it might not recover from.
> Under such circumstances, it makes sense to cancel or atleast warn the user
> on that page of the query exceeding a certain threshold.
> For detecting this, the user will find that the {{Last Progress}} column in
> the Fragments Overview section will show large times.
> !image-2018-12-04-11-54-54-247.png|width=969,height=336!
> In addition, there are instances where a query might have buffered operators
> spilling to disk, which also hits performance (and, subsequently, longer run
> times). Calling out this skew can be very useful.
> !image-2018-12-06-11-27-14-719.png|width=969,height=256!
> Or there might be cases where a single fragment takes much longer than the
> average (indicated by an extreme skew in the Gantt chart).
> !image-2018-12-06-11-19-00-339.png|width=969,height=150!
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6879) Indicate a warning in the WebUI when a query makes little to no progress for a while
[
https://issues.apache.org/jira/browse/DRILL-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821517#comment-16821517
]
Kunal Khatua commented on DRILL-6879:
-
Hi Bridget,
The content looks good, but the table in
[https://drill.apache.org/docs/query-profiles/#query-profile-warnings] is
unusually squeezed for the *Icon* column.
Can you see if something like suffixing {{}} to the *Icon* column header
helps?
> Indicate a warning in the WebUI when a query makes little to no progress for
> a while
>
>
> Key: DRILL-6879
> URL: https://issues.apache.org/jira/browse/DRILL-6879
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Monitoring, Web Server
>Affects Versions: 1.14.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Major
> Labels: doc-complete, ready-to-commit
> Fix For: 1.16.0
>
> Attachments: image-2018-12-04-11-54-54-247.png,
> image-2018-12-06-11-19-00-339.png, image-2018-12-06-11-27-14-719.png
>
>
> When running a very large query on a cluster with limited resource, we
> noticed that one of the node's VM thread freezes the fragment threads as it
> tries to do some work (GC perhaps?). This is a clear indication that the
> query is stuck in a weird state where it might not recover from.
> Under such circumstances, it makes sense to cancel or atleast warn the user
> on that page of the query exceeding a certain threshold.
> For detecting this, the user will find that the {{Last Progress}} column in
> the Fragments Overview section will show large times.
> !image-2018-12-04-11-54-54-247.png|width=969,height=336!
> In addition, there are instances where a query might have buffered operators
> spilling to disk, which also hits performance (and, subsequently, longer run
> times). Calling out this skew can be very useful.
> !image-2018-12-06-11-27-14-719.png|width=969,height=256!
> Or there might be cases where a single fragment takes much longer than the
> average (indicated by an extreme skew in the Gantt chart).
> !image-2018-12-06-11-19-00-339.png|width=969,height=150!
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6939) Indicate when a query is submitted and is in progress
[ https://issues.apache.org/jira/browse/DRILL-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821511#comment-16821511 ] Kunal Khatua commented on DRILL-6939: - LGTM. THanks! > Indicate when a query is submitted and is in progress > - > > Key: DRILL-6939 > URL: https://issues.apache.org/jira/browse/DRILL-6939 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.14.0 >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Critical > Labels: doc-complete, ready-to-commit, user-experience > Fix For: 1.16.0 > > > When submitting a long running query, the web UI shows no indication of the > query having been submitted. What is needed is some form of UI enhancement > that shows that the submitted query is in progress and the results will load > when available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6921) Add button to reset Options filter
[ https://issues.apache.org/jira/browse/DRILL-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821498#comment-16821498 ] Kunal Khatua commented on DRILL-6921: - LGTM. Thanks! > Add button to reset Options filter > -- > > Key: DRILL-6921 > URL: https://issues.apache.org/jira/browse/DRILL-6921 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.15.0 >Reporter: Arina Ielchiieva >Assignee: Kunal Khatua >Priority: Major > Labels: doc-complete, ready-to-commit > Fix For: 1.16.0 > > > Currently we have ability to search options or use quick filter on Web UI. To > reset the filter, user needs to delete input from the search pane manually. > It would be nice if we had Reset button. > Also we can consider leaving filter after option update / reset rather then > reloading page without filtering. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6050) Provide a limit to number of rows fetched for a query in UI
[ https://issues.apache.org/jira/browse/DRILL-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821470#comment-16821470 ] Kunal Khatua edited comment on DRILL-6050 at 4/18/19 9:23 PM: -- All looks good except the 3rd link. [https://drill.apache.org/docs/rest-api-introduction/#post-query-json] The *Request Body* example is missing the autolimit key-value. This is, however, normal since the parameter is optional. Do you want to call it out specifically? was (Author: kkhatua): All looks good except the 3rd link. [https://drill.apache.org/docs/rest-api-introduction/#post-query-json] The *Request Body* example is missing the autolimit key-value. This is, however, normal. > Provide a limit to number of rows fetched for a query in UI > --- > > Key: DRILL-6050 > URL: https://issues.apache.org/jira/browse/DRILL-6050 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Minor > Labels: doc-complete, ready-to-commit, user-experience > Fix For: 1.16.0, 1.17.0 > > > Currently, the WebServer side needs to process the entire set of results and > stream it back to the WebClient. > Since the WebUI does paginate results, we can load a larger set for > pagination on the browser client and relieve pressure off the WebServer to > host all the data. > e.g. Fetching all rows from a 1Billion records table is impractical and can > be capped at 10K. Currently, the user has to explicitly specify LIMIT in the > submitted query. > An option can be provided in the field to allow for this entry. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6050) Provide a limit to number of rows fetched for a query in UI
[ https://issues.apache.org/jira/browse/DRILL-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821470#comment-16821470 ] Kunal Khatua commented on DRILL-6050: - All looks good except the 3rd link. [https://drill.apache.org/docs/rest-api-introduction/#post-query-json] The *Request Body* example is missing the autolimit key-value. This is, however, normal. > Provide a limit to number of rows fetched for a query in UI > --- > > Key: DRILL-6050 > URL: https://issues.apache.org/jira/browse/DRILL-6050 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Minor > Labels: doc-complete, ready-to-commit, user-experience > Fix For: 1.16.0, 1.17.0 > > > Currently, the WebServer side needs to process the entire set of results and > stream it back to the WebClient. > Since the WebUI does paginate results, we can load a larger set for > pagination on the browser client and relieve pressure off the WebServer to > host all the data. > e.g. Fetching all rows from a 1Billion records table is impractical and can > be capped at 10K. Currently, the user has to explicitly specify LIMIT in the > submitted query. > An option can be provided in the field to allow for this entry. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-5509) Upgrade Drill protobuf support from 2.5.0 to latest 3.3
[ https://issues.apache.org/jira/browse/DRILL-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821469#comment-16821469 ] Sorabh Hamirwasia commented on DRILL-5509: -- Protobuf upgrade is reverted in 1.16 and is planned for 1.17 release > Upgrade Drill protobuf support from 2.5.0 to latest 3.3 > --- > > Key: DRILL-5509 > URL: https://issues.apache.org/jira/browse/DRILL-5509 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.10.0 >Reporter: Paul Rogers >Assignee: Anton Gozhiy >Priority: Minor > Fix For: 1.16.0 > > > Drill uses Google Protobufs for RPC. Drill's Maven compile requires version > 2.5.0 from Feb. 2013. The latest version is 3.3. Over time, it may become > increasingly hard to find and build a four-year-old version. > Upgrade Drill to use the latest Protobuf version. This will require updating > the Maven protobuf plugin, and may require other upgrades as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6050) Provide a limit to number of rows fetched for a query in UI
[ https://issues.apache.org/jira/browse/DRILL-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821461#comment-16821461 ] Bridget Bevens commented on DRILL-6050: --- Hi [~kkhatua], I added the info about autoLimit to the following pages: https://drill.apache.org/docs/planning-and-execution-options/#setting-an-auto-limit-on-the-number-of-rows-returned-for-result-sets Included a note in the following section about REST client config: https://drill.apache.org/docs/planning-and-execution-options/#drill-web-ui-row-limit-settings Added the autoLimit parameter and example here: https://drill.apache.org/docs/rest-api-introduction/#post-query-json Added a note to the Limit clause: https://drill.apache.org/docs/limit-clause/ Please let me know if I need to change anything. Thanks, Bridget > Provide a limit to number of rows fetched for a query in UI > --- > > Key: DRILL-6050 > URL: https://issues.apache.org/jira/browse/DRILL-6050 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Minor > Labels: doc-impacting, ready-to-commit, user-experience > Fix For: 1.16.0, 1.17.0 > > > Currently, the WebServer side needs to process the entire set of results and > stream it back to the WebClient. > Since the WebUI does paginate results, we can load a larger set for > pagination on the browser client and relieve pressure off the WebServer to > host all the data. > e.g. Fetching all rows from a 1Billion records table is impractical and can > be capped at 10K. Currently, the user has to explicitly specify LIMIT in the > submitted query. > An option can be provided in the field to allow for this entry. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6050) Provide a limit to number of rows fetched for a query in UI
[ https://issues.apache.org/jira/browse/DRILL-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens updated DRILL-6050: -- Labels: doc-complete ready-to-commit user-experience (was: doc-impacting ready-to-commit user-experience) > Provide a limit to number of rows fetched for a query in UI > --- > > Key: DRILL-6050 > URL: https://issues.apache.org/jira/browse/DRILL-6050 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Minor > Labels: doc-complete, ready-to-commit, user-experience > Fix For: 1.16.0, 1.17.0 > > > Currently, the WebServer side needs to process the entire set of results and > stream it back to the WebClient. > Since the WebUI does paginate results, we can load a larger set for > pagination on the browser client and relieve pressure off the WebServer to > host all the data. > e.g. Fetching all rows from a 1Billion records table is impractical and can > be capped at 10K. Currently, the user has to explicitly specify LIMIT in the > submitted query. > An option can be provided in the field to allow for this entry. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6562) Plugin Management improvements
[ https://issues.apache.org/jira/browse/DRILL-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens updated DRILL-6562: -- Labels: doc-complete ready-to-commit (was: doc-impacting ready-to-commit) > Plugin Management improvements > -- > > Key: DRILL-6562 > URL: https://issues.apache.org/jira/browse/DRILL-6562 > Project: Apache Drill > Issue Type: Improvement > Components: Client - HTTP, Web Server >Affects Versions: 1.14.0 >Reporter: Abhishek Girish >Assignee: Vitalii Diravka >Priority: Major > Labels: doc-complete, ready-to-commit > Fix For: 1.16.0 > > Attachments: Export.png, ExportAll.png, Screenshot from 2019-03-21 > 01-18-17.png, Screenshot from 2019-03-21 02-52-50.png, Storage.png, > UpdateExport.png, create.png, image-2018-07-23-02-55-02-024.png, > image-2018-10-22-20-20-24-658.png, image-2018-10-22-20-20-59-105.png > > > Follow-up to DRILL-4580. > Provide ability to export all storage plugin configurations at once, with a > new "Export All" option on the Storage page of the Drill web UI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6562) Plugin Management improvements
[ https://issues.apache.org/jira/browse/DRILL-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821460#comment-16821460 ] Bridget Bevens commented on DRILL-6562: --- Hi [~vitalii], I added info about exporting all storage plugins here: https://drill.apache.org/docs/configuring-storage-plugins/#exporting-storage-plugin-configurations I’m still not sure how the .conf file works – do you copy and paste the configurations into each storage plugin configuration manually, or is there an easier way to import the file? Thanks, Bridget > Plugin Management improvements > -- > > Key: DRILL-6562 > URL: https://issues.apache.org/jira/browse/DRILL-6562 > Project: Apache Drill > Issue Type: Improvement > Components: Client - HTTP, Web Server >Affects Versions: 1.14.0 >Reporter: Abhishek Girish >Assignee: Vitalii Diravka >Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.16.0 > > Attachments: Export.png, ExportAll.png, Screenshot from 2019-03-21 > 01-18-17.png, Screenshot from 2019-03-21 02-52-50.png, Storage.png, > UpdateExport.png, create.png, image-2018-07-23-02-55-02-024.png, > image-2018-10-22-20-20-24-658.png, image-2018-10-22-20-20-59-105.png > > > Follow-up to DRILL-4580. > Provide ability to export all storage plugin configurations at once, with a > new "Export All" option on the Storage page of the Drill web UI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6921) Add button to reset Options filter
[ https://issues.apache.org/jira/browse/DRILL-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens updated DRILL-6921: -- Labels: doc-complete ready-to-commit (was: doc-impacting ready-to-commit) > Add button to reset Options filter > -- > > Key: DRILL-6921 > URL: https://issues.apache.org/jira/browse/DRILL-6921 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.15.0 >Reporter: Arina Ielchiieva >Assignee: Kunal Khatua >Priority: Major > Labels: doc-complete, ready-to-commit > Fix For: 1.16.0 > > > Currently we have ability to search options or use quick filter on Web UI. To > reset the filter, user needs to delete input from the search pane manually. > It would be nice if we had Reset button. > Also we can consider leaving filter after option update / reset rather then > reloading page without filtering. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6921) Add button to reset Options filter
[ https://issues.apache.org/jira/browse/DRILL-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821457#comment-16821457 ] Bridget Bevens commented on DRILL-6921: --- Hi [~kkhatua], Add content about resetting filter options here, in the Search field section: https://drill.apache.org/docs/planning-and-execution-options/#setting-options-from-the-drill-web-ui Let me know if I need to make any changes. Thanks, Bridget > Add button to reset Options filter > -- > > Key: DRILL-6921 > URL: https://issues.apache.org/jira/browse/DRILL-6921 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.15.0 >Reporter: Arina Ielchiieva >Assignee: Kunal Khatua >Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.16.0 > > > Currently we have ability to search options or use quick filter on Web UI. To > reset the filter, user needs to delete input from the search pane manually. > It would be nice if we had Reset button. > Also we can consider leaving filter after option update / reset rather then > reloading page without filtering. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6939) Indicate when a query is submitted and is in progress
[ https://issues.apache.org/jira/browse/DRILL-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821454#comment-16821454 ] Bridget Bevens commented on DRILL-6939: --- Hi [~kkhatua], I added content about the pop-up here (may need to resize image to make smaller): https://drill.apache.org/docs/starting-the-web-ui/#running-queries-from-the-web-ui Let me know if I need to change anything. Thanks, Bridget > Indicate when a query is submitted and is in progress > - > > Key: DRILL-6939 > URL: https://issues.apache.org/jira/browse/DRILL-6939 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.14.0 >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Critical > Labels: doc-impacting, ready-to-commit, user-experience > Fix For: 1.16.0 > > > When submitting a long running query, the web UI shows no indication of the > query having been submitted. What is needed is some form of UI enhancement > that shows that the submitted query is in progress and the results will load > when available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6939) Indicate when a query is submitted and is in progress
[ https://issues.apache.org/jira/browse/DRILL-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens updated DRILL-6939: -- Labels: doc-complete ready-to-commit user-experience (was: doc-impacting ready-to-commit user-experience) > Indicate when a query is submitted and is in progress > - > > Key: DRILL-6939 > URL: https://issues.apache.org/jira/browse/DRILL-6939 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.14.0 >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Critical > Labels: doc-complete, ready-to-commit, user-experience > Fix For: 1.16.0 > > > When submitting a long running query, the web UI shows no indication of the > query having been submitted. What is needed is some form of UI enhancement > that shows that the submitted query is in progress and the results will load > when available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6942) Provide ability to sort list of profiles on Drill web UI
[ https://issues.apache.org/jira/browse/DRILL-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821452#comment-16821452 ] Bridget Bevens commented on DRILL-6942: --- hi [~kkhatua]. I added content about sorting columns here: https://drill.apache.org/docs/query-profiles/#viewing-a-query-profile Let me know if I need to make any changes. Thanks, Bridget > Provide ability to sort list of profiles on Drill web UI > > > Key: DRILL-6942 > URL: https://issues.apache.org/jira/browse/DRILL-6942 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.15.0 >Reporter: Khurram Faraaz >Assignee: Kunal Khatua >Priority: Major > Labels: doc-impacting, ready-to-commit, user-experience > Fix For: 1.16.0 > > > We need to provide a way to sort the query profiles, on the profiles tab on > Drill web UI. > The use case is when users run many queries (several hundred queries), they > want a way to list the queries that have taken the longest time (i.e. > duration) to complete. All queries that completed, failed or were canceled > and took very long, all such queries should be listed on the top in the > profiles tab page on Drill web ui. > An option to list the query profiles based on total duration should be > provided to user. That way user can easily identify such long running queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6942) Provide ability to sort list of profiles on Drill web UI
[ https://issues.apache.org/jira/browse/DRILL-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens updated DRILL-6942: -- Labels: doc-complete ready-to-commit user-experience (was: doc-impacting ready-to-commit user-experience) > Provide ability to sort list of profiles on Drill web UI > > > Key: DRILL-6942 > URL: https://issues.apache.org/jira/browse/DRILL-6942 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.15.0 >Reporter: Khurram Faraaz >Assignee: Kunal Khatua >Priority: Major > Labels: doc-complete, ready-to-commit, user-experience > Fix For: 1.16.0 > > > We need to provide a way to sort the query profiles, on the profiles tab on > Drill web UI. > The use case is when users run many queries (several hundred queries), they > want a way to list the queries that have taken the longest time (i.e. > duration) to complete. All queries that completed, failed or were canceled > and took very long, all such queries should be listed on the top in the > profiles tab page on Drill web ui. > An option to list the query profiles based on total duration should be > provided to user. That way user can easily identify such long running queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6879) Indicate a warning in the WebUI when a query makes little to no progress for a while
[
https://issues.apache.org/jira/browse/DRILL-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bridget Bevens updated DRILL-6879:
--
Labels: doc-complete ready-to-commit (was: doc-impacting ready-to-commit)
> Indicate a warning in the WebUI when a query makes little to no progress for
> a while
>
>
> Key: DRILL-6879
> URL: https://issues.apache.org/jira/browse/DRILL-6879
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Monitoring, Web Server
>Affects Versions: 1.14.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Major
> Labels: doc-complete, ready-to-commit
> Fix For: 1.16.0
>
> Attachments: image-2018-12-04-11-54-54-247.png,
> image-2018-12-06-11-19-00-339.png, image-2018-12-06-11-27-14-719.png
>
>
> When running a very large query on a cluster with limited resource, we
> noticed that one of the node's VM thread freezes the fragment threads as it
> tries to do some work (GC perhaps?). This is a clear indication that the
> query is stuck in a weird state where it might not recover from.
> Under such circumstances, it makes sense to cancel or atleast warn the user
> on that page of the query exceeding a certain threshold.
> For detecting this, the user will find that the {{Last Progress}} column in
> the Fragments Overview section will show large times.
> !image-2018-12-04-11-54-54-247.png|width=969,height=336!
> In addition, there are instances where a query might have buffered operators
> spilling to disk, which also hits performance (and, subsequently, longer run
> times). Calling out this skew can be very useful.
> !image-2018-12-06-11-27-14-719.png|width=969,height=256!
> Or there might be cases where a single fragment takes much longer than the
> average (indicated by an extreme skew in the Gantt chart).
> !image-2018-12-06-11-19-00-339.png|width=969,height=150!
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6879) Indicate a warning in the WebUI when a query makes little to no progress for a while
[
https://issues.apache.org/jira/browse/DRILL-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821451#comment-16821451
]
Bridget Bevens commented on DRILL-6879:
---
Hi [~kkhatua],
I added content about the warnings is here:
https://drill.apache.org/docs/query-profiles/#query-profile-warnings
You've reviewed this content already, but please let me know if you want me to
make any changes.
Thanks,
Bridget
> Indicate a warning in the WebUI when a query makes little to no progress for
> a while
>
>
> Key: DRILL-6879
> URL: https://issues.apache.org/jira/browse/DRILL-6879
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Monitoring, Web Server
>Affects Versions: 1.14.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Major
> Labels: doc-impacting, ready-to-commit
> Fix For: 1.16.0
>
> Attachments: image-2018-12-04-11-54-54-247.png,
> image-2018-12-06-11-19-00-339.png, image-2018-12-06-11-27-14-719.png
>
>
> When running a very large query on a cluster with limited resource, we
> noticed that one of the node's VM thread freezes the fragment threads as it
> tries to do some work (GC perhaps?). This is a clear indication that the
> query is stuck in a weird state where it might not recover from.
> Under such circumstances, it makes sense to cancel or atleast warn the user
> on that page of the query exceeding a certain threshold.
> For detecting this, the user will find that the {{Last Progress}} column in
> the Fragments Overview section will show large times.
> !image-2018-12-04-11-54-54-247.png|width=969,height=336!
> In addition, there are instances where a query might have buffered operators
> spilling to disk, which also hits performance (and, subsequently, longer run
> times). Calling out this skew can be very useful.
> !image-2018-12-06-11-27-14-719.png|width=969,height=256!
> Or there might be cases where a single fragment takes much longer than the
> average (indicated by an extreme skew in the Gantt chart).
> !image-2018-12-06-11-19-00-339.png|width=969,height=150!
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7105) Error while building the Drill native client
[
https://issues.apache.org/jira/browse/DRILL-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sorabh Hamirwasia updated DRILL-7105:
-
Fix Version/s: 1.17.0
> Error while building the Drill native client
>
>
> Key: DRILL-7105
> URL: https://issues.apache.org/jira/browse/DRILL-7105
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - C++
>Affects Versions: 1.16.0
>Reporter: Anton Gozhiy
>Assignee: Anton Gozhiy
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.16.0, 1.17.0
>
>
> *Steps:*
> # cd contrib/native/client
> # mkdir build
> # cd build && cmake -std=c++11 -G "Unix Makefiles" -D CMAKE_BUILD_TYPE=Debug
> ..
> # make
> *Expected result:*
> The native client is built successfully.
> *Actual result:*
> Error happens:
> {noformat}
> [ 4%] Built target y2038
> [ 7%] Building CXX object
> src/protobuf/CMakeFiles/protomsgs.dir/BitControl.pb.cc.o
> In file included from /usr/include/c++/5/mutex:35:0,
> from /usr/local/include/google/protobuf/stubs/mutex.h:33,
> from /usr/local/include/google/protobuf/stubs/common.h:52,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.h:9,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.cc:4:
> /usr/include/c++/5/bits/c++0x_warning.h:32:2: error: #error This file
> requires compiler and library support for the ISO C++ 2011 standard. This
> support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
> #error This file requires compiler and library support \
> ^
> In file included from /usr/local/include/google/protobuf/stubs/common.h:52:0,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.h:9,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.cc:4:
> /usr/local/include/google/protobuf/stubs/mutex.h:58:8: error: 'mutex' in
> namespace 'std' does not name a type
>std::mutex mu_;
> ^
> /usr/local/include/google/protobuf/stubs/mutex.h: In member function 'void
> google::protobuf::internal::WrappedMutex::Lock()':
> /usr/local/include/google/protobuf/stubs/mutex.h:51:17: error: 'mu_' was not
> declared in this scope
>void Lock() { mu_.lock(); }
> ^
> /usr/local/include/google/protobuf/stubs/mutex.h: In member function 'void
> google::protobuf::internal::WrappedMutex::Unlock()':
> /usr/local/include/google/protobuf/stubs/mutex.h:52:19: error: 'mu_' was not
> declared in this scope
>void Unlock() { mu_.unlock(); }
>^
> /usr/local/include/google/protobuf/stubs/mutex.h: At global scope:
> /usr/local/include/google/protobuf/stubs/mutex.h:61:7: error: expected
> nested-name-specifier before 'Mutex'
> using Mutex = WrappedMutex;
>^
> /usr/local/include/google/protobuf/stubs/mutex.h:66:28: error: expected ')'
> before '*' token
>explicit MutexLock(Mutex *mu) : mu_(mu) { this->mu_->Lock(); }
> ^
> /usr/local/include/google/protobuf/stubs/mutex.h:69:3: error: 'Mutex' does
> not name a type
>Mutex *const mu_;
>^
> /usr/local/include/google/protobuf/stubs/mutex.h: In destructor
> 'google::protobuf::internal::MutexLock::~MutexLock()':
> /usr/local/include/google/protobuf/stubs/mutex.h:67:24: error: 'class
> google::protobuf::internal::MutexLock' has no member named 'mu_'
>~MutexLock() { this->mu_->Unlock(); }
> ^
> /usr/local/include/google/protobuf/stubs/mutex.h: At global scope:
> /usr/local/include/google/protobuf/stubs/mutex.h:80:33: error: expected ')'
> before '*' token
>explicit MutexLockMaybe(Mutex *mu) :
> ^
> In file included from /usr/local/include/google/protobuf/arena.h:48:0,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.h:23,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.cc:4:
> /usr/include/c++/5/typeinfo:39:37: error: expected '}' before end of line
> /usr/include/c++/5/typeinfo:39:37: error: expected unqualified-id before end
> of line
> /usr/include/c++/5/typeinfo:39:37: error: expected '}' before end of line
> /usr/include/c++/5/typeinfo:39:37: error: expected '}' before end of line
> /usr/include/c++/5/typeinfo:39:37: error: expected '}' before end of line
> /usr/include/c++/5/typeinfo:39:37: error: expected declaration before end of
> line
> src/protobuf/CMakeFiles/protomsgs.dir/build.make:62: recipe for target
> 'src/protobuf/CMakeFiles/protomsgs.dir/BitControl.pb.cc.o' failed
> make[2]: ***
[jira] [Commented] (DRILL-7105) Error while building the Drill native client
[
https://issues.apache.org/jira/browse/DRILL-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821423#comment-16821423
]
Sorabh Hamirwasia commented on DRILL-7105:
--
This change is reverted in 1.16.0 branch
> Error while building the Drill native client
>
>
> Key: DRILL-7105
> URL: https://issues.apache.org/jira/browse/DRILL-7105
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - C++
>Affects Versions: 1.16.0
>Reporter: Anton Gozhiy
>Assignee: Anton Gozhiy
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.16.0, 1.17.0
>
>
> *Steps:*
> # cd contrib/native/client
> # mkdir build
> # cd build && cmake -std=c++11 -G "Unix Makefiles" -D CMAKE_BUILD_TYPE=Debug
> ..
> # make
> *Expected result:*
> The native client is built successfully.
> *Actual result:*
> Error happens:
> {noformat}
> [ 4%] Built target y2038
> [ 7%] Building CXX object
> src/protobuf/CMakeFiles/protomsgs.dir/BitControl.pb.cc.o
> In file included from /usr/include/c++/5/mutex:35:0,
> from /usr/local/include/google/protobuf/stubs/mutex.h:33,
> from /usr/local/include/google/protobuf/stubs/common.h:52,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.h:9,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.cc:4:
> /usr/include/c++/5/bits/c++0x_warning.h:32:2: error: #error This file
> requires compiler and library support for the ISO C++ 2011 standard. This
> support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
> #error This file requires compiler and library support \
> ^
> In file included from /usr/local/include/google/protobuf/stubs/common.h:52:0,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.h:9,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.cc:4:
> /usr/local/include/google/protobuf/stubs/mutex.h:58:8: error: 'mutex' in
> namespace 'std' does not name a type
>std::mutex mu_;
> ^
> /usr/local/include/google/protobuf/stubs/mutex.h: In member function 'void
> google::protobuf::internal::WrappedMutex::Lock()':
> /usr/local/include/google/protobuf/stubs/mutex.h:51:17: error: 'mu_' was not
> declared in this scope
>void Lock() { mu_.lock(); }
> ^
> /usr/local/include/google/protobuf/stubs/mutex.h: In member function 'void
> google::protobuf::internal::WrappedMutex::Unlock()':
> /usr/local/include/google/protobuf/stubs/mutex.h:52:19: error: 'mu_' was not
> declared in this scope
>void Unlock() { mu_.unlock(); }
>^
> /usr/local/include/google/protobuf/stubs/mutex.h: At global scope:
> /usr/local/include/google/protobuf/stubs/mutex.h:61:7: error: expected
> nested-name-specifier before 'Mutex'
> using Mutex = WrappedMutex;
>^
> /usr/local/include/google/protobuf/stubs/mutex.h:66:28: error: expected ')'
> before '*' token
>explicit MutexLock(Mutex *mu) : mu_(mu) { this->mu_->Lock(); }
> ^
> /usr/local/include/google/protobuf/stubs/mutex.h:69:3: error: 'Mutex' does
> not name a type
>Mutex *const mu_;
>^
> /usr/local/include/google/protobuf/stubs/mutex.h: In destructor
> 'google::protobuf::internal::MutexLock::~MutexLock()':
> /usr/local/include/google/protobuf/stubs/mutex.h:67:24: error: 'class
> google::protobuf::internal::MutexLock' has no member named 'mu_'
>~MutexLock() { this->mu_->Unlock(); }
> ^
> /usr/local/include/google/protobuf/stubs/mutex.h: At global scope:
> /usr/local/include/google/protobuf/stubs/mutex.h:80:33: error: expected ')'
> before '*' token
>explicit MutexLockMaybe(Mutex *mu) :
> ^
> In file included from /usr/local/include/google/protobuf/arena.h:48:0,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.h:23,
> from
> /home/agozhiy/git_repo/drill/contrib/native/client/src/protobuf/BitControl.pb.cc:4:
> /usr/include/c++/5/typeinfo:39:37: error: expected '}' before end of line
> /usr/include/c++/5/typeinfo:39:37: error: expected unqualified-id before end
> of line
> /usr/include/c++/5/typeinfo:39:37: error: expected '}' before end of line
> /usr/include/c++/5/typeinfo:39:37: error: expected '}' before end of line
> /usr/include/c++/5/typeinfo:39:37: error: expected '}' before end of line
> /usr/include/c++/5/typeinfo:39:37: error: expected declaration before end of
> line
> src/protobuf/CMakeFiles/protomsgs.dir/build.make:62: recipe for target
> 'src/protobuf/CMakeFiles/protomsgs.dir/BitControl.pb.cc.o' failed
[jira] [Commented] (DRILL-6642) Update protocol-buffers version
[
https://issues.apache.org/jira/browse/DRILL-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821422#comment-16821422
]
Sorabh Hamirwasia commented on DRILL-6642:
--
This change is reverted into 1.16 version
> Update protocol-buffers version
> ---
>
> Key: DRILL-6642
> URL: https://issues.apache.org/jira/browse/DRILL-6642
> Project: Apache Drill
> Issue Type: Task
> Components: Tools, Build Test
>Affects Versions: 1.14.0
>Reporter: Vitalii Diravka
>Assignee: Anton Gozhiy
>Priority: Major
> Fix For: 1.16.0, 1.17.0
>
>
> Currently Drill uses 2.5.0 {{protocol-buffers}} version.
> The last version is 3.6.0 in maven repo:
> [https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java]
> The new version has a lot of useful enhancements, which can be used in Drill.
> One of them is using {{UNRECOGNIZED Enum NullValue}}, which can help to
> handle them in place of null values for {{ProtocolMessageEnum}} - DRILL-6639.
> Looks like the NullValue can be used instead of null returned from
> {{valueOf()}} (_or {{forNumber()}}, since {{valueOf()}} is deprecated in the
> newer protobuf version_):
>
> [https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/NullValue]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6642) Update protocol-buffers version
[
https://issues.apache.org/jira/browse/DRILL-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sorabh Hamirwasia updated DRILL-6642:
-
Fix Version/s: 1.17.0
> Update protocol-buffers version
> ---
>
> Key: DRILL-6642
> URL: https://issues.apache.org/jira/browse/DRILL-6642
> Project: Apache Drill
> Issue Type: Task
> Components: Tools, Build Test
>Affects Versions: 1.14.0
>Reporter: Vitalii Diravka
>Assignee: Anton Gozhiy
>Priority: Major
> Fix For: 1.16.0, 1.17.0
>
>
> Currently Drill uses 2.5.0 {{protocol-buffers}} version.
> The last version is 3.6.0 in maven repo:
> [https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java]
> The new version has a lot of useful enhancements, which can be used in Drill.
> One of them is using {{UNRECOGNIZED Enum NullValue}}, which can help to
> handle them in place of null values for {{ProtocolMessageEnum}} - DRILL-6639.
> Looks like the NullValue can be used instead of null returned from
> {{valueOf()}} (_or {{forNumber()}}, since {{valueOf()}} is deprecated in the
> newer protobuf version_):
>
> [https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/NullValue]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7185) Drill Fails to Read Large Packets
[ https://issues.apache.org/jira/browse/DRILL-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sorabh Hamirwasia updated DRILL-7185: - Affects Version/s: (was: 1.15.0) 1.11.0 > Drill Fails to Read Large Packets > - > > Key: DRILL-7185 > URL: https://issues.apache.org/jira/browse/DRILL-7185 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.11.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Blocker > Fix For: 1.16.0 > > > Drill fails to read large packets and crashes. This small fix corrects that > issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7185) Drill Fails to Read Large Packets
[ https://issues.apache.org/jira/browse/DRILL-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sorabh Hamirwasia updated DRILL-7185: - Priority: Major (was: Blocker) > Drill Fails to Read Large Packets > - > > Key: DRILL-7185 > URL: https://issues.apache.org/jira/browse/DRILL-7185 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.11.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.16.0 > > > Drill fails to read large packets and crashes. This small fix corrects that > issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7185) Drill Fails to Read Large Packets
[ https://issues.apache.org/jira/browse/DRILL-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sorabh Hamirwasia updated DRILL-7185: - Fix Version/s: (was: 1.16.0) 1.17.0 > Drill Fails to Read Large Packets > - > > Key: DRILL-7185 > URL: https://issues.apache.org/jira/browse/DRILL-7185 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.11.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.17.0 > > > Drill fails to read large packets and crashes. This small fix corrects that > issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7180) Untarring the drill tarball and then tarring again makes drill unable to launch on YARN
[
https://issues.apache.org/jira/browse/DRILL-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821392#comment-16821392
]
Paul Rogers commented on DRILL-7180:
This is supposed to work. Indeed, there are instructions in the [Drill-on-YARN
documentation|http://drill.apache.org/docs/appendix-d-recreate-the-drill-archive/]
that covers this case. (Or, there was back when the feature was developed.)
As a general practice, you may not want to use the Drill package name for your
tar.gz file. You can name it something else. Then, you change a line in the DoY
config file to identify your new file.
Given that the issue is that {{$DRILL_HOME/bin/drill-am.sh}} is missing, it may
be that the structure of your new file does not match that of the original
Drill archive. (That is, a directory level may have been added or removed.) So,
perhaps extract your file and compare it with the structure you got when you
extracted the Drill archive.
The [Troubleshooting
guide|http://drill.apache.org/docs/appendix-c-troubleshooting/] may also be
helpful
> Untarring the drill tarball and then tarring again makes drill unable to
> launch on YARN
> ---
>
> Key: DRILL-7180
> URL: https://issues.apache.org/jira/browse/DRILL-7180
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
> Environment: hadoop 2.9.2
> RHEL 7.5
>Reporter: Georgi Perpeliev
>Priority: Minor
>
> Background: I tried substituting one of the drill jars with a modified
> version to fix a gzip bug. However I reproduced the bug with the original
> version of the distributed tarball
> I tried the following:
> # tar -xzf apache-drill-1.15.0.tar.gz
> # rm apache-drill-1.15.0.tar.gz
> # tar -czf apache-drill-1.15.0.tar.gz apache-drill-1.15.0
> # apache-drill-1.15.0/bin/drill-on-yarn.sh --site /apps/site start
> This fails with:
> /bin/bash:
> /tmp/hadoop-hadoop/nm-local-dir/usercache/hadoop/appcache/application_1555068097307_0003/container_1555068097307_0003_01_01/drill/apache-drill-1.15.0/bin/drill-am.sh:
> No such file or directory
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-7185) Drill Fails to Read Large Packets
Charles Givre created DRILL-7185: Summary: Drill Fails to Read Large Packets Key: DRILL-7185 URL: https://issues.apache.org/jira/browse/DRILL-7185 Project: Apache Drill Issue Type: Bug Affects Versions: 1.15.0 Reporter: Charles Givre Assignee: Charles Givre Fix For: 1.16.0 Drill fails to read large packets and crashes. This small fix corrects that issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7181) [Text V3 Reader] Exception with inadequate message is thrown if select columns as array with extractHeader set to true
[
https://issues.apache.org/jira/browse/DRILL-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821366#comment-16821366
]
Paul Rogers commented on DRILL-7181:
[~lhfei] are you working on this one? While we can't reproduce the V2
exception, we can provide a better message in V3. The fix is in the projection
framework, not the text reader.
Would be happy to either fix this, or provide suggestions for how to do so.
If you choose to fix it, you'll need to understand a bit about the new
framework, so let me know if I can provide an overview of that info.
> [Text V3 Reader] Exception with inadequate message is thrown if select
> columns as array with extractHeader set to true
> --
>
> Key: DRILL-7181
> URL: https://issues.apache.org/jira/browse/DRILL-7181
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.16.0
>Reporter: Anton Gozhiy
>Assignee: Hefei Li
>Priority: Major
>
> *Prerequisites:*
> # Create a simple .csv file with header, like this:
> {noformat}
> col1,col2,col3
> 1,2,3
> 4,5,6
> 7,8,9
> {noformat}
> # Set exec.storage.enable_v3_text_reader=true
> # Set "extractHeader": true for csv format in dfs storage plugin.
> *Query:*
> {code:sql}
> select columns[0] from dfs.tmp.`/test.csv`
> {code}
> *Expected result:* Exception should happen, here is the message from V2
> reader:
> {noformat}
> UNSUPPORTED_OPERATION ERROR: Drill Remote Exception
> (java.lang.Exception) UNSUPPORTED_OPERATION ERROR: With extractHeader
> enabled, only header names are supported
> column name columns
> column index
> Fragment 0:0
> [Error Id: 5affa696-1dbd-43d7-ac14-72d235c00f43 on userf87d-pc:31010]
> org.apache.drill.common.exceptions.UserException$Builder.build():630
>
> org.apache.drill.exec.store.easy.text.compliant.FieldVarCharOutput.():106
>
> org.apache.drill.exec.store.easy.text.compliant.CompliantTextRecordReader.setup():139
> org.apache.drill.exec.physical.impl.ScanBatch.getNextReaderIfHas():321
> org.apache.drill.exec.physical.impl.ScanBatch.internalNext():216
> org.apache.drill.exec.physical.impl.ScanBatch.next():271
> org.apache.drill.exec.record.AbstractRecordBatch.next():126
> org.apache.drill.exec.record.AbstractRecordBatch.next():116
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext():101
> org.apache.drill.exec.record.AbstractRecordBatch.next():186
> org.apache.drill.exec.record.AbstractRecordBatch.next():126
> org.apache.drill.exec.record.AbstractRecordBatch.next():116
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext():101
> org.apache.drill.exec.record.AbstractRecordBatch.next():186
> org.apache.drill.exec.record.AbstractRecordBatch.next():126
> org.apache.drill.exec.record.AbstractRecordBatch.next():116
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
> org.apache.drill.exec.record.AbstractRecordBatch.next():186
> org.apache.drill.exec.record.AbstractRecordBatch.next():126
> org.apache.drill.exec.record.AbstractRecordBatch.next():116
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():141
> org.apache.drill.exec.record.AbstractRecordBatch.next():186
> org.apache.drill.exec.physical.impl.BaseRootExec.next():104
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():83
> org.apache.drill.exec.physical.impl.BaseRootExec.next():94
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():296
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():283
> ...():0
> org.apache.hadoop.security.UserGroupInformation.doAs():1746
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():283
> org.apache.drill.common.SelfCleaningRunnable.run():38
> ...():0
> {noformat}
> *Actual result:* The exception message is inadequate:
> {noformat}
> org.apache.drill.common.exceptions.UserRemoteException: EXECUTION_ERROR
> ERROR: Table schema must have exactly one column.
> Exception thrown from
> org.apache.drill.exec.physical.impl.scan.ScanOperatorExec
> Fragment 0:0
> [Error Id: a76a1576-419a-413f-840f-088157167a6d on userf87d-pc:31010]
> (java.lang.IllegalStateException) Table schema must have exactly one column.
>
> org.apache.drill.exec.physical.impl.scan.columns.ColumnsArrayManager.resolveColumn():108
>
> org.apache.drill.exec.physical.impl.scan.project.ReaderLevelProjection.resolveSpecial():91
>
>
[jira] [Updated] (DRILL-7184) Set the IDs for the unique HTML tags in the Drill Web UI
[ https://issues.apache.org/jira/browse/DRILL-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-7184: Fix Version/s: 1.17.0 > Set the IDs for the unique HTML tags in the Drill Web UI > > > Key: DRILL-7184 > URL: https://issues.apache.org/jira/browse/DRILL-7184 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.16.0 >Reporter: Denys Ordynskiy >Assignee: Denys Ordynskiy >Priority: Major > Fix For: 1.17.0 > > > Selenium web pages automation requires identifiers on HTML tags. > We need to find all HTML tags without ID's that are useful for Drill Web UI > automation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7184) Set the IDs for the unique HTML tags in the Drill Web UI
[ https://issues.apache.org/jira/browse/DRILL-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-7184: Reviewer: Anton Gozhiy > Set the IDs for the unique HTML tags in the Drill Web UI > > > Key: DRILL-7184 > URL: https://issues.apache.org/jira/browse/DRILL-7184 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.16.0 >Reporter: Denys Ordynskiy >Assignee: Denys Ordynskiy >Priority: Major > Fix For: 1.17.0 > > > Selenium web pages automation requires identifiers on HTML tags. > We need to find all HTML tags without ID's that are useful for Drill Web UI > automation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7184) Set the IDs for the unique HTML tags in the Drill Web UI
[ https://issues.apache.org/jira/browse/DRILL-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-7184: Affects Version/s: 1.16.0 > Set the IDs for the unique HTML tags in the Drill Web UI > > > Key: DRILL-7184 > URL: https://issues.apache.org/jira/browse/DRILL-7184 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.16.0 >Reporter: Denys Ordynskiy >Assignee: Denys Ordynskiy >Priority: Major > > Selenium web pages automation requires identifiers on HTML tags. > We need to find all HTML tags without ID's that are useful for Drill Web UI > automation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-4232) Support for EXCEPT set operator
[
https://issues.apache.org/jira/browse/DRILL-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820990#comment-16820990
]
benj commented on DRILL-4232:
-
Of course, It's also possible to simulate *EXCEPT ALL*
{code:java}
SELECT a FROM `file1.csvh`
EXCEPT ALL
SELECT a FROM `file2.csvh`
{code}
Becomes
{code:java}
SELECT a FROM
(
SELECT a, idfile
, row_number() OVER (PARTITION BY a, idfile) AS rn
, count(CASE WHEN idfile = 1 THEN 1 END) OVER (PARTITION BY a) AS nb_elt_tbl1
, count(CASE WHEN idfile = 0 THEN 1 END) OVER (PARTITION BY a) AS nb_elt_tbl2
FROM
(
SELECT a, 1 idfile FROM `file1.csvh` AS tbl1
UNION ALL
SELECT a, 0 idfile FROM `file2.csvh` AS tbl2
) AS lvl_union
) AS lvl_over
WHERE idfile = 1 AND rn <= nb_elt_tbl1 - nb_elt_tbl2
{code}
> Support for EXCEPT set operator
> ---
>
> Key: DRILL-4232
> URL: https://issues.apache.org/jira/browse/DRILL-4232
> Project: Apache Drill
> Issue Type: New Feature
> Components: Query Planning Optimization
>Reporter: Victoria Markman
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-7184) Set the IDs for the unique HTML tags in the Drill Web UI
Denys Ordynskiy created DRILL-7184: -- Summary: Set the IDs for the unique HTML tags in the Drill Web UI Key: DRILL-7184 URL: https://issues.apache.org/jira/browse/DRILL-7184 Project: Apache Drill Issue Type: Task Reporter: Denys Ordynskiy Assignee: Denys Ordynskiy Selenium web pages automation requires identifiers on HTML tags. We need to find all HTML tags without ID's that are useful for Drill Web UI automation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DRILL-4092) Support for INTERSECT
[
https://issues.apache.org/jira/browse/DRILL-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797211#comment-16797211
]
benj edited comment on DRILL-4092 at 4/18/19 11:08 AM:
---
Until the feature is implemented, there are queries to achieve the same result.
with file1.csvh and file2.csvh each containing only column a
without INTERSECT we can do
{code:java}
/* Simulate INTERSECT */
SELECT DISTINCT a FROM
(
SELECT a
, min(idfile) over(PARTITION BY a) AS minc
, max(idfile) over(PARTITION BY a) AS maxc
FROM
(
SELECT a, 0 idfile FROM `file1.csvh`
UNION ALL
SELECT a, 1 idfile FROM `file2.csvh`
) AS lvl_union
) AS lvl_over WHERE minc = 0 and maxc = 1{code}
But it's a little bit more complicated for INTERSECT ALL (as defined in 7.12 of
ansi-iso-9075-2-1999.pdf)
{code:java}
/* Simulate INTERSECT ALL */
SELECT a FROM
(
SELECT a, idfile
, count(CASE WHEN idfile = 0 THEN 1 END) OVER (PARTITION BY a) AS nb_elt_tbl1
, count(CASE WHEN idfile = 1 THEN 1 END) OVER (PARTITION BY a) AS nb_elt_tbl2
FROM
(
SELECT a, 0 idfile FROM `file1.csvh`
UNION ALL
SELECT a, 1 idfile FROM `file2.csvh`
) AS lvl_union
) AS lvl_over
WHERE nb_elt_tbl1 > 0 AND nb_elt_tbl2 > 0
AND CASE WHEN nb_elt_tbl1 < nb_elt_tbl2 THEN idfile = 0 ELSE idfile = 1 END
{code}
Hoping that this can help until we can use the standard SQL syntax.
was (Author: benj641):
Until the feature is implemented, there are queries to achieve the same result.
with file1.csvh and file2.csvh each containing only column a
without INTERSECT we can do
{code:java}
/* Simulate INTERSECT */
SELECT DISTINCT a FROM
(
SELECT a
, min(idfile) over(PARTITION BY a) AS minc
, max(idfile) over(PARTITION BY a) AS maxc
FROM
(
SELECT a, 0 idfile FROM `file1.csvh`
UNION ALL
SELECT a, 1 idfile FROM `file2.csvh`
) AS lvl_union
) AS lvl_over WHERE minc = 0 and maxc = 1{code}
But it's a little bit more complicated for INTERSECT ALL (as defined in 7.12 of
ansi-iso-9075-2-1999.pdf)
{code:java}
/* Simulate INTERSECT ALL */
SELECT a FROM
(
SELECT a, c
, min(idfile) over(PARTITION BY a) AS minc
, max(idfile) over(PARTITION BY a) AS maxc
, count(a) OVER(PARTITION BY a, c) AS nb_elt_by_table
,LEAST(count(CASE WHEN c = 0 THEN 1 END) OVER (PARTITION BY a)
, count(CASE WHEN c = 1 THEN 1 END) OVER (PARTITION BY a)) min_elt
FROM
(
SELECT a, 0 idfile FROM `file1.csvh`
UNION ALL
SELECT a, 1 idfile FROM `file2.csvh`
) AS lvl_union
) AS lvl_over
WHERE minc = 0 and maxc = 1 AND nb_elt_by_table = min_elt
{code}
Hoping that this can help until we can use the standard SQL syntax.
> Support for INTERSECT
> --
>
> Key: DRILL-4092
> URL: https://issues.apache.org/jira/browse/DRILL-4092
> Project: Apache Drill
> Issue Type: New Feature
>Reporter: Victoria Markman
>Assignee: Prasad Nagaraj Subramanya
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6988) Utility of the too long error message when syntax error
[
https://issues.apache.org/jira/browse/DRILL-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Volodymyr Vysotskyi updated DRILL-6988:
---
Labels: ready-to-commit (was: )
> Utility of the too long error message when syntax error
> ---
>
> Key: DRILL-6988
> URL: https://issues.apache.org/jira/browse/DRILL-6988
> Project: Apache Drill
> Issue Type: Improvement
> Components: SQL Parser
>Affects Versions: 1.15.0
>Reporter: benj
>Assignee: Dmytriy Grinchenko
>Priority: Minor
> Labels: ready-to-commit
> Fix For: 1.17.0
>
> Attachments: error_picture_sample.png
>
>
> When executing query with syntax error, the too long and useless error
> message pollute log/screen and doesn't give any useful information.
> Maybe a more concise error message (1-3 line max) should be enough.
> {code:java}
> SELECT FROM (VALUES(1));
> Error: PARSE ERROR: Encountered "FROM" at line 1, column 8.
> Was expecting one of:
> "UNION" ...
> "INTERSECT" ...
> "EXCEPT" ...
> "MINUS" ...
> "ORDER" ...
> "LIMIT" ...
> "OFFSET" ...
> "FETCH" ...
> "STREAM" ...
> "DISTINCT" ...
> "ALL" ...
> "*" ...
> "+" ...
> "-" ...
> "NOT" ...
> "EXISTS" ...
> ...
> ...
> ...
> ...
> ...
> ...
> ...
> "TRUE" ...
> "FALSE" ...
> "UNKNOWN" ...
> "NULL" ...
> ...
> ...
> ...
> "DATE" ...
> "TIME" ...
> "TIMESTAMP" ...
> "INTERVAL" ...
> "?" ...
> "CAST" ...
> "EXTRACT" ...
> "POSITION" ...
> "CONVERT" ...
> "TRANSLATE" ...
> "OVERLAY" ...
> "FLOOR" ...
> "CEIL" ...
> "CEILING" ...
> "SUBSTRING" ...
> "TRIM" ...
> "CLASSIFIER" ...
> "MATCH_NUMBER" ...
> "RUNNING" ...
> "PREV" ...
> "NEXT" ...
> ...
> "MULTISET" ...
> "ARRAY" ...
> "PERIOD" ...
> "SPECIFIC" ...
> ...
> ...
> ...
> ...
> ...
> "ABS" ...
> "AVG" ...
> "CARDINALITY" ...
> "CHAR_LENGTH" ...
> "CHARACTER_LENGTH" ...
> "COALESCE" ...
> "COLLECT" ...
> "COVAR_POP" ...
> "COVAR_SAMP" ...
> "CUME_DIST" ...
> "COUNT" ...
> "CURRENT_DATE" ...
> "CURRENT_TIME" ...
> "CURRENT_TIMESTAMP" ...
> "DENSE_RANK" ...
> "ELEMENT" ...
> "EXP" ...
> "FIRST_VALUE" ...
> "FUSION" ...
> "GROUPING" ...
> "HOUR" ...
> "LAG" ...
> "LEAD" ...
> "LAST_VALUE" ...
> "LN" ...
> "LOCALTIME" ...
> "LOCALTIMESTAMP" ...
> "LOWER" ...
> "MAX" ...
> "MIN" ...
> "MINUTE" ...
> "MOD" ...
> "MONTH" ...
> "NTH_VALUE" ...
> "NTILE" ...
> "NULLIF" ...
> "OCTET_LENGTH" ...
> "PERCENT_RANK" ...
> "POWER" ...
> "RANK" ...
> "REGR_SXX" ...
> "REGR_SYY" ...
> "ROW_NUMBER" ...
> "SECOND" ...
> "SQRT" ...
> "STDDEV_POP" ...
> "STDDEV_SAMP" ...
> "SUM" ...
> "UPPER" ...
> "TRUNCATE" ...
> "USER" ...
> "VAR_POP" ...
> "VAR_SAMP" ...
> "YEAR" ...
> "CURRENT_CATALOG" ...
> "CURRENT_DEFAULT_TRANSFORM_GROUP" ...
> "CURRENT_PATH" ...
> "CURRENT_ROLE" ...
> "CURRENT_SCHEMA" ...
> "CURRENT_USER" ...
> "SESSION_USER" ...
> "SYSTEM_USER" ...
> "NEW" ...
> "CASE" ...
> "CURRENT" ...
> "CURSOR" ...
> "ROW" ...
> "(" ...
>
> SQL Query SELECT FROM VALUES(1)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-7098) File Metadata Metastore Plugin
[
https://issues.apache.org/jira/browse/DRILL-7098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820858#comment-16820858
]
ASF GitHub Bot commented on DRILL-7098:
---
vvysotskyi commented on pull request #1754: DRILL-7098: File Metadata Metastore
Plugin
URL: https://github.com/apache/drill/pull/1754#discussion_r276312361
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/expr/IsPredicate.java
##
@@ -76,7 +76,7 @@ static boolean isNullOrEmpty(ColumnStatistics stat) {
|| !stat.containsStatistic(ColumnStatisticsKind.MIN_VALUE)
|| !stat.containsStatistic(ColumnStatisticsKind.MAX_VALUE)
|| !stat.containsStatistic(ColumnStatisticsKind.NULLS_COUNT)
-|| (long) stat.getStatistic(ColumnStatisticsKind.NULLS_COUNT) ==
GroupScan.NO_COLUMN_STATS;
+|| (long) stat.getStatistic(ColumnStatisticsKind.NULLS_COUNT) ==
Statistic.NO_COLUMN_STATS;
Review comment:
Looks like there are some extra spaces before `Statistic` interface here and
in other places were added. Could you please remove them?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> File Metadata Metastore Plugin
> --
>
> Key: DRILL-7098
> URL: https://issues.apache.org/jira/browse/DRILL-7098
> Project: Apache Drill
> Issue Type: Sub-task
> Components: Server, Metadata
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Major
> Labels: Metastore
> Fix For: 2.0.0
>
>
> DRILL-6852 introduces Drill Metastore API.
> The second step is to create internal Drill Metastore mechanism (and File
> Metastore Plugin), which will involve Metastore API and can be extended for
> using by other Storage Plugins.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7183) TPCDS query 10, 35, 69 take longer with sf 1000 when Statistics are disabled
[
https://issues.apache.org/jira/browse/DRILL-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sorabh Hamirwasia updated DRILL-7183:
-
Labels: ready-to-commit (was: )
> TPCDS query 10, 35, 69 take longer with sf 1000 when Statistics are disabled
>
>
> Key: DRILL-7183
> URL: https://issues.apache.org/jira/browse/DRILL-7183
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning Optimization
>Affects Versions: 1.16.0
>Reporter: Robert Hou
>Assignee: Hanumath Rao Maduri
>Priority: Blocker
> Labels: ready-to-commit
> Fix For: 1.16.0
>
>
> Query 69 runs 150% slower when Statistics is disabled. Here is the query:
> {noformat}
> SELECT
> cd_gender,
> cd_marital_status,
> cd_education_status,
> count(*) cnt1,
> cd_purchase_estimate,
> count(*) cnt2,
> cd_credit_rating,
> count(*) cnt3
> FROM
> customer c, customer_address ca, customer_demographics
> WHERE
> c.c_current_addr_sk = ca.ca_address_sk AND
> ca_state IN ('KY', 'GA', 'NM') AND
> cd_demo_sk = c.c_current_cdemo_sk AND
> exists(SELECT *
>FROM store_sales, date_dim
>WHERE c.c_customer_sk = ss_customer_sk AND
> ss_sold_date_sk = d_date_sk AND
> d_year = 2001 AND
> d_moy BETWEEN 4 AND 4 + 2) AND
> (NOT exists(SELECT *
> FROM web_sales, date_dim
> WHERE c.c_customer_sk = ws_bill_customer_sk AND
> ws_sold_date_sk = d_date_sk AND
> d_year = 2001 AND
> d_moy BETWEEN 4 AND 4 + 2) AND
> NOT exists(SELECT *
> FROM catalog_sales, date_dim
> WHERE c.c_customer_sk = cs_ship_customer_sk AND
>cs_sold_date_sk = d_date_sk AND
>d_year = 2001 AND
>d_moy BETWEEN 4 AND 4 + 2))
> GROUP BY cd_gender, cd_marital_status, cd_education_status,
> cd_purchase_estimate, cd_credit_rating
> ORDER BY cd_gender, cd_marital_status, cd_education_status,
> cd_purchase_estimate, cd_credit_rating
> LIMIT 100;
> {noformat}
> This regression is caused by commit 982e98061e029a39f1c593f695c0d93ec7079f0d.
> This commit should be reverted for now.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-7183) TPCDS query 10, 35, 69 take longer with sf 1000 when Statistics are disabled
[
https://issues.apache.org/jira/browse/DRILL-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820758#comment-16820758
]
ASF GitHub Bot commented on DRILL-7183:
---
sohami commented on pull request #1755: DRILL-7183: TPCDS query 10, 35, 69 take
longer with sf 1000 when Statistics are disabled.
URL: https://github.com/apache/drill/pull/1755
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> TPCDS query 10, 35, 69 take longer with sf 1000 when Statistics are disabled
>
>
> Key: DRILL-7183
> URL: https://issues.apache.org/jira/browse/DRILL-7183
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning Optimization
>Affects Versions: 1.16.0
>Reporter: Robert Hou
>Assignee: Hanumath Rao Maduri
>Priority: Blocker
> Labels: ready-to-commit
> Fix For: 1.16.0
>
>
> Query 69 runs 150% slower when Statistics is disabled. Here is the query:
> {noformat}
> SELECT
> cd_gender,
> cd_marital_status,
> cd_education_status,
> count(*) cnt1,
> cd_purchase_estimate,
> count(*) cnt2,
> cd_credit_rating,
> count(*) cnt3
> FROM
> customer c, customer_address ca, customer_demographics
> WHERE
> c.c_current_addr_sk = ca.ca_address_sk AND
> ca_state IN ('KY', 'GA', 'NM') AND
> cd_demo_sk = c.c_current_cdemo_sk AND
> exists(SELECT *
>FROM store_sales, date_dim
>WHERE c.c_customer_sk = ss_customer_sk AND
> ss_sold_date_sk = d_date_sk AND
> d_year = 2001 AND
> d_moy BETWEEN 4 AND 4 + 2) AND
> (NOT exists(SELECT *
> FROM web_sales, date_dim
> WHERE c.c_customer_sk = ws_bill_customer_sk AND
> ws_sold_date_sk = d_date_sk AND
> d_year = 2001 AND
> d_moy BETWEEN 4 AND 4 + 2) AND
> NOT exists(SELECT *
> FROM catalog_sales, date_dim
> WHERE c.c_customer_sk = cs_ship_customer_sk AND
>cs_sold_date_sk = d_date_sk AND
>d_year = 2001 AND
>d_moy BETWEEN 4 AND 4 + 2))
> GROUP BY cd_gender, cd_marital_status, cd_education_status,
> cd_purchase_estimate, cd_credit_rating
> ORDER BY cd_gender, cd_marital_status, cd_education_status,
> cd_purchase_estimate, cd_credit_rating
> LIMIT 100;
> {noformat}
> This regression is caused by commit 982e98061e029a39f1c593f695c0d93ec7079f0d.
> This commit should be reverted for now.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
