[jira] [Commented] (DRILL-7261) Simplify Easy format config for new scan framework
[ https://issues.apache.org/jira/browse/DRILL-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858230#comment-16858230 ] ASF GitHub Bot commented on DRILL-7261: --- paul-rogers commented on issue #1796: DRILL-7261: Simplify Easy framework config URL: https://github.com/apache/drill/pull/1796#issuecomment-499724041 Squashed commits This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Simplify Easy format config for new scan framework > -- > > Key: DRILL-7261 > URL: https://issues.apache.org/jira/browse/DRILL-7261 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Labels: ready-to-commit > Fix For: 1.17.0 > > > Rollup of related CSV V3 fixes along with supporting row set framework fixes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7278) Refactor result set loader projection mechanism
[ https://issues.apache.org/jira/browse/DRILL-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858228#comment-16858228 ] ASF GitHub Bot commented on DRILL-7278: --- paul-rogers commented on issue #1797: DRILL-7278: Refactor result set loader projection mechanism URL: https://github.com/apache/drill/pull/1797#issuecomment-499723714 Commits squashed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refactor result set loader projection mechanism > --- > > Key: DRILL-7278 > URL: https://issues.apache.org/jira/browse/DRILL-7278 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Labels: ready-to-commit > Fix For: 1.17.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7279) Support provided schema for CSV without headers
[ https://issues.apache.org/jira/browse/DRILL-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858227#comment-16858227 ] ASF GitHub Bot commented on DRILL-7279: --- paul-rogers commented on issue #1798: DRILL-7279: Enable provided schema for text files without headers URL: https://github.com/apache/drill/pull/1798#issuecomment-499723430 Commits squashed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Support provided schema for CSV without headers > --- > > Key: DRILL-7279 > URL: https://issues.apache.org/jira/browse/DRILL-7279 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.17.0 > > > Extend the Drill 1.16 provided schema support for the text reader to allow a > provided schema for files without headers. Behavior: > * If the file is configured to not extract headers, and a schema is provided, > and the schema has at least one column, then use the provided schema to > create individual columns. Otherwise, continue to use {{columns}} as in > previous versions. > * The columns in the schema are assumed to match left-to-right with those in > the file. > * If the schema contains more columns than the file, the extra columns take > their default values. (This occurs in schema evolution when a column is added > to newer files.) > * If the file contains more columns than the schema, then the extra columns, > at the end of the line, are ignored. This is the same behavior as occurs if > the file contains headers. > h4. Table Properties > Also adds four table properties for text files. These properties, if present, > override those defined in the format plugin configuration. The properties > allow the user to have a single "csv" config, but to have many tables with > the "csv" suffix, each with different properties. That is, the user need not > define a new plugin config, and define a new extension, just to change a file > format property. With this system, the user can have a ".csv" file with > headers; the user need not define a different suffix (usually ".csvh" in > Drill) for this case. > || Table Property || Equivalent Plugin Config Property || > | {{drill.headers}} | {{extractHeader}} | > | {{drill.skipFirstLine}} | {{skipFirstLine}} | > | {{drill.delimiter}} | {{fieldDelimiter}} | > | {{drill.commentChar}} | {{comment}}| > For each, the rules are: > * If the table property is not set, then the plugin property is used. > * If the table property is set, then the property value replaces the plugin > property value for that one specific table. > * For the delimiter, if the property value is an empty string, then this is > the same as an unset property. > * For the comment, if the property value is an empty string, then the comment > is set to the ASCII NULL, which will never match. This effectively turns off > the comment feature for this one table. > * If the delimiter or comment value is longer than a single character, only > the first character is used. > It is possible to use the table properties without specifying a "provided" > schema. Just omit any columns from the schema: > {noformat} > create schema () for table `dfs.data`.`example` > PROPERTIES ('drill.headers'='false', 'drill.skipFirstLine'='false', > 'drill.delimiter'='|') > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6285) IllegalArgumentException: Self-suppression not permitted
[ https://issues.apache.org/jira/browse/DRILL-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Khurram Faraaz updated DRILL-6285: -- Affects Version/s: 1.15.0 > IllegalArgumentException: Self-suppression not permitted > > > Key: DRILL-6285 > URL: https://issues.apache.org/jira/browse/DRILL-6285 > Project: Apache Drill > Issue Type: Bug > Components: Execution - RPC >Affects Versions: 1.14.0, 1.15.0 >Reporter: Khurram Faraaz >Priority: Minor > > When a query is marked as Canceled on Web UI, there is this Exception in > drillbit.log > Drill 1.13.0 commit cac2882d5a9e22fbc251e4caf622fe30242ad557 > {noformat} > 2018-03-21 15:35:43,198 [UserServer-1] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 254d2796-7744-cc2a-d77e-5ec030b04211:0:0: State change requested RUNNING --> > FAILED > 2018-03-21 15:35:43,198 [254d2796-7744-cc2a-d77e-5ec030b04211:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 254d2796-7744-cc2a-d77e-5ec030b04211:0:0: State change requested FAILED --> > FAILED > 2018-03-21 15:35:43,199 [254d2796-7744-cc2a-d77e-5ec030b04211:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 254d2796-7744-cc2a-d77e-5ec030b04211:0:0: State change requested FAILED --> > FAILED > 2018-03-21 15:35:43,202 [254d2796-7744-cc2a-d77e-5ec030b04211:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 254d2796-7744-cc2a-d77e-5ec030b04211:0:0: State change requested FAILED --> > FAILED > 2018-03-21 15:35:43,202 [UserServer-1] WARN > o.apache.drill.exec.rpc.RequestIdMap - Failure while attempting to fail rpc > response. > java.lang.IllegalArgumentException: Self-suppression not permitted > at java.lang.Throwable.addSuppressed(Throwable.java:1043) ~[na:1.8.0_161] > at > org.apache.drill.common.DeferredException.addException(DeferredException.java:88) > ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.common.DeferredException.addThrowable(DeferredException.java:97) > ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.fail(FragmentExecutor.java:412) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.access$700(FragmentExecutor.java:55) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$ExecutorStateImpl.fail(FragmentExecutor.java:426) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContextImpl.fail(FragmentContextImpl.java:233) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContextImpl$1.accept(FragmentContextImpl.java:100) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContextImpl$1.accept(FragmentContextImpl.java:97) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at org.apache.drill.exec.ops.StatusHandler.failed(StatusHandler.java:42) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RequestIdMap$RpcListener.setException(RequestIdMap.java:139) > ~[drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RequestIdMap$SetExceptionProcedure.apply(RequestIdMap.java:76) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RequestIdMap$SetExceptionProcedure.apply(RequestIdMap.java:66) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at com.carrotsearch.hppc.IntObjectHashMap.forEach(IntObjectHashMap.java:692) > [hppc-0.7.1.jar:na] > at org.apache.drill.exec.rpc.RequestIdMap.channelClosed(RequestIdMap.java:62) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.AbstractRemoteConnection.channelClosed(AbstractRemoteConnection.java:192) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.AbstractServerConnection.channelClosed(AbstractServerConnection.java:165) > [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete(RpcBus.java:167) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete(RpcBus.java:144) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > [netty-common-4.0.48.Final.jar:4.0.48.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > [netty-common-4.0.48.Final.jar:4.0.48.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) > [netty-common-4.0.48.Final.jar:4.0.48.Final] > at >
[jira] [Updated] (DRILL-6285) IllegalArgumentException: Self-suppression not permitted
[ https://issues.apache.org/jira/browse/DRILL-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Khurram Faraaz updated DRILL-6285: -- Summary: IllegalArgumentException: Self-suppression not permitted (was: Failure while attempting to fail rpc response.) > IllegalArgumentException: Self-suppression not permitted > > > Key: DRILL-6285 > URL: https://issues.apache.org/jira/browse/DRILL-6285 > Project: Apache Drill > Issue Type: Bug > Components: Execution - RPC >Affects Versions: 1.14.0 >Reporter: Khurram Faraaz >Priority: Minor > > When a query is marked as Canceled on Web UI, there is this Exception in > drillbit.log > Drill 1.13.0 commit cac2882d5a9e22fbc251e4caf622fe30242ad557 > {noformat} > 2018-03-21 15:35:43,198 [UserServer-1] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 254d2796-7744-cc2a-d77e-5ec030b04211:0:0: State change requested RUNNING --> > FAILED > 2018-03-21 15:35:43,198 [254d2796-7744-cc2a-d77e-5ec030b04211:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 254d2796-7744-cc2a-d77e-5ec030b04211:0:0: State change requested FAILED --> > FAILED > 2018-03-21 15:35:43,199 [254d2796-7744-cc2a-d77e-5ec030b04211:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 254d2796-7744-cc2a-d77e-5ec030b04211:0:0: State change requested FAILED --> > FAILED > 2018-03-21 15:35:43,202 [254d2796-7744-cc2a-d77e-5ec030b04211:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 254d2796-7744-cc2a-d77e-5ec030b04211:0:0: State change requested FAILED --> > FAILED > 2018-03-21 15:35:43,202 [UserServer-1] WARN > o.apache.drill.exec.rpc.RequestIdMap - Failure while attempting to fail rpc > response. > java.lang.IllegalArgumentException: Self-suppression not permitted > at java.lang.Throwable.addSuppressed(Throwable.java:1043) ~[na:1.8.0_161] > at > org.apache.drill.common.DeferredException.addException(DeferredException.java:88) > ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.common.DeferredException.addThrowable(DeferredException.java:97) > ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.fail(FragmentExecutor.java:412) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.access$700(FragmentExecutor.java:55) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$ExecutorStateImpl.fail(FragmentExecutor.java:426) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContextImpl.fail(FragmentContextImpl.java:233) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContextImpl$1.accept(FragmentContextImpl.java:100) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContextImpl$1.accept(FragmentContextImpl.java:97) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at org.apache.drill.exec.ops.StatusHandler.failed(StatusHandler.java:42) > ~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RequestIdMap$RpcListener.setException(RequestIdMap.java:139) > ~[drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RequestIdMap$SetExceptionProcedure.apply(RequestIdMap.java:76) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RequestIdMap$SetExceptionProcedure.apply(RequestIdMap.java:66) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at com.carrotsearch.hppc.IntObjectHashMap.forEach(IntObjectHashMap.java:692) > [hppc-0.7.1.jar:na] > at org.apache.drill.exec.rpc.RequestIdMap.channelClosed(RequestIdMap.java:62) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.AbstractRemoteConnection.channelClosed(AbstractRemoteConnection.java:192) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.AbstractServerConnection.channelClosed(AbstractServerConnection.java:165) > [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete(RpcBus.java:167) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete(RpcBus.java:144) > [drill-rpc-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) > [netty-common-4.0.48.Final.jar:4.0.48.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) > [netty-common-4.0.48.Final.jar:4.0.48.Final] > at >
[jira] [Commented] (DRILL-7287) Search by selected name of option is performed when clicking on "Update" button in "Options" tab
[ https://issues.apache.org/jira/browse/DRILL-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857767#comment-16857767 ] Pavel Semenov commented on DRILL-7287: -- [~kkhatua] Please look at issue > Search by selected name of option is performed when clicking on "Update" > button in "Options" tab > > > Key: DRILL-7287 > URL: https://issues.apache.org/jira/browse/DRILL-7287 > Project: Apache Drill > Issue Type: Bug >Reporter: Pavel Semenov >Priority: Major > Attachments: chromeSettings.png > > > *STEPS TO REPRODUCE* > # Open UI drill admin panel > # Open "Options" tab > # Click on "Update" button for any option in the list (e.g. > "debug.validate_vectors" option) > # Observe the result > *EXPECTED RESULT* > Web page should be refreshed and option value should be applied > *ACTUAL RESULT* > Search by selected name of option is performed and option value is applied > *ADDITIONAL INFO* > Behavior of "Options" tab should be similar to e.g. "Settings" in Google > Chrome browser, because it has similar controls (see screenshot) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7287) Search by selected name of option is performed when clicking on "Update" button in "Options" tab
[ https://issues.apache.org/jira/browse/DRILL-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Semenov updated DRILL-7287: - Description: *STEPS TO REPRODUCE* # Open UI drill admin panel # Open "Options" tab # Click on "Update" button for any option in the list (e.g. "debug.validate_vectors" option) # Observe the result *EXPECTED RESULT* Web page should be refreshed and option value should be applied *ACTUAL RESULT* Search by selected name of option is performed and option value is applied *ADDITIONAL INFO* Behavior of "Options" tab should be similar to e.g. "Settings" in Google Chrome browser, because it has similar controls (see screenshot) was: *STEPS TO REPRODUCE* # Open UI drill admin panel # Open "Options" tab # Click on "Update" button for any option in the list (e.g. "debug.validate_vectors" option) # Observe the result *EXPECTED RESULT* Web page should be refreshed and option value should be applied *ACTUAL RESULT* Search by selected name of option is performed and option value is applied *ADDITIONAL INFO* Behavior of "Options" tab should be similar to e.g. "Settings" in Google Chrome browser, because it has similar controls ( see screenshot) > Search by selected name of option is performed when clicking on "Update" > button in "Options" tab > > > Key: DRILL-7287 > URL: https://issues.apache.org/jira/browse/DRILL-7287 > Project: Apache Drill > Issue Type: Bug >Reporter: Pavel Semenov >Priority: Major > Attachments: chromeSettings.png > > > *STEPS TO REPRODUCE* > # Open UI drill admin panel > # Open "Options" tab > # Click on "Update" button for any option in the list (e.g. > "debug.validate_vectors" option) > # Observe the result > *EXPECTED RESULT* > Web page should be refreshed and option value should be applied > *ACTUAL RESULT* > Search by selected name of option is performed and option value is applied > *ADDITIONAL INFO* > Behavior of "Options" tab should be similar to e.g. "Settings" in Google > Chrome browser, because it has similar controls (see screenshot) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7287) Search by selected name of option is performed when clicking on "Update" button in "Options" tab
[ https://issues.apache.org/jira/browse/DRILL-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Semenov updated DRILL-7287: - Description: *STEPS TO REPRODUCE* # Open UI drill admin panel # Open "Options" tab # Click on "Update" button for any option in the list (e.g. "debug.validate_vectors" option) # Observe the result *EXPECTED RESULT* Web page should be refreshed and option value should be applied *ACTUAL RESULT* Search by selected name of option is performed and option value is applied *ADDITIONAL INFO* Behavior of "Options" tab should be similar to e.g. "Settings" in Google Chrome browser, because it has similar controls ( see screenshot) was: *STEPS TO REPRODUCE* # Open UI drill admin panel # Open "Options" tab # Click on "Update" button for any option in the list (e.g. "debug.validate_vectors" option) # Observe the result *EXPECTED RESULT* Web page should be refreshed and option value should be applied *ACTUAL RESULT* Search by selected name of option is performed and option value is applied *ADDITIONAL INFO* Behavior of "Options" tab should be similar to e.g. "Settings" in Google Chrome browser, because it has similar controls !chromeSettings.png|thumbnail! > Search by selected name of option is performed when clicking on "Update" > button in "Options" tab > > > Key: DRILL-7287 > URL: https://issues.apache.org/jira/browse/DRILL-7287 > Project: Apache Drill > Issue Type: Bug >Reporter: Pavel Semenov >Priority: Major > Attachments: chromeSettings.png > > > *STEPS TO REPRODUCE* > # Open UI drill admin panel > # Open "Options" tab > # Click on "Update" button for any option in the list (e.g. > "debug.validate_vectors" option) > # Observe the result > *EXPECTED RESULT* > Web page should be refreshed and option value should be applied > *ACTUAL RESULT* > Search by selected name of option is performed and option value is applied > *ADDITIONAL INFO* > Behavior of "Options" tab should be similar to e.g. "Settings" in Google > Chrome browser, because it has similar controls ( see screenshot) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7287) Search by selected name of option is performed when clicking on "Update" button in "Options" tab
[ https://issues.apache.org/jira/browse/DRILL-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Semenov updated DRILL-7287: - Description: *STEPS TO REPRODUCE* # Open UI drill admin panel # Open "Options" tab # Click on "Update" button for any option in the list (e.g. "debug.validate_vectors" option) # Observe the result *EXPECTED RESULT* Web page should be refreshed and option value should be applied *ACTUAL RESULT* Search by selected name of option is performed and option value is applied *ADDITIONAL INFO* Behavior of "Options" tab should be similar to e.g. "Settings" in Google Chrome browser, because it has similar controls !chromeSettings.png|thumbnail! was: *STEPS TO REPRODUCE* # Open UI drill admin panel # Open "Options" tab # Click on "Update" button for any option in the list (e.g. "debug.validate_vectors" option) # Observe the result *EXPECTED RESULT* Web page should be refreshed and option value should be applied *ACTUAL RESULT* Search by selected name of option is performed and option value is applied *ADDITIONAL INFO* Behavior of "Options" tab should be similar to e.g. "Settings" in Google Chrome browser, because it has similar controls !chromeSettings.png! > Search by selected name of option is performed when clicking on "Update" > button in "Options" tab > > > Key: DRILL-7287 > URL: https://issues.apache.org/jira/browse/DRILL-7287 > Project: Apache Drill > Issue Type: Bug >Reporter: Pavel Semenov >Priority: Major > Attachments: chromeSettings.png > > > *STEPS TO REPRODUCE* > # Open UI drill admin panel > # Open "Options" tab > # Click on "Update" button for any option in the list (e.g. > "debug.validate_vectors" option) > # Observe the result > *EXPECTED RESULT* > Web page should be refreshed and option value should be applied > *ACTUAL RESULT* > Search by selected name of option is performed and option value is applied > *ADDITIONAL INFO* > Behavior of "Options" tab should be similar to e.g. "Settings" in Google > Chrome browser, because it has similar controls > !chromeSettings.png|thumbnail! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7287) Search by selected name of option is performed when clicking on "Update" button in "Options" tab
Pavel Semenov created DRILL-7287: Summary: Search by selected name of option is performed when clicking on "Update" button in "Options" tab Key: DRILL-7287 URL: https://issues.apache.org/jira/browse/DRILL-7287 Project: Apache Drill Issue Type: Bug Reporter: Pavel Semenov Attachments: chromeSettings.png *STEPS TO REPRODUCE* # Open UI drill admin panel # Open "Options" tab # Click on "Update" button for any option in the list (e.g. "debug.validate_vectors" option) # Observe the result *EXPECTED RESULT* Web page should be refreshed and option value should be applied *ACTUAL RESULT* Search by selected name of option is performed and option value is applied *ADDITIONAL INFO* Behavior of "Options" tab should be similar to e.g. "Settings" in Google Chrome browser, because it has similar controls !chromeSettings.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7286) Joining a table with itself using subquery results in exception.
Anton Gozhiy created DRILL-7286: --- Summary: Joining a table with itself using subquery results in exception. Key: DRILL-7286 URL: https://issues.apache.org/jira/browse/DRILL-7286 Project: Apache Drill Issue Type: Bug Affects Versions: 1.16.0 Reporter: Anton Gozhiy *Steps:* # Create some test table, like: {code:sql} create table t as select * from cp.`employee.json`; {code} # Execute the query: {code:sql} select * from (select * from t) d1 join t d2 on d1.employee_id = d2.employee_id limit 1; {code} *Expected result:* A result should be returned normally. *Actual result:* Exception happened: {noformat} Error: SYSTEM ERROR: IndexOutOfBoundsException: index (2) must be less than size (2) Please, refer to logs for more information. [Error Id: 92a5ce8e-8640-4636-a897-8f360ddf8ea3 on userf87d-pc:31010] (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception during fragment initialization: index (2) must be less than size (2) org.apache.drill.exec.work.foreman.Foreman.run():305 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748 Caused By (java.lang.IndexOutOfBoundsException) index (2) must be less than size (2) com.google.common.base.Preconditions.checkElementIndex():310 com.google.common.base.Preconditions.checkElementIndex():293 com.google.common.collect.RegularImmutableList.get():67 org.apache.calcite.util.Pair$3.get():295 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitProject():163 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitProject():44 org.apache.drill.exec.planner.physical.ProjectPrel.accept():105 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitPrel():196 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitPrel():44 org.apache.drill.exec.planner.physical.visitor.BasePrelVisitor.visitJoin():51 org.apache.drill.exec.planner.physical.JoinPrel.accept():71 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitPrel():196 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitPrel():44 org.apache.drill.exec.planner.physical.LimitPrel.accept():88 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitPrel():196 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitPrel():44 org.apache.drill.exec.planner.physical.visitor.BasePrelVisitor.visitExchange():46 org.apache.drill.exec.planner.physical.ExchangePrel.accept():36 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitPrel():196 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitPrel():44 org.apache.drill.exec.planner.physical.LimitPrel.accept():88 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitProject():157 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitProject():44 org.apache.drill.exec.planner.physical.ProjectPrel.accept():105 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitScreen():76 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.visitScreen():44 org.apache.drill.exec.planner.physical.ScreenPrel.accept():65 org.apache.drill.exec.planner.physical.visitor.StarColumnConverter.insertRenameProject():71 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPrel():513 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():178 org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():226 org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan():124 org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():90 org.apache.drill.exec.work.foreman.Foreman.runSQL():593 org.apache.drill.exec.work.foreman.Foreman.run():276 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748 (state=,code=0) {noformat} *Note:* The same query without subquery works fine: {code:sql} select * from t d1 join t d2 on d1.employee_id = d2.employee_id limit 1; {code} {noformat} +-+--++---+-++--+---++---+-+---+-+++---+--+--+-++--+-+---++-+---+-++--+-+-+---+ | employee_id | full_name | first_name | last_name | position_id | position_title |
[jira] [Created] (DRILL-7285) A temporary table has a higher priority than the cte table with the same name.
Anton Gozhiy created DRILL-7285: --- Summary: A temporary table has a higher priority than the cte table with the same name. Key: DRILL-7285 URL: https://issues.apache.org/jira/browse/DRILL-7285 Project: Apache Drill Issue Type: Bug Affects Versions: 1.16.0 Reporter: Anton Gozhiy *Steps:* # Switch to a workspace: {code:sql} use dfs.tmp {code} # Create a temporary table: {code:sql} create temporary table t as select 'temp table' as a; {code} # Run the following query: {code:sql} with t as (select 'cte' as a) select * from t; {code} *Expected result:* content from the CTE table should be returned: {noformat} ++ | a | ++ |cte | ++ {noformat} *Actual result:* the temporary table content is returned instead: {noformat} ++ | a | ++ | temp table | ++ {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7284) reusing the hashCodes computed at exchange nodes
[ https://issues.apache.org/jira/browse/DRILL-7284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857498#comment-16857498 ] weijie.tong commented on DRILL-7284: It will have a option for user to decide whether to pay for the additional network and memory cost. > reusing the hashCodes computed at exchange nodes > > > Key: DRILL-7284 > URL: https://issues.apache.org/jira/browse/DRILL-7284 > Project: Apache Drill > Issue Type: New Feature >Reporter: weijie.tong >Assignee: weijie.tong >Priority: Major > Fix For: 1.17.0 > > > To HashJoin or HashAggregate, we will shuffle the input data according to > hashCodes of join conditions or group by keys at the exchange nodes. This > computing of the hash codes will be redo at the HashJoin or HashAggregate > nodes. We could send the computed hashCodes of exchange nodes to the upper > nodes. So the HashJoin or HashAggregate nodes will not need to do the hash > computing again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7284) reusing the hashCodes computed at exchange nodes
weijie.tong created DRILL-7284: -- Summary: reusing the hashCodes computed at exchange nodes Key: DRILL-7284 URL: https://issues.apache.org/jira/browse/DRILL-7284 Project: Apache Drill Issue Type: New Feature Reporter: weijie.tong Assignee: weijie.tong Fix For: 1.17.0 To HashJoin or HashAggregate, we will shuffle the input data according to hashCodes of join conditions or group by keys at the exchange nodes. This computing of the hash codes will be redo at the HashJoin or HashAggregate nodes. We could send the computed hashCodes of exchange nodes to the upper nodes. So the HashJoin or HashAggregate nodes will not need to do the hash computing again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7279) Support provided schema for CSV without headers
[ https://issues.apache.org/jira/browse/DRILL-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857394#comment-16857394 ] Arina Ielchiieva commented on DRILL-7279: - [~Paul.Rogers] could you please update Jira description to be actual? > Support provided schema for CSV without headers > --- > > Key: DRILL-7279 > URL: https://issues.apache.org/jira/browse/DRILL-7279 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Labels: ready-to-commit > Fix For: 1.17.0 > > > Extend the Drill 1.16 provided schema support for the text reader to allow a > provided schema for files without headers. Behavior: > * If the file is configured to not extract headers, and a schema is provided, > and the schema has at least one column, then use the provided schema to > create individual columns. Otherwise, continue to use {{columns}} as in > previous versions. > * The columns in the schema are assumed to match left-to-right with those in > the file. > * If the schema contains more columns than the file, the extra columns take > their default values. (This occurs in schema evolution when a column is added > to newer files.) > * If the file contains more columns than the schema, then the extra columns, > at the end of the line, are ignored. This is the same behavior as occurs if > the file contains headers. > h4. Table Properties > Also adds four table properties for text files. These properties, if present, > override those defined in the format plugin configuration. The properties > allow the user to have a single "csv" config, but to have many tables with > the "csv" suffix, each with different properties. That is, the user need not > define a new plugin config, and define a new extension, just to change a file > format property. With this system, the user can have a ".csv" file with > headers; the user need not define a different suffix (usually ".csvh" in > Drill) for this case. > || Table Property || Equivalent Plugin Config Property || > | {{drill.headers}} | {{extractHeader}} | > | {{drill.skipFirstLine}} | {{skipFirstLine}} | > | {{drill.delimiter}} | {{fieldDelimiter}} | > | {{drill.commentChar}} | {{comment}}| > For each, the rules are: > * If the table property is not set, then the plugin property is used. > * If the table property is set, then the property value replaces the plugin > property value for that one specific table. > * For the delimiter, if the property value is an empty string, then this is > the same as an unset property. > * For the comment, if the property value is an empty string, then the comment > is set to the ASCII NULL, which will never match. This effectively turns off > the comment feature for this one table. > * If the delimiter or comment value is longer than a single character, only > the first character is used. > It is possible to use the table properties without specifying a "provided" > schema. Just omit any columns from the schema: > {noformat} > create schema () for table `dfs.data`.`example` > PROPERTIES ('drill.headers'='false', 'drill.skipFirstLine'='false', > 'drill.delimiter'='|') > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7279) Support provided schema for CSV without headers
[ https://issues.apache.org/jira/browse/DRILL-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-7279: Labels: doc-impacting ready-to-commit (was: ready-to-commit) > Support provided schema for CSV without headers > --- > > Key: DRILL-7279 > URL: https://issues.apache.org/jira/browse/DRILL-7279 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.17.0 > > > Extend the Drill 1.16 provided schema support for the text reader to allow a > provided schema for files without headers. Behavior: > * If the file is configured to not extract headers, and a schema is provided, > and the schema has at least one column, then use the provided schema to > create individual columns. Otherwise, continue to use {{columns}} as in > previous versions. > * The columns in the schema are assumed to match left-to-right with those in > the file. > * If the schema contains more columns than the file, the extra columns take > their default values. (This occurs in schema evolution when a column is added > to newer files.) > * If the file contains more columns than the schema, then the extra columns, > at the end of the line, are ignored. This is the same behavior as occurs if > the file contains headers. > h4. Table Properties > Also adds four table properties for text files. These properties, if present, > override those defined in the format plugin configuration. The properties > allow the user to have a single "csv" config, but to have many tables with > the "csv" suffix, each with different properties. That is, the user need not > define a new plugin config, and define a new extension, just to change a file > format property. With this system, the user can have a ".csv" file with > headers; the user need not define a different suffix (usually ".csvh" in > Drill) for this case. > || Table Property || Equivalent Plugin Config Property || > | {{drill.headers}} | {{extractHeader}} | > | {{drill.skipFirstLine}} | {{skipFirstLine}} | > | {{drill.delimiter}} | {{fieldDelimiter}} | > | {{drill.commentChar}} | {{comment}}| > For each, the rules are: > * If the table property is not set, then the plugin property is used. > * If the table property is set, then the property value replaces the plugin > property value for that one specific table. > * For the delimiter, if the property value is an empty string, then this is > the same as an unset property. > * For the comment, if the property value is an empty string, then the comment > is set to the ASCII NULL, which will never match. This effectively turns off > the comment feature for this one table. > * If the delimiter or comment value is longer than a single character, only > the first character is used. > It is possible to use the table properties without specifying a "provided" > schema. Just omit any columns from the schema: > {noformat} > create schema () for table `dfs.data`.`example` > PROPERTIES ('drill.headers'='false', 'drill.skipFirstLine'='false', > 'drill.delimiter'='|') > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7279) Support provided schema for CSV without headers
[ https://issues.apache.org/jira/browse/DRILL-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-7279: Labels: ready-to-commit (was: ) > Support provided schema for CSV without headers > --- > > Key: DRILL-7279 > URL: https://issues.apache.org/jira/browse/DRILL-7279 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Labels: ready-to-commit > Fix For: 1.17.0 > > > Extend the Drill 1.16 provided schema support for the text reader to allow a > provided schema for files without headers. Behavior: > * If the file is configured to not extract headers, and a schema is provided, > and the schema has at least one column, then use the provided schema to > create individual columns. Otherwise, continue to use {{columns}} as in > previous versions. > * The columns in the schema are assumed to match left-to-right with those in > the file. > * If the schema contains more columns than the file, the extra columns take > their default values. (This occurs in schema evolution when a column is added > to newer files.) > * If the file contains more columns than the schema, then the extra columns, > at the end of the line, are ignored. This is the same behavior as occurs if > the file contains headers. > h4. Table Properties > Also adds four table properties for text files. These properties, if present, > override those defined in the format plugin configuration. The properties > allow the user to have a single "csv" config, but to have many tables with > the "csv" suffix, each with different properties. That is, the user need not > define a new plugin config, and define a new extension, just to change a file > format property. With this system, the user can have a ".csv" file with > headers; the user need not define a different suffix (usually ".csvh" in > Drill) for this case. > || Table Property || Equivalent Plugin Config Property || > | {{drill.headers}} | {{extractHeader}} | > | {{drill.skipFirstLine}} | {{skipFirstLine}} | > | {{drill.delimiter}} | {{fieldDelimiter}} | > | {{drill.commentChar}} | {{comment}}| > For each, the rules are: > * If the table property is not set, then the plugin property is used. > * If the table property is set, then the property value replaces the plugin > property value for that one specific table. > * For the delimiter, if the property value is an empty string, then this is > the same as an unset property. > * For the comment, if the property value is an empty string, then the comment > is set to the ASCII NULL, which will never match. This effectively turns off > the comment feature for this one table. > * If the delimiter or comment value is longer than a single character, only > the first character is used. > It is possible to use the table properties without specifying a "provided" > schema. Just omit any columns from the schema: > {noformat} > create schema () for table `dfs.data`.`example` > PROPERTIES ('drill.headers'='false', 'drill.skipFirstLine'='false', > 'drill.delimiter'='|') > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7279) Support provided schema for CSV without headers
[ https://issues.apache.org/jira/browse/DRILL-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857393#comment-16857393 ] ASF GitHub Bot commented on DRILL-7279: --- arina-ielchiieva commented on issue #1798: DRILL-7279: Enable provided schema for text files without headers URL: https://github.com/apache/drill/pull/1798#issuecomment-499390818 Thanks for adding all text properties. +1, LGTM. Please squash the commits. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Support provided schema for CSV without headers > --- > > Key: DRILL-7279 > URL: https://issues.apache.org/jira/browse/DRILL-7279 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Fix For: 1.17.0 > > > Extend the Drill 1.16 provided schema support for the text reader to allow a > provided schema for files without headers. Behavior: > * If the file is configured to not extract headers, and a schema is provided, > and the schema has at least one column, then use the provided schema to > create individual columns. Otherwise, continue to use {{columns}} as in > previous versions. > * The columns in the schema are assumed to match left-to-right with those in > the file. > * If the schema contains more columns than the file, the extra columns take > their default values. (This occurs in schema evolution when a column is added > to newer files.) > * If the file contains more columns than the schema, then the extra columns, > at the end of the line, are ignored. This is the same behavior as occurs if > the file contains headers. > h4. Table Properties > Also adds four table properties for text files. These properties, if present, > override those defined in the format plugin configuration. The properties > allow the user to have a single "csv" config, but to have many tables with > the "csv" suffix, each with different properties. That is, the user need not > define a new plugin config, and define a new extension, just to change a file > format property. With this system, the user can have a ".csv" file with > headers; the user need not define a different suffix (usually ".csvh" in > Drill) for this case. > || Table Property || Equivalent Plugin Config Property || > | {{drill.headers}} | {{extractHeader}} | > | {{drill.skipFirstLine}} | {{skipFirstLine}} | > | {{drill.delimiter}} | {{fieldDelimiter}} | > | {{drill.commentChar}} | {{comment}}| > For each, the rules are: > * If the table property is not set, then the plugin property is used. > * If the table property is set, then the property value replaces the plugin > property value for that one specific table. > * For the delimiter, if the property value is an empty string, then this is > the same as an unset property. > * For the comment, if the property value is an empty string, then the comment > is set to the ASCII NULL, which will never match. This effectively turns off > the comment feature for this one table. > * If the delimiter or comment value is longer than a single character, only > the first character is used. > It is possible to use the table properties without specifying a "provided" > schema. Just omit any columns from the schema: > {noformat} > create schema () for table `dfs.data`.`example` > PROPERTIES ('drill.headers'='false', 'drill.skipFirstLine'='false', > 'drill.delimiter'='|') > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7278) Refactor result set loader projection mechanism
[ https://issues.apache.org/jira/browse/DRILL-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-7278: Labels: ready-to-commit (was: ) > Refactor result set loader projection mechanism > --- > > Key: DRILL-7278 > URL: https://issues.apache.org/jira/browse/DRILL-7278 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Labels: ready-to-commit > Fix For: 1.17.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7278) Refactor result set loader projection mechanism
[ https://issues.apache.org/jira/browse/DRILL-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857388#comment-16857388 ] ASF GitHub Bot commented on DRILL-7278: --- arina-ielchiieva commented on issue #1797: DRILL-7278: Refactor result set loader projection mechanism URL: https://github.com/apache/drill/pull/1797#issuecomment-499389130 +1, LGTM. Please squash the commits. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refactor result set loader projection mechanism > --- > > Key: DRILL-7278 > URL: https://issues.apache.org/jira/browse/DRILL-7278 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Fix For: 1.17.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7261) Simplify Easy format config for new scan framework
[ https://issues.apache.org/jira/browse/DRILL-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857386#comment-16857386 ] ASF GitHub Bot commented on DRILL-7261: --- arina-ielchiieva commented on issue #1796: DRILL-7261: Simplify Easy framework config URL: https://github.com/apache/drill/pull/1796#issuecomment-499388249 +1, please squash the commits. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Simplify Easy format config for new scan framework > -- > > Key: DRILL-7261 > URL: https://issues.apache.org/jira/browse/DRILL-7261 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.16.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Labels: ready-to-commit > Fix For: 1.17.0 > > > Rollup of related CSV V3 fixes along with supporting row set framework fixes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)