[jira] [Created] (DRILL-6682) Cast integer to binary returns incorrect result
Robert Hou created DRILL-6682: - Summary: Cast integer to binary returns incorrect result Key: DRILL-6682 URL: https://issues.apache.org/jira/browse/DRILL-6682 Project: Apache Drill Issue Type: Bug Components: Execution - Relational Operators Affects Versions: 1.12.0 Reporter: Robert Hou Assignee: Pritesh Maker This query returns an empty binary string: select cast(123 as binary) from (values(1)); The same problem occurs for bigint, float and double. Casting works if the data type is date, time, timestamp, interval, varchar and binary. select cast(date '2018-08-10' as binary) from (values(1)); select length(string_binary(cast(123 as binary))), length(string_binary(cast(date '2018-08-10' as binary))) from (values(1)); +-+-+ | EXPR$0 | EXPR$1 | +-+-+ | 0 | 10 | +-+-+ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6453) TPC-DS query 72 has regressed
[
https://issues.apache.org/jira/browse/DRILL-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Farkas updated DRILL-6453:
--
Labels: ready-to-commit (was: )
> TPC-DS query 72 has regressed
> -
>
> Key: DRILL-6453
> URL: https://issues.apache.org/jira/browse/DRILL-6453
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.14.0
>Reporter: Khurram Faraaz
>Assignee: Timothy Farkas
>Priority: Blocker
> Labels: ready-to-commit
> Fix For: 1.15.0
>
> Attachments: 24f75b18-014a-fb58-21d2-baeab5c3352c.sys.drill,
> jstack_29173_June_10_2018.txt, jstack_29173_June_10_2018.txt,
> jstack_29173_June_10_2018_b.txt, jstack_29173_June_10_2018_b.txt,
> jstack_29173_June_10_2018_c.txt, jstack_29173_June_10_2018_c.txt,
> jstack_29173_June_10_2018_d.txt, jstack_29173_June_10_2018_d.txt,
> jstack_29173_June_10_2018_e.txt, jstack_29173_June_10_2018_e.txt
>
>
> TPC-DS query 72 seems to have regressed, query profile for the case where it
> Canceled after 2 hours on Drill 1.14.0 is attached here.
> {noformat}
> On, Drill 1.14.0-SNAPSHOT
> commit : 931b43e (TPC-DS query 72 executed successfully on this commit, took
> around 55 seconds to execute)
> SF1 parquet data on 4 nodes;
> planner.memory.max_query_memory_per_node = 10737418240.
> drill.exec.hashagg.fallback.enabled = true
> TPC-DS query 72 executed successfully & took 47 seconds to complete execution.
> {noformat}
> {noformat}
> TPC-DS data in the below run has date values stored as DATE datatype and not
> VARCHAR type
> On, Drill 1.14.0-SNAPSHOT
> commit : 82e1a12
> SF1 parquet data on 4 nodes;
> planner.memory.max_query_memory_per_node = 10737418240.
> drill.exec.hashagg.fallback.enabled = true
> and
> alter system set `exec.hashjoin.num_partitions` = 1;
> TPC-DS query 72 executed for 2 hrs and 11 mins and did not complete, I had to
> Cancel it by stopping the Foreman drillbit.
> As a result several minor fragments are reported to be in
> CANCELLATION_REQUESTED state on UI.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6680) Expose SHOW FILES command into INFORMATION_SCHEMA
[ https://issues.apache.org/jira/browse/DRILL-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6680: Reviewer: Volodymyr Vysotskyi > Expose SHOW FILES command into INFORMATION_SCHEMA > - > > Key: DRILL-6680 > URL: https://issues.apache.org/jira/browse/DRILL-6680 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.14.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Major > Labels: doc-impacting > Fix For: 1.15.0 > > > Link to design document - > https://docs.google.com/document/d/1UnvATwH4obn1-XsA83xMz3LtylbMu867eBLN9r3eV3k/edit# -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6680) Expose SHOW FILES command into INFORMATION_SCHEMA
[ https://issues.apache.org/jira/browse/DRILL-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6680: Labels: doc-impacting (was: ) > Expose SHOW FILES command into INFORMATION_SCHEMA > - > > Key: DRILL-6680 > URL: https://issues.apache.org/jira/browse/DRILL-6680 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.14.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Major > Labels: doc-impacting > Fix For: 1.15.0 > > > Link to design document - > https://docs.google.com/document/d/1UnvATwH4obn1-XsA83xMz3LtylbMu867eBLN9r3eV3k/edit# -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6680) Expose SHOW FILES command into INFORMATION_SCHEMA
[ https://issues.apache.org/jira/browse/DRILL-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576560#comment-16576560 ] ASF GitHub Bot commented on DRILL-6680: --- arina-ielchiieva opened a new pull request #1430: DRILL-6680: Expose show files command into INFORMATION_SCHEMA URL: https://github.com/apache/drill/pull/1430 Details in [DRILL-6680](https://issues.apache.org/jira/browse/DRILL-6680). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Expose SHOW FILES command into INFORMATION_SCHEMA > - > > Key: DRILL-6680 > URL: https://issues.apache.org/jira/browse/DRILL-6680 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.14.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Major > Fix For: 1.15.0 > > > Link to design document - > https://docs.google.com/document/d/1UnvATwH4obn1-XsA83xMz3LtylbMu867eBLN9r3eV3k/edit# -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6671) Multi level lateral unnest join is throwing an exception during materializing the plan.
[ https://issues.apache.org/jira/browse/DRILL-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated DRILL-6671: --- Fix Version/s: 1.15.0 > Multi level lateral unnest join is throwing an exception during materializing > the plan. > --- > > Key: DRILL-6671 > URL: https://issues.apache.org/jira/browse/DRILL-6671 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.15.0 >Reporter: Hanumath Rao Maduri >Assignee: Hanumath Rao Maduri >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > > testMultiUnnestAtSameLevel in TestE2EUnnestAndLateral is throwing an > execution in Materializer.java. This is due to incorrect matching of Unnest > and Lateral join. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6671) Multi level lateral unnest join is throwing an exception during materializing the plan.
[ https://issues.apache.org/jira/browse/DRILL-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated DRILL-6671: --- Labels: ready-to-commit (was: ) Reviewer: Volodymyr Vysotskyi > Multi level lateral unnest join is throwing an exception during materializing > the plan. > --- > > Key: DRILL-6671 > URL: https://issues.apache.org/jira/browse/DRILL-6671 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning Optimization >Affects Versions: 1.15.0 >Reporter: Hanumath Rao Maduri >Assignee: Hanumath Rao Maduri >Priority: Major > Labels: ready-to-commit > > testMultiUnnestAtSameLevel in TestE2EUnnestAndLateral is throwing an > execution in Materializer.java. This is due to incorrect matching of Unnest > and Lateral join. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-3637) Elasticsearch storage plugin
[ https://issues.apache.org/jira/browse/DRILL-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576355#comment-16576355 ] Charles Givre commented on DRILL-3637: -- Hi [~arina] I was doing some research and stumbled on the Calcite adapters ([https://calcite.apache.org/docs/adapter.html)]. I know Drill uses its own version of Calcite for the time being, but would it make sense to use these adapters to build storage plugins? I was thinking of attempting that for a Splunk storage plugin. If this would simplify the process, it would be nice to create plugins for Spark, Cassandra, Druid and Geode. > Elasticsearch storage plugin > > > Key: DRILL-3637 > URL: https://issues.apache.org/jira/browse/DRILL-3637 > Project: Apache Drill > Issue Type: New Feature > Components: Storage - ElasticSearch >Reporter: Andrew >Assignee: Arina Ielchiieva >Priority: Major > Fix For: Future > > > Create a storage plugin for elasticsearch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6681) Add support for SHOW VIEWS statement;
[ https://issues.apache.org/jira/browse/DRILL-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Gozhiy updated DRILL-6681: Description: Currently, SHOW TABLES statement displays both tables and views. So it would be good to have SHOW VIEWS command to list them separately. (was: Add ability to list views similar to "show tables".) > Add support for SHOW VIEWS statement; > - > > Key: DRILL-6681 > URL: https://issues.apache.org/jira/browse/DRILL-6681 > Project: Apache Drill > Issue Type: New Feature >Reporter: Anton Gozhiy >Priority: Major > > Currently, SHOW TABLES statement displays both tables and views. So it would > be good to have SHOW VIEWS command to list them separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6681) Add support for SHOW VIEWS statement;
Anton Gozhiy created DRILL-6681: --- Summary: Add support for SHOW VIEWS statement; Key: DRILL-6681 URL: https://issues.apache.org/jira/browse/DRILL-6681 Project: Apache Drill Issue Type: New Feature Reporter: Anton Gozhiy Add ability to list views similar to "show tables". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6680) Expose SHOW FILES command into INFORMATION_SCHEMA
Arina Ielchiieva created DRILL-6680: --- Summary: Expose SHOW FILES command into INFORMATION_SCHEMA Key: DRILL-6680 URL: https://issues.apache.org/jira/browse/DRILL-6680 Project: Apache Drill Issue Type: New Feature Affects Versions: 1.14.0 Reporter: Arina Ielchiieva Assignee: Arina Ielchiieva Fix For: 1.15.0 Link to design document - https://docs.google.com/document/d/1UnvATwH4obn1-XsA83xMz3LtylbMu867eBLN9r3eV3k/edit# -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6671) Multi level lateral unnest join is throwing an exception during materializing the plan.
[
https://issues.apache.org/jira/browse/DRILL-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576332#comment-16576332
]
ASF GitHub Bot commented on DRILL-6671:
---
HanumathRao commented on a change in pull request #1426: DRILL-6671: Multi
level lateral unnest join is throwing an exception …
URL: https://github.com/apache/drill/pull/1426#discussion_r209274566
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/LateralUnnestRowIDVisitor.java
##
@@ -59,16 +66,55 @@ public Prel visitPrel(Prel prel, Boolean isRightOfLateral)
throws RuntimeExcepti
}
@Override
- public Prel visitLateral(LateralJoinPrel prel, Boolean value) throws
RuntimeException {
+ public Prel visitLateral(LateralJoinPrel prel, Boolean isRightOfLateral)
throws RuntimeException {
List children = Lists.newArrayList();
-children.add(((Prel)prel.getInput(0)).accept(this, value));
+children.add(((Prel) prel.getInput(0)).accept(this, isRightOfLateral));
children.add(((Prel) prel.getInput(1)).accept(this, true));
-return (Prel) prel.copy(prel.getTraitSet(), children);
+if (!isRightOfLateral) {
+ return (Prel) prel.copy(prel.getTraitSet(), children);
+} else {
+ //Adjust the column numbering due to an additional column
"$drill_implicit_field$" is added to the inputs.
+ Map requiredColsMap = new HashMap<>();
+ for (Integer corrColIndex : prel.getRequiredColumns()) {
+requiredColsMap.put(corrColIndex, corrColIndex + 1);
+ }
+ ImmutableBitSet requiredColumns = prel.getRequiredColumns().shift(1);
+
+ CorrelationId corrId = prel.getCluster().createCorrel();
+ RexCorrelVariable rexCorrel =
+ (RexCorrelVariable) prel.getCluster().getRexBuilder().makeCorrel(
+ children.get(0).getRowType(),
+ corrId);
+ RelNode rightChild = children.get(1).accept(
+ new CorrelateVarReplacer(
+ new
ProjectCorrelateTransposeRule.RexFieldAccessReplacer(corrId,
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Multi level lateral unnest join is throwing an exception during materializing
> the plan.
> ---
>
> Key: DRILL-6671
> URL: https://issues.apache.org/jira/browse/DRILL-6671
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning Optimization
>Affects Versions: 1.15.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
>
> testMultiUnnestAtSameLevel in TestE2EUnnestAndLateral is throwing an
> execution in Materializer.java. This is due to incorrect matching of Unnest
> and Lateral join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6670) Error in parquet record reader - previously readable file fails to be read in 1.14
[
https://issues.apache.org/jira/browse/DRILL-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576103#comment-16576103
]
ASF GitHub Bot commented on DRILL-6670:
---
okalinin commented on a change in pull request #1428: DRILL-6670: align Parquet
TIMESTAMP_MICROS logical type handling with earlier versions + minor fixes
URL: https://github.com/apache/drill/pull/1428#discussion_r209200022
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/AbstractParquetScanBatchCreator.java
##
@@ -108,7 +108,7 @@ protected ScanBatch getBatch(ExecutorFragmentContext
context, AbstractParquetRow
if
(!context.getOptions().getBoolean(ExecConstants.PARQUET_NEW_RECORD_READER)
&& !ParquetReaderUtility.containsComplexColumn(footer,
rowGroupScan.getColumns())) {
- logger.debug("Query {} qualifies for new Parquet reader",
+ logger.debug("Query {} qualifies for ParquetRecordReader",
Review comment:
Thanks for the suggestion. Done.
Instead of 'enabled/disabled' I used 'enforced/not enforced' though, because
it would be confusing to see two subsequent debug lines like:
```
... PARQUET_NEW_RECORD_READER disabled ...
... Query ... uses DrillParquetReader ...
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Error in parquet record reader - previously readable file fails to be read in
> 1.14
> --
>
> Key: DRILL-6670
> URL: https://issues.apache.org/jira/browse/DRILL-6670
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.14.0
>Reporter: Dave Challis
>Assignee: Oleksandr Kalinin
>Priority: Major
> Fix For: 1.15.0
>
> Attachments: example.parquet
>
>
> Parquet file which was generated by PyArrow was readable in Apache Drill 1.12
> and 1.13, but fails to be read with 1.14.
> Running the query "SELECT * FROM dfs.`foo.parquet`" results in the following
> error message from the Drill web query UI:
> {code}
> Query Failed: An Error Occurred
> org.apache.drill.common.exceptions.UserRemoteException: INTERNAL_ERROR ERROR:
> Error in parquet record reader. Message: Failure in setting up reader Parquet
> Metadata: ParquetMetaData{FileMetaData{schema: message schema { optional
> binary name (UTF8); optional binary creation_parameters (UTF8); optional
> int64 creation_date (TIMESTAMP_MICROS); optional int32 data_version; optional
> int32 schema_version; } , metadata: {pandas={"index_columns": [],
> "column_indexes": [], "columns": [{"name": "name", "field_name": "name",
> "pandas_type": "unicode", "numpy_type": "object", "metadata": null}, {"name":
> "creation_parameters", "field_name": "creation_parameters", "pandas_type":
> "unicode", "numpy_type": "object", "metadata": null}, {"name":
> "creation_date", "field_name": "creation_date", "pandas_type": "datetime",
> "numpy_type": "datetime64[ns]", "metadata": null}, {"name": "data_version",
> "field_name": "data_version", "pandas_type": "int32", "numpy_type": "int32",
> "metadata": null}, {"name": "schema_version", "field_name": "schema_version",
> "pandas_type": "int32", "numpy_type": "int32", "metadata": null}],
> "pandas_version": "0.22.0"}}}, blocks: [BlockMetaData{1, 27142
> [ColumnMetaData{SNAPPY [name] optional binary name (UTF8) [PLAIN, RLE], 4},
> ColumnMetaData{SNAPPY [creation_parameters] optional binary
> creation_parameters (UTF8) [PLAIN, RLE], 252}, ColumnMetaData{SNAPPY
> [creation_date] optional int64 creation_date (TIMESTAMP_MICROS) [PLAIN, RLE],
> 46334}, ColumnMetaData{SNAPPY [data_version] optional int32 data_version
> [PLAIN, RLE], 46478}, ColumnMetaData{SNAPPY [schema_version] optional int32
> schema_version [PLAIN, RLE], 46593}]}]} Fragment 0:0 [Error Id:
> bdb2e4d5-5982-4cc6-b95e-244782f827d2 on f9d0456cddd2:31010]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DRILL-6679) Error should be displayed when trying to connect Drill to unsupported version of Hive
[ https://issues.apache.org/jira/browse/DRILL-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva reassigned DRILL-6679: --- Assignee: Bohdan Kazydub > Error should be displayed when trying to connect Drill to unsupported version > of Hive > - > > Key: DRILL-6679 > URL: https://issues.apache.org/jira/browse/DRILL-6679 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.14.0 >Reporter: Anton Gozhiy >Assignee: Bohdan Kazydub >Priority: Major > Fix For: 1.15.0 > > > For example, there is no backward compatibility between Hive 2.3 and Hive > 2.1. But it is possible to connect Drill with Hive 2.3 client to Hive 2.1, it > just won't work correctly. So I suggest that enabling Hive storage plugin > should not be allowed if Hive version is unsupported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6679) Error should be displayed when trying to connect Drill to unsupported version of Hive
[ https://issues.apache.org/jira/browse/DRILL-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6679: Fix Version/s: 1.15.0 > Error should be displayed when trying to connect Drill to unsupported version > of Hive > - > > Key: DRILL-6679 > URL: https://issues.apache.org/jira/browse/DRILL-6679 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.14.0 >Reporter: Anton Gozhiy >Assignee: Bohdan Kazydub >Priority: Major > Fix For: 1.15.0 > > > For example, there is no backward compatibility between Hive 2.3 and Hive > 2.1. But it is possible to connect Drill with Hive 2.3 client to Hive 2.1, it > just won't work correctly. So I suggest that enabling Hive storage plugin > should not be allowed if Hive version is unsupported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6679) Error should be displayed when trying to connect Drill to unsupported version of Hive
Anton Gozhiy created DRILL-6679: --- Summary: Error should be displayed when trying to connect Drill to unsupported version of Hive Key: DRILL-6679 URL: https://issues.apache.org/jira/browse/DRILL-6679 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.14.0 Reporter: Anton Gozhiy For example, there is no backward compatibility between Hive 2.3 and Hive 2.1. But it is possible to connect Drill with Hive 2.3 client to Hive 2.1, it just won't work correctly. So I suggest that enabling Hive storage plugin should not be allowed if Hive version is unsupported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6670) Error in parquet record reader - previously readable file fails to be read in 1.14
[
https://issues.apache.org/jira/browse/DRILL-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576010#comment-16576010
]
ASF GitHub Bot commented on DRILL-6670:
---
okalinin commented on a change in pull request #1428: DRILL-6670: align Parquet
TIMESTAMP_MICROS logical type handling with earlier versions + minor fixes
URL: https://github.com/apache/drill/pull/1428#discussion_r209200022
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/AbstractParquetScanBatchCreator.java
##
@@ -108,7 +108,7 @@ protected ScanBatch getBatch(ExecutorFragmentContext
context, AbstractParquetRow
if
(!context.getOptions().getBoolean(ExecConstants.PARQUET_NEW_RECORD_READER)
&& !ParquetReaderUtility.containsComplexColumn(footer,
rowGroupScan.getColumns())) {
- logger.debug("Query {} qualifies for new Parquet reader",
+ logger.debug("Query {} qualifies for ParquetRecordReader",
Review comment:
Thanks for the suggestion. Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Error in parquet record reader - previously readable file fails to be read in
> 1.14
> --
>
> Key: DRILL-6670
> URL: https://issues.apache.org/jira/browse/DRILL-6670
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.14.0
>Reporter: Dave Challis
>Assignee: Oleksandr Kalinin
>Priority: Major
> Fix For: 1.15.0
>
> Attachments: example.parquet
>
>
> Parquet file which was generated by PyArrow was readable in Apache Drill 1.12
> and 1.13, but fails to be read with 1.14.
> Running the query "SELECT * FROM dfs.`foo.parquet`" results in the following
> error message from the Drill web query UI:
> {code}
> Query Failed: An Error Occurred
> org.apache.drill.common.exceptions.UserRemoteException: INTERNAL_ERROR ERROR:
> Error in parquet record reader. Message: Failure in setting up reader Parquet
> Metadata: ParquetMetaData{FileMetaData{schema: message schema { optional
> binary name (UTF8); optional binary creation_parameters (UTF8); optional
> int64 creation_date (TIMESTAMP_MICROS); optional int32 data_version; optional
> int32 schema_version; } , metadata: {pandas={"index_columns": [],
> "column_indexes": [], "columns": [{"name": "name", "field_name": "name",
> "pandas_type": "unicode", "numpy_type": "object", "metadata": null}, {"name":
> "creation_parameters", "field_name": "creation_parameters", "pandas_type":
> "unicode", "numpy_type": "object", "metadata": null}, {"name":
> "creation_date", "field_name": "creation_date", "pandas_type": "datetime",
> "numpy_type": "datetime64[ns]", "metadata": null}, {"name": "data_version",
> "field_name": "data_version", "pandas_type": "int32", "numpy_type": "int32",
> "metadata": null}, {"name": "schema_version", "field_name": "schema_version",
> "pandas_type": "int32", "numpy_type": "int32", "metadata": null}],
> "pandas_version": "0.22.0"}}}, blocks: [BlockMetaData{1, 27142
> [ColumnMetaData{SNAPPY [name] optional binary name (UTF8) [PLAIN, RLE], 4},
> ColumnMetaData{SNAPPY [creation_parameters] optional binary
> creation_parameters (UTF8) [PLAIN, RLE], 252}, ColumnMetaData{SNAPPY
> [creation_date] optional int64 creation_date (TIMESTAMP_MICROS) [PLAIN, RLE],
> 46334}, ColumnMetaData{SNAPPY [data_version] optional int32 data_version
> [PLAIN, RLE], 46478}, ColumnMetaData{SNAPPY [schema_version] optional int32
> schema_version [PLAIN, RLE], 46593}]}]} Fragment 0:0 [Error Id:
> bdb2e4d5-5982-4cc6-b95e-244782f827d2 on f9d0456cddd2:31010]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6671) Multi level lateral unnest join is throwing an exception during materializing the plan.
[
https://issues.apache.org/jira/browse/DRILL-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575914#comment-16575914
]
ASF GitHub Bot commented on DRILL-6671:
---
vvysotskyi commented on a change in pull request #1426: DRILL-6671: Multi level
lateral unnest join is throwing an exception …
URL: https://github.com/apache/drill/pull/1426#discussion_r209177260
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/LateralUnnestRowIDVisitor.java
##
@@ -59,16 +66,55 @@ public Prel visitPrel(Prel prel, Boolean isRightOfLateral)
throws RuntimeExcepti
}
@Override
- public Prel visitLateral(LateralJoinPrel prel, Boolean value) throws
RuntimeException {
+ public Prel visitLateral(LateralJoinPrel prel, Boolean isRightOfLateral)
throws RuntimeException {
List children = Lists.newArrayList();
-children.add(((Prel)prel.getInput(0)).accept(this, value));
+children.add(((Prel) prel.getInput(0)).accept(this, isRightOfLateral));
children.add(((Prel) prel.getInput(1)).accept(this, true));
-return (Prel) prel.copy(prel.getTraitSet(), children);
+if (!isRightOfLateral) {
+ return (Prel) prel.copy(prel.getTraitSet(), children);
+} else {
+ //Adjust the column numbering due to an additional column
"$drill_implicit_field$" is added to the inputs.
+ Map requiredColsMap = new HashMap<>();
+ for (Integer corrColIndex : prel.getRequiredColumns()) {
+requiredColsMap.put(corrColIndex, corrColIndex + 1);
+ }
+ ImmutableBitSet requiredColumns = prel.getRequiredColumns().shift(1);
+
+ CorrelationId corrId = prel.getCluster().createCorrel();
+ RexCorrelVariable rexCorrel =
+ (RexCorrelVariable) prel.getCluster().getRexBuilder().makeCorrel(
+ children.get(0).getRowType(),
+ corrId);
+ RelNode rightChild = children.get(1).accept(
+ new CorrelateVarReplacer(
+ new
ProjectCorrelateTransposeRule.RexFieldAccessReplacer(corrId,
Review comment:
The first argument in `RexFieldAccessReplacer` is a `CorrelationId` which
should be replaced, but you have specified the new one.
Please replace here `corrId` with `prel.getCorrelationId()`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Multi level lateral unnest join is throwing an exception during materializing
> the plan.
> ---
>
> Key: DRILL-6671
> URL: https://issues.apache.org/jira/browse/DRILL-6671
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning Optimization
>Affects Versions: 1.15.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
>Priority: Major
>
> testMultiUnnestAtSameLevel in TestE2EUnnestAndLateral is throwing an
> execution in Materializer.java. This is due to incorrect matching of Unnest
> and Lateral join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DRILL-6656) Add Regex To Disallow Extra Semicolons In Imports
[ https://issues.apache.org/jira/browse/DRILL-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi reassigned DRILL-6656: -- Assignee: Timothy Farkas (was: Volodymyr Vysotskyi) > Add Regex To Disallow Extra Semicolons In Imports > - > > Key: DRILL-6656 > URL: https://issues.apache.org/jira/browse/DRILL-6656 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6656) Add Regex To Disallow Extra Semicolons In Imports
[ https://issues.apache.org/jira/browse/DRILL-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated DRILL-6656: --- Reviewer: Volodymyr Vysotskyi (was: Arina Ielchiieva) > Add Regex To Disallow Extra Semicolons In Imports > - > > Key: DRILL-6656 > URL: https://issues.apache.org/jira/browse/DRILL-6656 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6656) Add Regex To Disallow Extra Semicolons In Imports
[ https://issues.apache.org/jira/browse/DRILL-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated DRILL-6656: --- Labels: ready-to-commit (was: ) > Add Regex To Disallow Extra Semicolons In Imports > - > > Key: DRILL-6656 > URL: https://issues.apache.org/jira/browse/DRILL-6656 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Timothy Farkas >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-6656) Add Regex To Disallow Extra Semicolons In Imports
[ https://issues.apache.org/jira/browse/DRILL-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi reassigned DRILL-6656: -- Assignee: Volodymyr Vysotskyi (was: Timothy Farkas) > Add Regex To Disallow Extra Semicolons In Imports > - > > Key: DRILL-6656 > URL: https://issues.apache.org/jira/browse/DRILL-6656 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Volodymyr Vysotskyi >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
