[
https://issues.apache.org/jira/browse/IMPALA-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy updated IMPALA-12843:
---------------------------------------
Description:
Repro:
COPY TESTCASE TO '/some/random/path' <SELECT query that involves Iceberg table>;
COPY TESTCASE FROM '/some/random/path';
The latter throws a NullPointerException:
{noformat}
[1] java.util.Objects.requireNonNull (Objects.java:203)
[2] java.util.Collections$SynchronizedCollection.<init>
(Collections.java:2,007)
[3] java.util.Collections$SynchronizedList.<init> (Collections.java:2,401)
[4] java.util.Collections.synchronizedList (Collections.java:2,379)
[5] org.apache.impala.util.ListMap.populate (ListMap.java:78)
[6] org.apache.impala.catalog.HdfsTable.loadFromThrift (HdfsTable.java:2,033)
[7] org.apache.impala.catalog.IcebergTable.loadFromThrift
(IcebergTable.java:542)
[8] org.apache.impala.catalog.Table.fromThrift (Table.java:579)
[9] org.apache.impala.service.CatalogOpExecutor.copyTestCaseData
(CatalogOpExecutor.java:708)
[10] org.apache.impala.service.CatalogOpExecutor.execDdlRequest
(CatalogOpExecutor.java:612)
[11] org.apache.impala.service.JniCatalog.lambda$execDdl$3
(JniCatalog.java:308)
[12] org.apache.impala.service.JniCatalog$$Lambda$281.910241970.call (null)
[13] org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1
(JniCatalogOp.java:90)
[14] org.apache.impala.service.JniCatalogOp$$Lambda$38.1978862713.call (null)
[15] org.apache.impala.service.JniCatalogOp.execOp (JniCatalogOp.java:58)
[16] org.apache.impala.service.JniCatalogOp.execAndSerialize
(JniCatalogOp.java:89)
[17] org.apache.impala.service.JniCatalogOp.execAndSerialize
(JniCatalogOp.java:100)
[18] org.apache.impala.service.JniCatalog.execAndSerialize
(JniCatalog.java:237)
[19] org.apache.impala.service.JniCatalog.execAndSerialize
(JniCatalog.java:251)
[20] org.apache.impala.service.JniCatalog.execDdl (JniCatalog.java:307)
Thread-10 [COPY_TESTCASE issued by boroknagyz][1] Input stream closed.
{noformat}
There are several problems here, so this might not be doable for Iceberg
tables, at least not easily. E.g. COPY TESTCASE FROM only creates the tables in
the Coordinator's memory, but Iceberg tables require to have a valid metadata
JSON file on the storage system.
was:
Repro:
COPY TESTCASE TO '/some/random/path' <SELECT query that involves Iceberg table>;
COPY TESTCASE FROM '/some/random/path';
The latter throws a NullPointerException:
{noformat}
[1] java.util.Objects.requireNonNull (Objects.java:203)
[2] java.util.Collections$SynchronizedCollection.<init>
(Collections.java:2,007)
[3] java.util.Collections$SynchronizedList.<init> (Collections.java:2,401)
[4] java.util.Collections.synchronizedList (Collections.java:2,379)
[5] org.apache.impala.util.ListMap.populate (ListMap.java:78)
[6] org.apache.impala.catalog.HdfsTable.loadFromThrift (HdfsTable.java:2,033)
[7] org.apache.impala.catalog.IcebergTable.loadFromThrift
(IcebergTable.java:542)
[8] org.apache.impala.catalog.Table.fromThrift (Table.java:579)
[9] org.apache.impala.service.CatalogOpExecutor.copyTestCaseData
(CatalogOpExecutor.java:708)
[10] org.apache.impala.service.CatalogOpExecutor.execDdlRequest
(CatalogOpExecutor.java:612)
[11] org.apache.impala.service.JniCatalog.lambda$execDdl$3
(JniCatalog.java:308)
[12] org.apache.impala.service.JniCatalog$$Lambda$281.910241970.call (null)
[13] org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1
(JniCatalogOp.java:90)
[14] org.apache.impala.service.JniCatalogOp$$Lambda$38.1978862713.call (null)
[15] org.apache.impala.service.JniCatalogOp.execOp (JniCatalogOp.java:58)
[16] org.apache.impala.service.JniCatalogOp.execAndSerialize
(JniCatalogOp.java:89)
[17] org.apache.impala.service.JniCatalogOp.execAndSerialize
(JniCatalogOp.java:100)
[18] org.apache.impala.service.JniCatalog.execAndSerialize
(JniCatalog.java:237)
[19] org.apache.impala.service.JniCatalog.execAndSerialize
(JniCatalog.java:251)
[20] org.apache.impala.service.JniCatalog.execDdl (JniCatalog.java:307)
Thread-10 [COPY_TESTCASE issued by boroknagyz][1] Input stream closed.
{noformat}
> COPY TESTCASE is broken for Iceberg tables
> ------------------------------------------
>
> Key: IMPALA-12843
> URL: https://issues.apache.org/jira/browse/IMPALA-12843
> Project: IMPALA
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
>
> Repro:
> COPY TESTCASE TO '/some/random/path' <SELECT query that involves Iceberg
> table>;
> COPY TESTCASE FROM '/some/random/path';
> The latter throws a NullPointerException:
> {noformat}
> [1] java.util.Objects.requireNonNull (Objects.java:203)
> [2] java.util.Collections$SynchronizedCollection.<init>
> (Collections.java:2,007)
> [3] java.util.Collections$SynchronizedList.<init> (Collections.java:2,401)
> [4] java.util.Collections.synchronizedList (Collections.java:2,379)
> [5] org.apache.impala.util.ListMap.populate (ListMap.java:78)
> [6] org.apache.impala.catalog.HdfsTable.loadFromThrift
> (HdfsTable.java:2,033)
> [7] org.apache.impala.catalog.IcebergTable.loadFromThrift
> (IcebergTable.java:542)
> [8] org.apache.impala.catalog.Table.fromThrift (Table.java:579)
> [9] org.apache.impala.service.CatalogOpExecutor.copyTestCaseData
> (CatalogOpExecutor.java:708)
> [10] org.apache.impala.service.CatalogOpExecutor.execDdlRequest
> (CatalogOpExecutor.java:612)
> [11] org.apache.impala.service.JniCatalog.lambda$execDdl$3
> (JniCatalog.java:308)
> [12] org.apache.impala.service.JniCatalog$$Lambda$281.910241970.call (null)
> [13] org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1
> (JniCatalogOp.java:90)
> [14] org.apache.impala.service.JniCatalogOp$$Lambda$38.1978862713.call
> (null)
> [15] org.apache.impala.service.JniCatalogOp.execOp (JniCatalogOp.java:58)
> [16] org.apache.impala.service.JniCatalogOp.execAndSerialize
> (JniCatalogOp.java:89)
> [17] org.apache.impala.service.JniCatalogOp.execAndSerialize
> (JniCatalogOp.java:100)
> [18] org.apache.impala.service.JniCatalog.execAndSerialize
> (JniCatalog.java:237)
> [19] org.apache.impala.service.JniCatalog.execAndSerialize
> (JniCatalog.java:251)
> [20] org.apache.impala.service.JniCatalog.execDdl (JniCatalog.java:307)
> Thread-10 [COPY_TESTCASE issued by boroknagyz][1] Input stream closed.
> {noformat}
> There are several problems here, so this might not be doable for Iceberg
> tables, at least not easily. E.g. COPY TESTCASE FROM only creates the tables
> in the Coordinator's memory, but Iceberg tables require to have a valid
> metadata JSON file on the storage system.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
