[ https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903351#comment-16903351 ]
Tim Armstrong commented on IMPALA-8849: --------------------------------------- So it looks like it only happens when stats are computed and the column is a VARCHAR. {noformat} [localhost:21000] tpcds> use tpcds_10000_text; Query: use tpcds_10000_text [localhost:21000] tpcds_10000_text> explain select c_customer_id from tpcds_10000_text.customer; Query: explain select c_customer_id from tpcds_10000_text.customer +------------------------------------------------------------+ | Explain String | +------------------------------------------------------------+ | Max Per-Host Resource Reservation: Memory=8.00MB Threads=3 | | Per-Host Resource Estimates: Memory=48MB | | | | PLAN-ROOT SINK | | | | | 01:EXCHANGE [UNPARTITIONED] | | | | | 00:SCAN HDFS [tpcds_10000_text.customer] | | HDFS partitions=1/1 files=1 size=12.50MB | | row-size=-1B cardinality=100.00K | +------------------------------------------------------------+ {noformat} Here's a repro in my dev env: {noformat} [localhost:21000] tpcds> create external table repro_cust(c_customer_sk INT, c_customer_id VARCHAR(16)) row format delimited fields terminated by '|' WITH SERDEPROPERTIES ('field.delim'='|', 'serialization.format'='|') STORED AS TEXTFILE LOCATION 'hdfs://localhost:20500/test-warehouse/tpcds.customer'; Query: create external table repro_cust(c_customer_sk INT, c_customer_id VARCHAR(16)) row format delimited fields terminated by '|' WITH SERDEPROPERTIES ('field.delim'='|', 'serialization.format'='|') STORED AS TEXTFILE LOCATION 'hdfs://localhost:20500/test-warehouse/tpcds.customer' +-------------------------+ | summary | +-------------------------+ | Table has been created. | +-------------------------+ Fetched 1 row(s) in 0.39s [localhost:21000] tpcds> compute stats repro_cust; Query: compute stats repro_cust +-----------------------------------------+ | summary | +-----------------------------------------+ | Updated 1 partition(s) and 2 column(s). | +-----------------------------------------+ Fetched 1 row(s) in 3.62s [localhost:21000] tpcds> explain select c_customer_id from repro_cust; Query: explain select c_customer_id from repro_cust +------------------------------------------------------------+ | Explain String | +------------------------------------------------------------+ | Max Per-Host Resource Reservation: Memory=8.00MB Threads=3 | | Per-Host Resource Estimates: Memory=48MB | | | | PLAN-ROOT SINK | | | | | 01:EXCHANGE [UNPARTITIONED] | | | | | 00:SCAN HDFS [tpcds.repro_cust] | | HDFS partitions=1/1 files=1 size=12.60MB | | row-size=-1B cardinality=100.00K | +------------------------------------------------------------+ Fetched 10 row(s) in 0.01s {noformat} > IllegalStateException in HashJoinNode because of missing memory estimate > ------------------------------------------------------------------------ > > Key: IMPALA-8849 > URL: https://issues.apache.org/jira/browse/IMPALA-8849 > Project: IMPALA > Issue Type: Bug > Components: Frontend > Affects Versions: Impala 3.3.0 > Reporter: Tim Armstrong > Assignee: Tim Armstrong > Priority: Blocker > Attachments: query4.sql > > > [~grahn] reports the below error when running the attached TPC-DS query on > uncompressed text with stats. > {noformat} > I0808 03:27:16.621085 150090 jni-util.cc:288] > 8945e9f76f33dca0:8bf31c4700000000] java.lang.IllegalStateException: Mem > estimate must be set > at com.google.common.base.Preconditions.checkState(Preconditions.java:149) > at > org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73) > at > org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252) > at > org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238) > at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416) > at > org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162) > at > org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477) > at > org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341) > at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236) > at > org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206) > at > org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154) > I0808 03:27:16.621861 150090 status.cc:124] > 8945e9f76f33dca0:8bf31c4700000000] IllegalStateException: Mem estimate must > be set > @ 0xb9af99 > @ 0x11f785e > @ 0x10ab9a3 > @ 0x10d0f54 > @ 0x10e2a2c > @ 0x1125f8d > @ 0x1458c54 > @ 0x145810c > @ 0xb68769 > @ 0xf91ca0 > @ 0xf8791e > @ 0xf887b1 > @ 0x127cd2f > @ 0x127d8d9 > @ 0x1ac0709 > @ 0x7f11ea4d7dd4 > @ 0x7f11e6f1e02c > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org