1> Are you using YARN integration?
No, I use internal Hawq RM
2> Provide a copy of your log file where the query was running. Sometimes the
you error you describe can be related to RM configuration
This is the last portion of the log. After all dpfdist processes are started
the load fails:
executing on node-03 ./start_gpfdist.sh 9010
/opt/gpadmin/segmentdd/pivotalguru_10
Started gpfdist on port 9010
executing on node -03 ./start_gpfdist.sh 9011
/opt/gpadmin/segmentdd/pivotalguru_11
Started gpfdist on port 9011
executing on node -03 ./start_gpfdist.sh 9012
/opt/gpadmin/segmentdd/pivotalguru_12
Started gpfdist on port 9012
psql -v ON_ERROR_STOP=ON -f
/home/gpadmin/tpc_test/hawq_tpc/TPC-DS-master/04_load/051.insert.call_center.sql
| grep INSERT | awk -F ' ' '{print $3}'
psql:/home/gpadmin/tpc_test/hawq_tpc/TPC-DS-master/04_load/051.insert.call_center.sql:1:
ERROR: Could not allocate enough memory! bucket number of result hash table
and external table should match each other (cdbdatalocality.c:4245)
3> Provide a snipped of the DDL used to create the table. You don’t have to
provide every column name if the table has a lot of columns.
psql -a -P pager=off -v ON_ERROR_STOP=ON -f
/home/gpadmin/tpc_test/hawq_tpc/TPC-DS-master/03_ddl/001.tpcds.call_center.sql
-v SMALL_STORAGE="appendonly=true, orientation=parquet, pagesize=1048576,
rowgroupsize=16777216" -v MEDIUM_STORAGE="appendonly=true, orientation=parquet,
compresstype=snappy, pagesize=1048576, rowgroupsize=16777216" -v
LARGE_STORAGE="appendonly=true, orientation=parquet, compresstype=snappy,
pagesize=1048576, rowgroupsize=16777216" -v DISTRIBUTED_BY="DISTRIBUTED BY
(cc_call_center_sk)"
set search_path=tpcds,public;
SET
\timing
Timing is on.
CREATE TABLE tpcds.call_center (
cc_call_center_sk integer,
cc_call_center_id character varying(16),
cc_rec_start_date date,
cc_rec_end_date date,
cc_closed_date_sk integer,
cc_open_date_sk integer,
cc_name character varying(50),
cc_class character varying(50),
cc_employees integer,
cc_sq_ft integer,
cc_hours character varying(20),
cc_manager character varying(40),
cc_mkt_id integer,
cc_mkt_class character varying(50),
cc_mkt_desc character varying(100),
cc_market_manager character varying(40),
cc_division text,
cc_division_name character varying(50),
cc_company text,
cc_company_name character varying(50),
cc_street_number character varying(10),
cc_street_name character varying(60),
cc_street_type character varying(15),
cc_suite_number character varying(10),
cc_city character varying(60),
cc_county character varying(30),
cc_state text,
cc_zip character varying(10),
cc_country character varying(20),
cc_gmt_offset numeric(5,2),
cc_tax_percentage numeric(5,2)
)
WITH (:SMALL_STORAGE)
:DISTRIBUTED_BY;
CREATE TABLE
Time: 9.717 ms
External table is created like so:
psql -a -P pager=off -v ON_ERROR_STOP=ON -f <a long list of dat files>
set search_path=tpcds,public;
SET
\timing
Timing is on.
CREATE EXTERNAL TABLE ext_tpcds.call_center (like tpcds.call_center)
LOCATION (:LOCATION)
FORMAT 'TEXT' (DELIMITER '|' NULL AS '' ESCAPE AS E'\\');
CREATE EXTERNAL TABLE
Time: 2.164 ms
4> What parameters did you set at the server level if different from default?
Particularly pertaining to resource management. If using YARN, how are your
resource queues setup?
I’ve only set 2 parameters, when I was troubleshooting this issue, however this
issue was presented even on default HAWQ parameters. I initially reported it
here: https://github.com/pivotalguru/TPC-DS/issues/7
hawq_rm_stmt_vseg_memory=128mb
default_hash_table_bucket_number=12 # default 6
hawq_rm_nvseg_perquery_perseg_limit=12 # default 6
5> Can you provide at least an explain plan if not an explain analyze output.
Neither explains give the execution plan:
gpadmin=# explain INSERT INTO tpcds.call_center SELECT * FROM
ext_tpcds.call_center;
ERROR: Could not allocate enough memory! bucket number of result hash table
and external table should match each other (cdbdatalocality.c:4245)
From: Jim Campbell [mailto:[email protected]]
Sent: Friday, January 13, 2017 1:09 PM
To: [email protected]; Dmitry Buzolin
Subject: Re: Could not allocate enough memory
WARNING - External email; exercise caution
There are several things that could cause this error. Can you answer the
following questions:
1> Are you using YARN integration?
2> Provide a copy of your log file where the query was running. Sometimes the
you error you describe can be related to RM configuration
3> Provide a snipped of the DDL used to create the table. You don’t have to
provide every column name if the table has a lot of columns.
4> What parameters did you set at the server level if different from default?
Particularly pertaining to resource management. If using YARN, how are your
resource queues setup?
5> Can you provide at least an explain plan if not an explain analyze output.
That should help for starts. I have seen a similar error message before.
James Campbell
Data Eningeer
Pivotal Software
P: 571-247-6511
E: [email protected]<mailto:[email protected]>
On January 13, 2017 at 1:00:25 PM, Dmitry Buzolin
([email protected]<mailto:[email protected]>) wrote:
I am trying to run TPC-DS with has distributed tables and run into this issue:
master/04_load/051.insert.call_center.sql:1: ERROR: Could not allocate enough
memory! bucket number of result hash table and external table should match each
other (cdbdatalocality.c:4245)
I have verified my memory and O/S kernel configuration is correct. Is there a
problem between external and hash table definition mismatch in terms on number
of buckets? What else I can check?
Thanks.
________________________________
This message may contain confidential information and is intended for specific
recipients unless explicitly noted otherwise. If you have reason to believe you
are not an intended recipient of this message, please delete it and notify the
sender. This message may not represent the opinion of Intercontinental
Exchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a
contract or guarantee. Unencrypted electronic mail is not secure and the
recipient of this message is expected to provide safeguards from viruses and
pursue alternate means of communication where privacy or a binding message is
desired.
________________________________
This message may contain confidential information and is intended for specific
recipients unless explicitly noted otherwise. If you have reason to believe you
are not an intended recipient of this message, please delete it and notify the
sender. This message may not represent the opinion of Intercontinental
Exchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a
contract or guarantee. Unencrypted electronic mail is not secure and the
recipient of this message is expected to provide safeguards from viruses and
pursue alternate means of communication where privacy or a binding message is
desired.
