[
https://issues.apache.org/jira/browse/IMPALA-5790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-5790.
-----------------------------------
Resolution: Duplicate
> Failure to get reservation should print buffer pool limit
> ---------------------------------------------------------
>
> Key: IMPALA-5790
> URL: https://issues.apache.org/jira/browse/IMPALA-5790
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.10.0
> Reporter: Matthew Jacobs
> Priority: Major
> Labels: supportability, usability
>
> When playing around with a custom buffer pool limit, i.e.
> {{--buffer_pool_limit}}, I found that failure to reserve enough memory error
> message prints the mem tracker hierarchy but does not include the
> buffer_pool_limit.
> E.g.
> {code}
> start-impala-cluster.py --impalad_args="--buffer_pool_limit=2300m "
> {code}
> {code}
> [localhost:21001] > set DEFAULT_SPILLABLE_BUFFER_SIZE=33554432;
> DEFAULT_SPILLABLE_BUFFER_SIZE set to 33554432
> [localhost:21001] > set MIN_SPILLABLE_BUFFER_SIZE=33554432;
> MIN_SPILLABLE_BUFFER_SIZE set to 33554432
> [localhost:21001] > select straight_join * from tpch_parquet.lineitem join
> (select n_regionkey, count(distinct n_nationkey) from tpch_parquet.nation
> group by 1) v on n_regionkey = l_orderkey;
> Query: select straight_join * from tpch_parquet.lineitem join (select
> n_regionkey, count(distinct n_nationkey) from tpch_parquet.nation group by 1)
> v on n_regionkey = l_orderkey
> Query submitted at: 2017-08-10 18:54:14 (Coordinator:
> http://mj-desktop.ca.cloudera.com:25001)
> ERROR: ExecQueryFInstances rpc query_id=b94a87085426982d:641c5a9600000000
> failed: Failed to get minimum memory reservation of 1.59 GB on daemon
> mj-desktop.ca.cloudera.com:22000 for query b94a87085426982d:641c5a9600000000
> because it would exceed an applicable query, request pool or process memory
> limit. Memory usage:
> Process: Limit=8.35 GB Total=1.11 GB Peak=1.94 GB
> Buffer Pool: Free Buffers: Total=768.00 MB
> Buffer Pool: Clean Pages: Total=0
> Free Disk IO Buffers: Total=256.00 KB Peak=68.00 MB
> RequestPool=default-pool: Total=621.45 MB Peak=1.75 GB
> Query(2540ad2fb916785d:17301f8a00000000): Reservation=544.00 MB
> ReservationLimit=6.68 GB OtherMemory=77.45 MB Total=621.45 MB Peak=1.75 GB
> Fragment 2540ad2fb916785d:17301f8a00000000: Reservation=0
> OtherMemory=16.00 KB Total=16.00 KB Peak=16.00 KB
> EXCHANGE_NODE (id=8): Total=0 Peak=0
> DataStreamRecvr: Total=0 Peak=0
> PLAN_ROOT_SINK: Total=0 Peak=0
> CodeGen: Total=0 Peak=0
> Fragment 2540ad2fb916785d:17301f8a00000005: Reservation=544.00 MB
> OtherMemory=77.43 MB Total=621.43 MB Peak=621.43 MB
> Runtime Filter Bank: Total=1.00 MB Peak=1.00 MB
> HASH_JOIN_NODE (id=4): Reservation=544.00 MB OtherMemory=42.25 KB
> Total=544.04 MB Peak=544.06 MB
> Exprs: Total=13.12 KB Peak=13.12 KB
> Hash Join Builder (join_node_id=4): Total=13.12 KB Peak=21.12 KB
> Hash Join Builder (join_node_id=4) Exprs: Total=13.12 KB
> Peak=13.12 KB
> HDFS_SCAN_NODE (id=0): Total=76.35 MB Peak=76.35 MB
> EXCHANGE_NODE (id=7): Total=0 Peak=0
> DataStreamRecvr: Total=0 Peak=20.04 KB
> DataStreamSender (dst_id=8): Total=1.27 KB Peak=1.27 KB
> CodeGen: Total=21.40 KB Peak=1.41 MB
> Fragment 2540ad2fb916785d:17301f8a00000002: Reservation=0 OtherMemory=0
> Total=0 Peak=1.06 GB
> AGGREGATION_NODE (id=3): Reservation=0 OtherMemory=0 Total=0
> Peak=544.02 MB
> AGGREGATION_NODE (id=6): Reservation=0 OtherMemory=0 Total=0
> Peak=544.04 MB
> EXCHANGE_NODE (id=5): Total=0 Peak=0
> DataStreamRecvr: Total=0 Peak=20.20 KB
> DataStreamSender (dst_id=7): Total=0 Peak=34.90 KB
> CodeGen: Total=0 Peak=529.00 KB
> Fragment 2540ad2fb916785d:17301f8a00000001: Reservation=0 OtherMemory=0
> Total=0 Peak=160.21 MB
> AGGREGATION_NODE (id=2): Reservation=0 OtherMemory=0 Total=0
> Peak=160.02 MB
> HDFS_SCAN_NODE (id=1): Total=0 Peak=157.00 KB
> DataStreamSender (dst_id=5): Total=0 Peak=25.59 KB
> CodeGen: Total=0 Peak=417.50 KB
> Query(b94a87085426982d:641c5a9600000000): Reservation=0
> ReservationLimit=6.68 GB OtherMemory=0 Total=0 Peak=0
> Untracked Memory: Total=-266538560.00 B
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
