[
https://issues.apache.org/jira/browse/SPARK-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davies Liu updated SPARK-9829:
------------------------------
Description:
When run a query with 8G memory, the peakExecutionMemory in WebUI said that
40344371200 (40G).
Alos there are lots of accumulators with the same name, can't know what do they
mean
{code}
Accumulable Value
number of output rows 439614
number of output rows 7711
number of output rows 965
number of rows 7829
number of rows 7711
number of input rows 965
number of rows 52
number of input rows 439614
number of output rows 30
number of input rows 7726
number of rows 277000
peakExecutionMemory 40344371200
number of rows 7829
number of rows 965
number of rows 7726
number of rows 30
number of rows 138000
number of rows 8028
number of rows 439614
number of input rows 30
{code}
How to reproduce:
run TPCDS q19 with scale=5, checkout out the Web UI
was:
When run a query with 8G memory, the peakExecutionMemory in WebUI said that
40344371200 (40G).
Alos there are lots of accumulators with the same name, can't know what do they
mean
{code}
Accumulable Value
number of output rows 439614
number of output rows 7711
number of output rows 965
number of rows 7829
number of rows 7711
number of input rows 965
number of rows 52
number of input rows 439614
number of output rows 30
number of input rows 7726
number of rows 277000
peakExecutionMemory 40344371200
number of rows 7829
number of rows 965
number of rows 7726
number of rows 30
number of rows 138000
number of rows 8028
number of rows 439614
number of input rows 30
{code}
> peakExecutionMemory is not correct
> ----------------------------------
>
> Key: SPARK-9829
> URL: https://issues.apache.org/jira/browse/SPARK-9829
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Davies Liu
> Assignee: Shixiong Zhu
>
> When run a query with 8G memory, the peakExecutionMemory in WebUI said that
> 40344371200 (40G).
> Alos there are lots of accumulators with the same name, can't know what do
> they mean
> {code}
> Accumulable Value
> number of output rows 439614
> number of output rows 7711
> number of output rows 965
> number of rows 7829
> number of rows 7711
> number of input rows 965
> number of rows 52
> number of input rows 439614
> number of output rows 30
> number of input rows 7726
> number of rows 277000
> peakExecutionMemory 40344371200
> number of rows 7829
> number of rows 965
> number of rows 7726
> number of rows 30
> number of rows 138000
> number of rows 8028
> number of rows 439614
> number of input rows 30
> {code}
> How to reproduce:
> run TPCDS q19 with scale=5, checkout out the Web UI
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
