yucai created SPARK-24925:
-----------------------------
Summary: input bytesRead metrics fluctuate from time to time
Key: SPARK-24925
URL: https://issues.apache.org/jira/browse/SPARK-24925
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.3.1
Reporter: yucai
input bytesRead metrics fluctuate from time to time, it is worse when pushdown
enabled.
Query
{code:java}
CREATE TABLE dev AS
SELECT
...
FROM lstg_item cold, lstg_item_vrtn v
WHERE cold.auct_end_dt = CAST(SUBSTR('2018-03-18 00:00:00',1,10) AS DATE)
AND v.auct_end_dt = CAST(SUBSTR('2018-03-18 00:00:00',1,10) AS DATE)
...
{code}
Issue
See attached gif, input bytesRead shows 48GB, 52GB, 51GB, 50GB, 54GB, 53GB ...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
