[ 
https://issues.apache.org/jira/browse/SPARK-62067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haotian Sun updated SPARK-62067:
--------------------------------
    Description: Python worker timing metrics are floored to whole milliseconds 
per task, so sub-millisecond work reports 0 and sums over many tasks 
undercount. Report the worker's timestamps and durations in nanoseconds, and 
expose the Python timing SQL metrics as nanosecond timing metrics whose keys 
and names include the unit.  (was: Python worker timing metrics are floored to 
whole milliseconds per task, so sub-millisecond work reports 0 and sums over 
many tasks undercount. Report the worker's timestamps and durations in 
microseconds, and expose the Python timing SQL metrics as microsecond sums 
whose keys and names include the unit.)

> Report Python worker timing metrics in nanoseconds
> --------------------------------------------------
>
>                 Key: SPARK-62067
>                 URL: https://issues.apache.org/jira/browse/SPARK-62067
>             Project: Spark
>          Issue Type: Sub-task
>          Components: PySpark
>    Affects Versions: 5.0.0
>            Reporter: Haotian Sun
>            Priority: Major
>              Labels: pull-request-available
>
> Python worker timing metrics are floored to whole milliseconds per task, so 
> sub-millisecond work reports 0 and sums over many tasks undercount. Report 
> the worker's timestamps and durations in nanoseconds, and expose the Python 
> timing SQL metrics as nanosecond timing metrics whose keys and names include 
> the unit.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to