[
https://issues.apache.org/jira/browse/SPARK-44591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Francois Desjeans Gauthier updated SPARK-44591:
----------------------------------------------------
Description:
As part SPARK-43952, users can define job tags via SparkContext.addJobTag.
These tags are then used to trigger cancelation via
SparkContext.cancelJobsByTag. Furthermore, these tags can be used to logically
group multiple jobs together.
Listener of job events can retrieve job tags via
SparkListenerJobStart.props.getProperty(SparkContext.SPARK_JOB_TAGS)
Listener of SQL events can link SparkListenerJobStart &
SparkListenerSQLExecutionStart via
SparkListenerJobStart.props.getProperty(SQLExecution.EXECUTION_ID_KEY).
However, some SQL executions do not trigger jobs (i.e. commands). As such
listeners of SQL executions cannot resolve job tags of all executions.
was:
As part SPARK-43952, users can define job tags via SparkContext.addJobTag.
These tags are then used to trigger cancelation via
SparkContext.cancelJobsByTag.
Listener of job events can retrieve the jobTag via
SparkListenerJobStart.props.getProperty(SparkContext.SPARK_JOB_TAGS)
Listener of SQL events can link SparkListenerJobStart &
SparkListenerSQLExecutionStart via
SparkListenerJobStart.props.getProperty(SQLExecution.EXECUTION_ID_KEY).
However, some SQL executions do not trigger jobs (i.e. commands). As such
listeners of SQL executions cannot resolve jobTags of all executions.
To solve this, add jobTags to SparkListenerSQLExecutionStart
> Add jobTags to SparkListenerSQLExecutionStart
> ---------------------------------------------
>
> Key: SPARK-44591
> URL: https://issues.apache.org/jira/browse/SPARK-44591
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: 3.5.0, 4.0.0
> Reporter: Jean-Francois Desjeans Gauthier
> Priority: Major
>
> As part SPARK-43952, users can define job tags via SparkContext.addJobTag.
> These tags are then used to trigger cancelation via
> SparkContext.cancelJobsByTag. Furthermore, these tags can be used to
> logically group multiple jobs together.
> Listener of job events can retrieve job tags via
> SparkListenerJobStart.props.getProperty(SparkContext.SPARK_JOB_TAGS)
> Listener of SQL events can link SparkListenerJobStart &
> SparkListenerSQLExecutionStart via
> SparkListenerJobStart.props.getProperty(SQLExecution.EXECUTION_ID_KEY).
> However, some SQL executions do not trigger jobs (i.e. commands). As such
> listeners of SQL executions cannot resolve job tags of all executions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
