Wei Liu created SPARK-48890:
-------------------------------
Summary: Add Streaming related fields to log4j ThreadContext
Key: SPARK-48890
URL: https://issues.apache.org/jira/browse/SPARK-48890
Project: Spark
Issue Type: Sub-task
Components: Spark Core, SS
Affects Versions: 4.0.0
Reporter: Wei Liu
There are some special informations needed for structured streaming queries.
Specifically, each query has a query_id and run_id. Also if using
MicrobatchExecution (default), there is a batch_id.
A (query_id, run_id, batch_id) identifies the microbatch the streaming query
runs. Adding these field to a threadContext would help especially when there
are multiple queries running.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
