HeartSaVioR commented on a change in pull request #27146:
[SPARK-21869][SS][DOCS][FOLLOWUP] Document Kafka producer pool configuration
URL: https://github.com/apache/spark/pull/27146#discussion_r365500406
##########
File path: docs/structured-streaming-kafka-integration.md
##########
@@ -802,6 +802,31 @@ df.selectExpr("topic", "CAST(key AS STRING)", "CAST(value
AS STRING)") \
</div>
</div>
+### Producer Caching
+
+Given Kafka producer instance is designed to be thread-safe, Spark initializes
a Kafka producer instance and co-use across tasks for same caching key.
+
+The caching key is built up from the following information:
Review comment:
Hmm... would we like to guide how to tweak it, or would like to just hide
it? The reason I explain this is that end users may encounter an issue on
producer pool and would like to debug a bit, but if it feels us to be too
internal, sounds OK to hide it as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
