vanzin commented on a change in pull request #22138: [SPARK-25151][SS] Apply
Apache Commons Pool to KafkaDataConsumer
URL: https://github.com/apache/spark/pull/22138#discussion_r318273681
##########
File path:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/package.scala
##########
@@ -38,4 +38,25 @@ package object kafka010 { // scalastyle:ignore
" (check Structured Streaming Kafka integration guide for further
details).")
.intConf
.createWithDefault(64)
+
+ private[kafka010] val CONSUMER_CACHE_JMX_ENABLED =
+ ConfigBuilder("spark.kafka.consumer.cache.jmx.enable")
+ .doc("Enable or disable JMX for pools created with this configuration
instance.")
+ .booleanConf
+ .createWithDefault(false)
+
+ private[kafka010] val CONSUMER_CACHE_MIN_EVICTABLE_IDLE_TIME_MILLIS =
Review comment:
These configs refer to the "consumer cache" but are also used for the
"fetched data" cache. Should they have a more generic name?
I'm also not sure I like the name, it's worded in the opposite way this kind
of setting is generally used. e.g. calling it an "idle timeout" as used in
other places means it may be evicted after it's been idle for at least that
time.
Config names (the string the user has to set in their configuration) also
should not mention units.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
