srowen commented on a change in pull request #24651:
[SPARK-27420][DSTREAMS][Kinesis] KinesisInputDStream should expose a way to
configure CloudWatch metrics
URL: https://github.com/apache/spark/pull/24651#discussion_r319979785
##########
File path:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
##########
@@ -259,12 +268,45 @@ object KinesisInputDStream {
* endpoint. Will use the same credentials used for AWS Kinesis if no
custom value is set.
*
* @param credentials [[SparkAWSCredentials]] to use for CloudWatch
authentication
+ * @return Reference to this [[KinesisInputDStream.Builder]]
*/
def cloudWatchCredentials(credentials: SparkAWSCredentials): Builder = {
cloudWatchCredsProvider = Option(credentials)
this
}
+ // scalastyle:off line.size.limit
+ /**
+ * Sets the CloudWatch metrics level. Defaults to
+ * [[KinesisClientLibConfiguration.DEFAULT_METRICS_LEVEL]] if no custom
value is specified.
+ *
+ * @param metricsLevel [[MetricsLevel]] to specify the CloudWatch metrics
level
+ * @return Reference to this [[KinesisInputDStream.Builder]]
+ * @see
[[https://docs.aws.amazon.com/streams/latest/dev/monitoring-with-kcl.html#metric-levels]]
+ */
+ // scalastyle:on line.size.limit
+ def metricsLevel(metricsLevel: MetricsLevel): Builder = {
+ this.metricsLevel = Option(metricsLevel)
+ this
+ }
+
+ // scalastyle:off line.size.limit
+ /**
+ * Sets the enabled CloudWatch metrics dimensions. Defaults to
+ * [[KinesisClientLibConfiguration.DEFAULT_METRICS_ENABLED_DIMENSIONS]]
+ * if no custom value is specified.
+ *
+ * @param metricsEnabledDimensions [[Set[String]]] to specify
Review comment:
This is a small thing, but I know we have had problems generating scaladoc
with references to library code. It might be worth building the doc HTML (only)
as in https://github.com/apache/spark/blob/master/docs/README.md to make sure.
You don't really have to link a type like Set.
Also we usually do a continuation indent of two spaces.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
