itholic commented on a change in pull request #30346:
URL: https://github.com/apache/spark/pull/30346#discussion_r523768721
##########
File path: python/pyspark/streaming/kinesis.py
##########
@@ -43,38 +43,59 @@ def createStream(ssc, kinesisAppName, streamName,
endpointUrl, regionName,
Create an input stream that pulls messages from a Kinesis stream. This
uses the
Kinesis Client Library (KCL) to pull messages from Kinesis.
- .. note:: The given AWS credentials will get saved in DStream
checkpoints if checkpointing
- is enabled. Make sure that your checkpoint directory is secure.
+ Parameters
+ ----------
+ ssc : :class:`StreamingContext`
+ StreamingContext object
+ kinesisAppName : str
+ Kinesis application name used by the Kinesis Client Library (KCL)
to
+ update DynamoDB
+ streamName : str
+ Kinesis stream name
+ endpointUrl : str
+ Url of Kinesis service (e.g.,
https://kinesis.us-east-1.amazonaws.com)
+ regionName : str
+ Name of region used by the Kinesis Client Library (KCL) to update
+ DynamoDB (lease coordination and checkpointing) and CloudWatch
(metrics)
+ initialPositionInStream : int
+ In the absence of Kinesis checkpoint info, this is the
+ worker's initial starting position in the stream. The
+ values are either the beginning of the stream per Kinesis'
+ limit of 24 hours (InitialPositionInStream.TRIM_HORIZON) or
+ the tip of the stream (InitialPositionInStream.LATEST).
+ checkpointInterval : int
+ Checkpoint interval for Kinesis checkpointing. See the Kinesis
Review comment:
Add to docs, thanks :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
