zero323 commented on a change in pull request #30346:
URL: https://github.com/apache/spark/pull/30346#discussion_r521986217
##########
File path: python/pyspark/streaming/kinesis.py
##########
@@ -43,38 +43,59 @@ def createStream(ssc, kinesisAppName, streamName,
endpointUrl, regionName,
Create an input stream that pulls messages from a Kinesis stream. This
uses the
Kinesis Client Library (KCL) to pull messages from Kinesis.
- .. note:: The given AWS credentials will get saved in DStream
checkpoints if checkpointing
- is enabled. Make sure that your checkpoint directory is secure.
+ Parameters
+ ----------
+ ssc : :class:`StreamingContext`
+ StreamingContext object
+ kinesisAppName : str
+ Kinesis application name used by the Kinesis Client Library (KCL)
to
+ update DynamoDB
+ streamName : str
+ Kinesis stream name
+ endpointUrl : str
+ Url of Kinesis service (e.g.,
https://kinesis.us-east-1.amazonaws.com)
+ regionName : str
+ Name of region used by the Kinesis Client Library (KCL) to update
+ DynamoDB (lease coordination and checkpointing) and CloudWatch
(metrics)
+ initialPositionInStream : int
+ In the absence of Kinesis checkpoint info, this is the
+ worker's initial starting position in the stream. The
+ values are either the beginning of the stream per Kinesis'
+ limit of 24 hours (InitialPositionInStream.TRIM_HORIZON) or
+ the tip of the stream (InitialPositionInStream.LATEST).
+ checkpointInterval : int
+ Checkpoint interval for Kinesis checkpointing. See the Kinesis
+ Spark Streaming documentation for more details on the different
+ types of checkpoints.
+ storageLevel : :class:`StorageLevel`, optional
Review comment:
As above
:class:`pyspark.StorageLevel`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
