Github user cfregly commented on a diff in the pull request:
https://github.com/apache/spark/pull/6147#discussion_r30774982
--- Diff:
extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -59,92 +60,103 @@ import
com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker
* (InitialPositionInStream.TRIM_HORIZON)
or
* the tip of the stream
(InitialPositionInStream.LATEST).
* @param storageLevel Storage level to use for storing the received
objects
- *
- * @return ReceiverInputDStream[Array[Byte]]
+ * @param awsCredentialsOption Optional AWS credentials, used when user
directly specifies
+ * the credentials
*/
private[kinesis] class KinesisReceiver(
appName: String,
streamName: String,
endpointUrl: String,
- checkpointInterval: Duration,
+ regionName: String,
initialPositionInStream: InitialPositionInStream,
- storageLevel: StorageLevel)
- extends Receiver[Array[Byte]](storageLevel) with Logging { receiver =>
-
- /*
- * The following vars are built in the onStart() method which executes
in the Spark Worker after
- * this code is serialized and shipped remotely.
- */
-
- /*
- * workerId should be based on the ip address of the actual Spark
Worker where this code runs
- * (not the Driver's ip address.)
- */
- var workerId: String = null
+ checkpointInterval: Duration,
--- End diff --
i think this goes above initialPositionInStream
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
