Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/9421#discussion_r44077155
--- Diff:
extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala
---
@@ -27,26 +27,23 @@ import
com.amazonaws.services.kinesis.clientlibrary.types.ShutdownReason
import com.amazonaws.services.kinesis.model.Record
import org.apache.spark.Logging
+import org.apache.spark.streaming.Duration
/**
* Kinesis-specific implementation of the Kinesis Client Library (KCL)
IRecordProcessor.
* This implementation operates on the Array[Byte] from the
KinesisReceiver.
* The Kinesis Worker creates an instance of this KinesisRecordProcessor
for each
- * shard in the Kinesis stream upon startup. This is normally done in
separate threads,
- * but the KCLs within the KinesisReceivers will balance themselves out
if you create
- * multiple Receivers.
+ * shard in the Kinesis stream upon startup. This is normally done in
separate threads,
+ * but the KCLs within the KinesisReceivers will balance themselves out if
you create
+ * multiple Receivers.
*
* @param receiver Kinesis receiver
* @param workerId for logging purposes
- * @param checkpointState represents the checkpoint state including the
next checkpoint time.
- * It's injected here for mocking purposes.
*/
-private[kinesis] class KinesisRecordProcessor[T](
- receiver: KinesisReceiver[T],
- workerId: String,
- checkpointState: KinesisCheckpointState) extends IRecordProcessor with
Logging {
+private[kinesis] class KinesisRecordProcessor[T](receiver:
KinesisReceiver[T], workerId: String)
+ extends IRecordProcessor with Logging {
- // shardId to be populated during initialize()
+ // shardId and checkpointState to be populated during initialize()
--- End diff --
this is irrelevant now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
