[
https://issues.apache.org/jira/browse/SPARK-36520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115681#comment-18115681
]
Yuvraj Soni edited comment on SPARK-36520 at 9/15/26 4:53 PM:
--------------------------------------------------------------
I've opened a pull request to fix this issue:
https://github.com/apache/spark/pull/58829
Changes:
- Extracted `KinesisInitialPosition` into its own public file
`KinesisInitialPosition.java`
- Removed the interface from `KinesisInitialPositions.java`, since Java doesn't
allow two public top-level types in the same file
Tested locally with the existing KinesisInputDStreamBuilderSuite,
JavaKinesisInputDStreamBuilderSuite, and KinesisReceiverSuite (23 tests, all
passing), plus a clean compile and checkstyle run.
All CI checks on the PR passed as well.
was (Author: JIRAUSER313704):
PR up: https://github.com/apache/spark/pull/58829
> KinesisInitialPosition interface should be public
> -------------------------------------------------
>
> Key: SPARK-36520
> URL: https://issues.apache.org/jira/browse/SPARK-36520
> Project: Spark
> Issue Type: Improvement
> Components: DStreams
> Affects Versions: 3.1.2
> Reporter: Noam Tamim
> Priority: Minor
> Labels: kinesis, pull-request-available
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In `org.apache.spark.streaming.kinesis` there's a Java interface
> `KinesisInitialPosition` - for some reason it's not public, so can't be used.
> The reason it needs to be used is simple: it is the parent of
> `KinesisInitialPositions.Latest` and `KinesisInitialPositions.TrimHorizon`.
> Without using it, it's impossible to pass instances of `Latest` and
> `TrimHorizon` around without calling them by name.
> The workaround is to use the enum `InitialPositionInStream` instead, and a
> factory method `fromKinesisInitialPosition(..)` - but it doesn't seem like
> the correct pattern, because it belongs to Amazon Kinesis Client library and
> not Spark (it's currently the only import in my class which is outside of
> `org.apache.spark`).
> The change is trivial, so it's mainly a matter of why *not* to do it.
> KinesisInitialPositions:
> https://github.com/apache/spark/blob/master/external/kinesis-asl/src/main/java/org/apache/spark/streaming/kinesis/KinesisInitialPositions.java
> KinesisInitialPosition:
> https://github.com/awslabs/amazon-kinesis-client/blob/master/amazon-kinesis-client/src/main/java/software/amazon/kinesis/common/InitialPositionInStream.java
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
