[
https://issues.apache.org/jira/browse/SPARK-43036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved SPARK-43036.
----------------------------------
Resolution: Invalid
Please leverage the user mailing list for questions.
> Spark structuring streaming with kinesis and rocksdb integration
> ----------------------------------------------------------------
>
> Key: SPARK-43036
> URL: https://issues.apache.org/jira/browse/SPARK-43036
> Project: Spark
> Issue Type: Question
> Components: Structured Streaming
> Affects Versions: 3.3.1
> Reporter: Rajesh
> Priority: Blocker
>
> Hi Spark Team,
> We need to read/write the kinesis streams using spark streaming.
> We checked the official documentation -
> [https://spark.apache.org/docs/latest/streaming-kinesis-integration.html]
> It does not mention about kinesis connector. Alternative is -
> [https://github.com/qubole/kinesis-sql] which is not active now. This is now
> handed over here - [https://github.com/roncemer/spark-sql-kinesis]
> Also according to SPARK-18165 , Spark officially do not have any kinesis
> connector
> We have few below questions , It would be great if you can answer
> # Does Spark provides officially any kinesis connector which have
> readstream/writestream and endorse any connector for production use cases ?
> # [https://spark.apache.org/docs/latest/streaming-kinesis-integration.html]
> This documentation does not mention about how to write to kinesis. This
> method have default dynamodb as checkpoint, can we override it ?
> # We have rocksdb as state store but when we ran application using official
> [https://spark.apache.org/docs/latest/streaming-kinesis-integration.html]
> rocksdb configurations were not effective. Can you please confirm if rocksdb
> is not applicable in these cases?
> # rocksdb however works with qubole connector , do you have any plan to
> release kinesis connector?
> # Please help/recommend us for any good stable kinesis connector or some
> pointers around it
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
