xuranyang commented on issue #4082: URL: https://github.com/apache/hudi/issues/4082#issuecomment-1005488249
> @xuranyang : without further info, we can't do much here. Can you please let us know what exactly you are looking for. I am not an expert in structured streaming, but if you are looking to read from one stream and write to diff hudi tables based on some condition, here is one hacky way which I can think of. > > https://gist.github.com/nsivabalan/f7ee7fa611cfc864db7506c016a73787 > > This intercepts each mirco batch from source stream and writes to hudi table. You can build upon this to have different switch cases and write to different hudi tables. But this would mean your writes are going through spark datasource write and not as streaming write. @nsivabalan ,Thanks for your reply. I want read from multiple streams and write to diff hudi tables. Such as writing hudi from multiple Kafka sources. I know it can be done with DeltaStreamer.But I wonder if Spark Streaming can do the same. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
