nsivabalan commented on issue #4082: URL: https://github.com/apache/hudi/issues/4082#issuecomment-1003784185
@xuranyang : without further info, we can't do much here. Can you please let us know what exactly you are looking for. I am not an expert in structured streaming, but if you are looking to read from one stream and write to diff hudi tables based on some condition, here is one hacky way which I can think of. https://gist.github.com/nsivabalan/f7ee7fa611cfc864db7506c016a73787 This intercepts each mirco batch from source stream and writes to hudi table. You can build upon this to have different switch cases and write to different hudi tables. But this would mean your writes are going through spark datasource write and not as streaming write. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
