Kavin88 commented on issue #3344: URL: https://github.com/apache/hudi/issues/3344#issuecomment-954433916
> I tried MOR with multi table deltastreamer and it worked(one time sync) for multiple tables. May be we can start with MOR table type and you can schedule multi-table deltastreamer once every 5 or 10 mins. Should achieve what you are looking for. In the mean time, we can try to see how to add continuous mode support in multi-table deltastreamer. Eventhough MOR is selected in MultitablesDeltastreamer , output written is COW type. We have the similar use case ingesting 1000 tables with MOR .Submitting 1000 spark jobs in continuous run mode through Deltastreamer should not be a ideal solution . Capacity of cluster is the question here . Any best approaches ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
