Kavin88 commented on issue #3831: URL: https://github.com/apache/hudi/issues/3831#issuecomment-964268810
i need to ingest 100 tables through deltastreamer and spark job would be running in continuous mode. My thought is to implement this through python code with sparksession and do the livy submit calling the pyfile. Could find anywhere to integrate the --props and --source-class parameter directly in python code rather than passing it through spark-submit. Planning to achieve this in generic code and change param values alone for each table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
