rawwar commented on issue #43438: URL: https://github.com/apache/airflow/issues/43438#issuecomment-2774401824
I've done some reading and I now find this doable. @kaxil , correct me if I am wrong here. All I need to do is, consider a reasonably large TI table. Also, consider a reasonable-sized DB instance type(probably with 2vcpu and 8 GB RAM)(probably going to run DB on k8s pod and limit the resources). I think time taken should be the only metric I should be keeping in mind. So, I'll start without batching and time it, then with batching, look for other optimizations and try to figure out the best way to reduce the time taken. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
