parisni commented on issue #8925: URL: https://github.com/apache/hudi/issues/8925#issuecomment-1585787382
Sum of time from stage 0 to 57 takes 40 min. So 2 hours is spend after stage 57 has finished, can you confirm ? I' am not familiar with MOR table, so I'm not sure what's going on after committing. Likely not cleaning or compaction since it would show up a dedicated stage. Have you looked at the executor logs to see if something happens there ? On June 10, 2023 4:42:03 PM UTC, Samarth Raval ***@***.***> wrote: >I have disabled the metadata [I was suspecting it is making my emr job super slow, but I was wrong, as its still take same amount of time] > >This is first screen shot; >1st page: ><img width="1690" alt="image" src="https://github.com/apache/hudi/assets/8738019/ac8e8608-d8a3-4254-920e-12f23d0edc0e";> > >2nd page screen shot: >https://github.com/apache/hudi/issues/8925#issuecomment-1585103310 > >3rd page screen shot: >https://github.com/apache/hudi/issues/8925#issuecomment-1585102673 > > >Entire execution took around 2.5 hours: ><img width="1552" alt="image" src="https://github.com/apache/hudi/assets/8738019/bfe112fb-0c02-4580-a86c-d5793550cd02";> > > > > >-- >Reply to this email directly or view it on GitHub: >https://github.com/apache/hudi/issues/8925#issuecomment-1585731130 >You are receiving this because you were mentioned. > >Message ID: ***@***.***> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
