codejoyan edited a comment on issue #3581: URL: https://github.com/apache/hudi/issues/3581#issuecomment-911894350
@vinothchandar/@nsivabalan/@umehrot2 As per suggestion, I turned on hoodie.metadata.enable=true and hoodie.metadata.metrics.enable = true during the write. I could see some improvement, but the next step doesn't start immediately. Does it mean that it is still listing the files and not reading from the metadata table? **Approach 3:** <img width="1701" alt="Screenshot 2021-09-02 at 10 37 32 PM" src="https://user-images.githubusercontent.com/48707638/131887598-5e6d36b7-5e5f-400f-a337-cfea79f0aafd.png";> Few other observations: 1. The number of stages for 0 in Approach 2 was 71. In Approach 3 after hoodie.metadata.enable=true it is 34. 2. Also I executed the same query in loop. The first run (Job Id 0,1,2) took more time compared to (Job Id 3,4,5) and (Job Id 6,7,8). Any particular reason behind that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://user-images.githubusercontent.com/48707638/131887598-5e6d36b7-5e5f-400f-a337-cfea79f0aafd.png"](https://user-images.githubusercontent.com/48707638/131887598-5e6d36b7-5e5f-400f-a337-cfea79f0aafd.png"){kind=link}