stream2000 commented on PR #9118: URL: https://github.com/apache/hudi/pull/9118#issuecomment-1715387731
> can we add list a table of metrics we support here, illustrating the usage of each metrics, and then we are clear whether it deserves to introduce the metrics. @danny0405 Hi Danny, I have listed all the metrics I add and show when to use them. Could you help review this? Thanks~ <ne-clipboard data="%7B%22type%22%3A%22fragment%22%2C%22name%22%3A%22%23fragment%22%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u859c46f9%22%2C%22name%22%3A%22tableHole%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22kjeHw%22%2C%22name%22%3A%22table%22%2C%22attrs%22%3A%7B%22rowCount%22%3A11%2C%22colCount%22%3A3%2C%22colWidths%22%3A%5B193%2C193%2C256%5D%2C%22widthMode%22%3A%22contain%22%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u071acac9%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ub3a52c27%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u2872bd58%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u83d93748%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%22color%22 %3A%22rgb(28%2C%2030%2C%2033)%22%2C%22fontsize%22%3A16%2C%22bold%22%3Atrue%7D%2C%22data%22%3A%22Name%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uf5e7ff28%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u3377d721%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22uc66967ad%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%22color%22%3A%22rgb(28%2C%2030%2C%2033)%22%2C%22fontsize%22%3A16%2C%22bold%22%3Atrue%7D%2C%22data%22%3A%22Description%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ued81e5ad%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u2c81cfd6%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ua86e6485%22%2C%22name%22%3A%22%23text%22%2C%22attrs%2 2%3A%7B%22color%22%3A%22rgb(28%2C%2030%2C%2033)%22%2C%22fontsize%22%3A16%2C%22bold%22%3Atrue%7D%2C%22data%22%3A%22When%20to%20Use%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u0812c141%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u6ef7eecb%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u418c79a8%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u128c63e7%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22recordWrittenPerSecond%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u20a1613c%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22uac4fac7a%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C %7B%22type%22%3A%22element%22%2C%22id%22%3A%22uba62bede%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ufcbe6233%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ue78c350f%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Number%20of%20records%20written%20per%20seconds.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u0c36532e%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u8250f97b%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uf5410d41%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uec6c9a6d%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22childre n%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u58d0eea0%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22To%20see%20TPS%20per%20TM%2Fwrite%20task%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u5b4981f5%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uf695c994%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u0b7be029%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u13514836%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22currentCommitWrittenRecords%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uc9145649%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ua554e9ec%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7 B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uae0d3304%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u511b8423%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u045946ac%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Number%20of%20records%20written%20in%20during%20a%20checkpoint%20window.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ud8afe82f%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u59d65a34%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uc5edccfe%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22% 3A%22u05196c5d%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u378b6830%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22To%20see%20number%20of%20records%20written%20in%20during%20a%20checkpoint%20window.%20If%20this%20number%20various%20a%20lot%20between%20Taskmanagers%2C%20there%20may%20be%20data%20skew.%20%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u43fbc25b%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u9be3d8da%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u70883619%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u5664fcff%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22writerFlushCosts%22%7D%5D%7D%2C%7B%22type%22%3A %22element%22%2C%22id%22%3A%22ue449050f%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ufc5fec5f%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u3fb2ec08%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ub204ddd7%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u78805088%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Flush%20data%20costs%20during%20checkpoint.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u7b931cc8%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22uf319a578%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%22%7D%5D%7D%5D%7D%2C%7B%2 2type%22%3A%22element%22%2C%22id%22%3A%22u0632cd5b%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u9985cfd8%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ud297d9b3%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22To%20see%20if%20%20flink%20checkpoint%20is%20block%20by%20hudi%20data%20flush.%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u21c13f0c%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%22height%22%3A71%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u6c40e637%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ucc6889a1%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u183daf2e%22%2C%22name%22%3A%22%23text%22%2C%22 attrs%22%3A%7B%7D%2C%22data%22%3A%22writeBufferedSize%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ubb1522cc%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ud7cd1c29%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ub5dd1588%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u7582a070%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u4e40b81e%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Current%20write%20buffer%20size%20in%20StreamWriteFunction.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u2c79a790%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u42fa4b0b %22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u53bcc1dc%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uc7e5ef9f%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u9d3a7fd7%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22To%20track%20current%20buffer%20size%20of%20each%20task%20managers.%20Useful%20when%20we%20adjust%20%20%60write.task.max.size%60%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u0028f09e%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%22height%22%3A116%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u767ab0ae%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u80edb183%22 %2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u277c4887%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22handleCloseCosts%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u2eedc497%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u8d077242%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u6801422f%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ub89e7b64%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u0364cbe3%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Cost%20of%20write%20handle%20closing.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A %22u63f56512%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ubee57ca0%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u421a2697%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u88c54352%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ufe427f94%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Close%20a%20handle%20means%20flush%20the%20data%20in%20buffer%20to%20file%20system.%20This%20metrics%20track%20the%20costs%20of%20one%20single%20flush.%20%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u90bb7daa%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%22height%22%3A97%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id% 22%3A%22u0bc3ece7%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uffe3692a%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u173c567c%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22writerHandleCloseTotalCosts%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uf1cb28c0%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ue9da2463%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u1630c4d9%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u6fa40d2a%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u79f3f e6f%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Total%20costs%20for%20closing%20write%20handles.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u951914fd%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u97fc7bf5%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u0db7ab7d%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u9f3faea1%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22uf4f4210e%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Total%20costs%20for%20closing%20write%20handles%2C%20similar%20to%20%22%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u2e12372a%22%2C%22name%22%3A%22code%22%2C%22attrs%22%3A%7B %7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ufbf29b51%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22writerFlushCost%22%7D%5D%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u29b1c744%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ud67c7a95%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ufcf19e7e%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22uc609c1f7%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22numOfFilesWritten%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uf7f73869%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u9be57c25%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22dat a%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ub39babd2%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ub8a67a26%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u3783e489%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Number%20of%20files%20written%20%20during%20a%20checkpoint%20window.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u58415c66%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22uc8ebf192%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u37c525c8%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ue7025494%22 %2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u1899ac84%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Track%20the%20number%20of%20files%20writtern.%20In%20append%20write%20if%20there%20too%20many%20files%2C%20we%20may%20run%20out%20of%20%20memory%20or%20get%20a%20bad%20write%20performance.%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ua562ce76%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u04617a27%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u81e4a705%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u0d04db84%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22numOfOpenHandle%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22 id%22%3A%22u048afe5c%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u0fe3de10%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u53275a73%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u2a95f744%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22udfc1097a%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Number%20of%20handles%20opened%20during%20a%20checkpoint%20window.%20Increased%20with%20partition%20number%2Fbucket%20number%20etc.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uf9fa79b4%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u4fdd5f36%22%2C%22name%22%3A%22%23text%2 2%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u290b7ccb%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ua75fa83b%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22ub4cf2fe2%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Similar%20to%20%22%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u4df00d23%22%2C%22name%22%3A%22code%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u758cadb7%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22numOfFilesWritten%22%7D%5D%7D%2C%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u74da924b%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22.%20Track%20number%20of%20open%20files%20in%20the%20task%20manager%2C%20if%20we%20open%20too% 20many%20file%20hanfles%20we%20may%20run%20out%20of%20memory%20or%20get%20a%20bad%20write%20performance.%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u583f98ba%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%22height%22%3A97%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u14bbc002%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uf73f2775%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u3d8b3961%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22handleSwitchPerSecond%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ua228e3ed%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22uc9aacda1%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22typ e%22%3A%22element%22%2C%22id%22%3A%22u5047e5da%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u17db5ee4%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u0b1955a4%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Number%20of%20write%20handle%20switches%20per%20seconds.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uadfc2221%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u9620e9bb%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uafa16269%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ud9814af5%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children %22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u9196a146%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Track%20the%20number%20of%20write%20handle%20switches%20per%20seconds.%20If%20this%20values%20increases%2C%20we%20may%20get%20a%20worse%20write%20performance.%22%7D%5D%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22uff6eea2b%22%2C%22name%22%3A%22tr%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u033a62de%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A0%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u97298feb%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22uc95fe2f5%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22handleCreationCosts%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ue3b862bb%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22 %3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u1724d462%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u807e3d50%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A1%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u826472d4%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u06b3a769%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Cost%20of%20write%20handle%20creation.%22%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u33596dcb%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u60707f67%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22%5Cu0000%22%7D%5D%7D%5D%7D%2C%7B%22type%22%3A%22element%22%2C%22id%22%3A%22u80aeb0f8%22%2C%22name%22%3A%22td%22%2C%22attrs%22%3A%7B%22col%22%3A2 %7D%2C%22children%22%3A%5B%7B%22type%22%3A%22element%22%2C%22id%22%3A%22ud5b48b36%22%2C%22name%22%3A%22p%22%2C%22attrs%22%3A%7B%7D%2C%22children%22%3A%5B%7B%22type%22%3A%22text%22%2C%22id%22%3A%22u21f5e2af%22%2C%22name%22%3A%22%23text%22%2C%22attrs%22%3A%7B%7D%2C%22data%22%3A%22Track%20cost%20of%20craete%20a%20new%20file%20in%20file%20system.%20Note%20that%20object%20stores%20may%20have%20rate%20limit%20for%20file%20listing%2Fcreating%2Fdeletion.%20So%20if%20this%20value%20is%20too%20large%20we%20may%20try%20to%20reduce%20requests%20sent%20to%20object%20stores.%20%22%7D%5D%7D%5D%7D%5D%7D%5D%7D%5D%7D%5D%2C%22attrs%22%3A%7B%7D%7D" source="https%3A%2F%2Faliyuque.antfin.com%2Fqijun.fqj%2Fldcg25%2Ffvgghany8wazauh1"></ne-clipboard><div class="lake-content" typography="classic"> Name | Description | When to Use -- | -- | -- recordWrittenPerSecond | Number of records written per seconds. | To see TPS per TM/write task currentCommitWrittenRecords | Number of records written in during a checkpoint window. | To see number of records written in during a checkpoint window. If this number various a lot between Taskmanagers, there may be data skew. writerFlushCosts | Flush data costs during checkpoint. | To see if flink checkpoint is block by hudi data flush. writeBufferedSize | Current write buffer size in StreamWriteFunction. | To track current buffer size of each task managers. Useful when we adjust `write.task.max.size` handleCloseCosts | Cost of write handle closing. | Close a handle means flush the data in buffer to file system. This metrics track the costs of one single flush. writerHandleCloseTotalCosts | Total costs for closing write handles. | Total costs for closing write handles, similar to writerFlushCost numOfFilesWritten | Number of files written during a checkpoint window. | Track the number of files writtern. In append write if there too many files, we may run out of memory or get a bad write performance. numOfOpenHandle | Number of handles opened during a checkpoint window. Increased with partition number/bucket number etc. | Similar to numOfFilesWritten. Track number of open files in the task manager, if we open too many file hanfles we may run out of memory or get a bad write performance. handleSwitchPerSecond | Number of write handle switches per seconds. | Track the number of write handle switches per seconds. If this values increases, we may get a worse write performance. handleCreationCosts | Cost of write handle creation. | Track cost of craete a new file in file system. Note that object stores may have rate limit for file listing/creating/deletion. So if this value is too large we may try to reduce requests sent to object stores. </div> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
