Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-22 Thread via GitHub
nsivabalan merged PR #10135: URL: https://github.com/apache/hudi/pull/10135 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-22 Thread via GitHub
lokeshj1703 commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1822572274 https://github.com/apache/hudi/assets/9255455/8afcdba5-645e-42cc-bfd5-57896bb8f17f;> -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1822079113 ## CI report: * 34ffc8261d951bde500df7688800b2ed6afb4fa6 Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1821703416 ## CI report: * 3d48bfc5c41a59a1114eb73a5ef9a7b7fda5eccf Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1821544132 ## CI report: * 4913158456e1dfaa1366ba7bd5029578f3bf4cef Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1821528006 ## CI report: * 4913158456e1dfaa1366ba7bd5029578f3bf4cef Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
lokeshj1703 commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1821449929 Added a fix for the test failure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
codope commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1821037286 @lokeshj1703 There is a test failure, can you take a look - https://github.com/apache/hudi/actions/runs/6940682251/job/18889779707?pr=10135#step:6:20607 ``` Test alter column

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1820504957 ## CI report: * 4913158456e1dfaa1366ba7bd5029578f3bf4cef Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1820443377 ## CI report: * 2e26a7d1b87f4ca4e1f818612decfe0eb130a5fb Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-21 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1820431435 ## CI report: * 2e26a7d1b87f4ca4e1f818612decfe0eb130a5fb Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-20 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1820369436 ## CI report: * 2e26a7d1b87f4ca4e1f818612decfe0eb130a5fb Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-20 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1820360875 ## CI report: * 2e26a7d1b87f4ca4e1f818612decfe0eb130a5fb Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-20 Thread via GitHub
lokeshj1703 commented on code in PR #10135: URL: https://github.com/apache/hudi/pull/10135#discussion_r1400093042 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -474,8 +474,11 @@ public static boolean isLogFile(Path logPath) { } public static

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-20 Thread via GitHub
lokeshj1703 commented on code in PR #10135: URL: https://github.com/apache/hudi/pull/10135#discussion_r1400093409 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala: ## @@ -242,4 +243,57 @@ object AvroConversionUtils { val nameParts =

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-20 Thread via GitHub
codope commented on code in PR #10135: URL: https://github.com/apache/hudi/pull/10135#discussion_r1400018858 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala: ## @@ -242,4 +243,57 @@ object AvroConversionUtils { val nameParts =

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-20 Thread via GitHub
nsivabalan commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1819567942 hey @lokeshj1703 : can you follow up on CI failures. just 1 minor comment. once addressed and once CI passes, we are good to go -- This is an automated message from the Apache Git

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-20 Thread via GitHub
nsivabalan commented on code in PR #10135: URL: https://github.com/apache/hudi/pull/10135#discussion_r1399569286 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -474,8 +474,11 @@ public static boolean isLogFile(Path logPath) { } public static

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-17 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1816931057 ## CI report: * 2e26a7d1b87f4ca4e1f818612decfe0eb130a5fb Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-17 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1816857355 ## CI report: * 2e26a7d1b87f4ca4e1f818612decfe0eb130a5fb Azure:

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-17 Thread via GitHub
hudi-bot commented on PR #10135: URL: https://github.com/apache/hudi/pull/10135#issuecomment-1816802107 ## CI report: * 2e26a7d1b87f4ca4e1f818612decfe0eb130a5fb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

[PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-17 Thread via GitHub
lokeshj1703 opened a new pull request, #10135: URL: https://github.com/apache/hudi/pull/10135 ### Change Logs Makes improvements based on findings from CPU profiling for the executor code path. 1. Fixes repetitive execution of string split operation 2. reduces number of