Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
Yohahaha commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2986757019 > , thank you binwei! I have realized the whole context from your message in wecaht group. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
jinchengchenghh merged PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
FelixYBW commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2986609893 @Yohahaha The enhanced repo is going to solve the issue that some features can't be merged into upstream velox for long time. So we will pick the PR into ibm/velox:main branch, pass the Velox and Gluten CI, rebase the PR for every upstream PR merged. So the Gluten customer can use the feature if they want. It's just for the big features, not every PRs. If one want their PR to be picked as well, they need to take the responsibility to fix the PR conflict every day. @zhli1142015 in case you didn't noted it. We are still trying to find some solution to the Velox governor issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
jinchengchenghh commented on code in PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#discussion_r2153907512 ## ep/build-velox/src/get_velox.sh: ## @@ -20,17 +20,22 @@ VELOX_REPO=https://github.com/oap-project/velox.git VELOX_BRANCH=2025_06_11 VELOX_HOME="" RUN_SETUP_SCRIPT=ON +VELOX_ENHANCED_REPO=https://github.com/oap-project/velox.git +VELOX_ENHANCED_BRANCH=2025_06_10 Review Comment: The ibm/velox repo exists, we prepare the ci now, it will be ready soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
PHILO-HE commented on code in PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#discussion_r2153886164 ## ep/build-velox/src/get_velox.sh: ## @@ -20,17 +20,22 @@ VELOX_REPO=https://github.com/oap-project/velox.git VELOX_BRANCH=2025_06_11 VELOX_HOME="" RUN_SETUP_SCRIPT=ON +VELOX_ENHANCED_REPO=https://github.com/oap-project/velox.git +VELOX_ENHANCED_BRANCH=2025_06_10 Review Comment: @jinchengchenghh, will the current setting for VELOX_ENHANCED_REPO & VELOX_ENHANCED_BRANCH be merged to the code base? Will they soon be substituted with genuine enhanced repo/branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
jinchengchenghh commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2982788912 Could you help review agin? All the comments resolved. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
github-actions[bot] commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2982754965 Run Gluten Clickhouse CI on x86 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
github-actions[bot] commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2982740251 Run Gluten Clickhouse CI on x86 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
github-actions[bot] commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2978731722 Run Gluten Clickhouse CI on x86 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
jinchengchenghh commented on code in PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#discussion_r2151157248 ## ep/build-velox/src/get_velox.sh: ## @@ -20,17 +20,22 @@ VELOX_REPO=https://github.com/oap-project/velox.git VELOX_BRANCH=2025_06_11 VELOX_HOME="" RUN_SETUP_SCRIPT=ON +VELOX_ENHANCED_REPO=https://github.com/oap-project/velox.git +VELOX_ENHANCED_BRANCH=2025_06_10 Review Comment: Yes, I place it here because it is easy to update daily and understand. If we move it to the logic, it might be hard for user to find the repo and branch. Now it is configurable, user can set --velox_repo and --velox_branch to specify the enhanced feature branch. Do I need to update? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
FelixYBW commented on code in PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#discussion_r2150766709 ## ep/build-velox/src/get_velox.sh: ## @@ -20,17 +20,22 @@ VELOX_REPO=https://github.com/oap-project/velox.git VELOX_BRANCH=2025_06_11 VELOX_HOME="" RUN_SETUP_SCRIPT=ON +VELOX_ENHANCED_REPO=https://github.com/oap-project/velox.git +VELOX_ENHANCED_BRANCH=2025_06_10 Review Comment: Can we reuse the VELOX_REPO and make it configurable, instead of adding the new VELOX_ENHANCED_REPO? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
github-actions[bot] commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2977296116 Run Gluten Clickhouse CI on x86 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
github-actions[bot] commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2977281874 Run Gluten Clickhouse CI on x86 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
jinchengchenghh commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2975382772 We will support to write iceberg partition table with statistics, read equality delete, other features not only iceberg or not essential fix may also be merged to this enhanced-features branch, (essential fix will be merged to daily updated oap-velox branch), once Meta mainline velox merge the PR, we will remove it from this branch. @Yohahaha -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
Yohahaha commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2969316893 oh I see the `enhanced` means `Some Iceberg features`, why just clarify the compile options with iceberg? and I wonder the details of `Some Iceberg features`, could you explain more about that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
Yohahaha commented on PR #9957: URL: https://github.com/apache/incubator-gluten/pull/9957#issuecomment-2969308935 what is `enhanced features`? could you add a link or enrich the PR description? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [GLUTEN-9949][VL] Enable enhanced features compile [incubator-gluten]
zhztheplayer commented on code in PR #9957:
URL: https://github.com/apache/incubator-gluten/pull/9957#discussion_r2142305637
##
gluten-core/src/main/scala/org/apache/gluten/config/GlutenConfig.scala:
##
@@ -117,6 +117,9 @@ class GlutenConfig(conf: SQLConf) extends Logging {
def enableExtendedColumnPruning: Boolean =
getConf(ENABLE_EXTENDED_COLUMN_PRUNING)
+ def enableEnhancedFeature(): Boolean =
+System.getenv().getOrDefault("GLUTEN_ENABLE_ENHANCED_FEATURE",
"false").toBoolean
+
Review Comment:
@jinchengchenghh Got it. I think we need a backend API for the feature then.
E.g., `BackendSettingsApi#isEnhancedFeatureEnabled()`
For CH backend, we can make it false by default.
Then we can move `ConfigJniWrapper.java` which I see is newly added to
`gluten-core` to `backends-velox`. This is to follow the rule that we don't put
any JNI code to `gluten-core` so it's more flexible for backends.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
