Zouxxyy commented on code in PR #5445:
URL: https://github.com/apache/paimon/pull/5445#discussion_r2140498408
##########
paimon-spark/paimon-spark-ut/pom.xml:
##########
@@ -34,9 +34,23 @@ under the License.
<properties>
<spark.version>${paimon-spark-common.spark.version}</spark.version>
<jackson.version>2.15.2</jackson.version>
+ <janino.test.version>3.1.9</janino.test.version>
</properties>
<dependencies>
+
+ <dependency>
Review Comment:
why we need these two dependencies
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/PaimonScan.scala:
##########
@@ -53,7 +54,11 @@ case class PaimonScan(
table match {
case fileStoreTable: FileStoreTable =>
val bucketSpec = fileStoreTable.bucketSpec()
- if (bucketSpec.getBucketMode != BucketMode.HASH_FIXED) {
+ // todo introduce bucket transform for different bucket function type
+ if (
+ bucketSpec.getBucketMode != BucketMode.HASH_FIXED || coreOptions
+ .bucketFunctionType() != BucketFunctionType.PAIMON
Review Comment:
This condition also needs to be added to the v2 writer enable
##########
paimon-api/src/main/java/org/apache/paimon/CoreOptions.java:
##########
@@ -125,6 +125,18 @@ public class CoreOptions implements Serializable {
+ "if there is no primary
key, the full row will be used.")
.build());
+ @Immutable
+ public static final ConfigOption<BucketFunctionType> BUCKET_FUNCTION_TYPE =
+ key("bucket-function.type")
+ .enumType(BucketFunctionType.class)
+ .defaultValue(BucketFunctionType.PAIMON)
+ .withDescription("The bucket function for paimon bucket");
+
+ /** Paimon bucket function type. */
+ public enum BucketFunctionType {
+ PAIMON,
Review Comment:
This name is a bit strange.
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/catalog/functions/PaimonFunctions.scala:
##########
@@ -60,7 +69,8 @@ object PaimonFunctions {
*
* params arg0: bucket number, arg1...argn bucket keys.
*/
-class BucketFunction extends UnboundFunction {
+class BucketFunction(funcName: String, bucketFunctionType: BucketFunctionType)
Review Comment:
No need function name, it can only be "bucket"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@paimon.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
