HonahX commented on code in PR #945:
URL: https://github.com/apache/polaris/pull/945#discussion_r1943731069
##########
polaris-core/src/main/resources/schemas/policies/system/data-compaction/2025-02-03.json:
##########
@@ -0,0 +1,41 @@
+{
+ "license": "Licensed under the Apache License, Version 2.0
(http://www.apache.org/licenses/LICENSE-2.0)",
+ "$id":
"https://polaris.apache.org/schemas/policies/system/data-compaction/2025-02-03.json";,
+ "title": "Data Compaction Policy",
+ "description": "Inheritable Polaris policy schema for Iceberg table data
compaction.",
+ "type": "object",
+ "properties": {
+ "version": {
+ "type": "string",
+ "const": "2025-02-03",
+ "description": "Schema version."
Review Comment:
Since we use date instead of package version, does this mean the pre-defined
policy type schema (which evolves through community-driven changes) will be
released separately from `polaris-core`, and we'll need to track which Polaris
version supports which schema version? If that's the case, would it make more
sense to move this outside of `polaris-core`?
##########
polaris-core/src/main/resources/schemas/policies/system/data-compaction/2025-02-03.json:
##########
@@ -0,0 +1,41 @@
+{
+ "license": "Licensed under the Apache License, Version 2.0
(http://www.apache.org/licenses/LICENSE-2.0)",
+ "$id":
"https://polaris.apache.org/schemas/policies/system/data-compaction/2025-02-03.json";,
+ "title": "Data Compaction Policy",
+ "description": "Inheritable Polaris policy schema for Iceberg table data
compaction.",
+ "type": "object",
+ "properties": {
+ "version": {
+ "type": "string",
+ "const": "2025-02-03",
+ "description": "Schema version."
+ },
+ "enable": {
+ "type": "boolean",
+ "description": "Enable or disable data compaction."
Review Comment:
Will this be a common field for all policy types? I guess the general
question is whether we want to extend the concept of enabling/disabling
policies to all types or keep it limited to TMS-related policies like
compaction and snapshot expiry.
##########
polaris-core/src/main/resources/schemas/policies/system/data-compaction/2025-02-03.json:
##########
@@ -0,0 +1,41 @@
+{
+ "license": "Licensed under the Apache License, Version 2.0
(http://www.apache.org/licenses/LICENSE-2.0)",
+ "$id":
"https://polaris.apache.org/schemas/policies/system/data-compaction/2025-02-03.json";,
+ "title": "Data Compaction Policy",
+ "description": "Inheritable Polaris policy schema for Iceberg table data
compaction.",
+ "type": "object",
+ "properties": {
+ "version": {
+ "type": "string",
+ "const": "2025-02-03",
+ "description": "Schema version."
+ },
+ "enable": {
+ "type": "boolean",
+ "description": "Enable or disable data compaction."
+ },
+ "target_file_size_bytes": {
+ "type": "number",
+ "description": "Target data file size in bytes."
+ },
+ "config": {
+ "type": "object",
+ "description": "A map containing custom configuration properties. Please
note that interoperability is not guaranteed.",
+ "additionalProperties": {}
+ }
+ },
+ "required": ["enable"],
+ "additionalProperties": false,
+ "examples": [
+ {
+ "version": "2025-02-03",
+ "enable": true,
+ "target_file_size_bytes": 134217728,
+ "config": {
+ "compaction_strategy": "bin-pack",
+ "max-concurrent-file-group-rewrites": 5,
Review Comment:
Could these fields be pre-defined in the config schema? For example
```json
"config": {
"type": "object",
"description": "A map containing custom configuration properties.
Please note that interoperability is not guaranteed.",
"properties": {
"compaction-strategy": {"type" : "string"},
"max-concurrent-file-group-rewrites": {"type": "number"}
},
"additionalProperties": {}
}
```
I think it will be good to "reserve" these names for reference and
standardization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
