Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
yihua merged PR #17691: URL: https://github.com/apache/hudi/pull/17691 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-369305 ## CI report: * ff6538099ab0229d8222baeb949aeb532d3c27d4 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10588) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3693202201 ## CI report: * 15896c5a4dfa87f89740e4b2d4c78c0f22c3b2fa Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10585) * ff6538099ab0229d8222baeb949aeb532d3c27d4 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10588) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3693198967 ## CI report: * 15896c5a4dfa87f89740e4b2d4c78c0f22c3b2fa Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10585) * ff6538099ab0229d8222baeb949aeb532d3c27d4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3693176485 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10566) * 15896c5a4dfa87f89740e4b2d4c78c0f22c3b2fa Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10585) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3693175090 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10566) * 15896c5a4dfa87f89740e4b2d4c78c0f22c3b2fa UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3692284636 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10566) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3692186436 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10566) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3692184970 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3692025630 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10552) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3691987409 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10552) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3691985773 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3691892210 ## CI report: * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10552) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3691866464 ## CI report: * 8e4e109689b46f051e767306c55b298a28e6c388 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10523) * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10552) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3691865191 ## CI report: * 8e4e109689b46f051e767306c55b298a28e6c388 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10523) * a87f0f6867b6579d99fcb42c7113ec96aee2dcfc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3690349426 ## CI report: * 8e4e109689b46f051e767306c55b298a28e6c388 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10523) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3690246392 ## CI report: * a9cdaaa9a7748eed43104a967c05a9b0a4157804 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10491) * 8e4e109689b46f051e767306c55b298a28e6c388 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10523) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3690244267 ## CI report: * a9cdaaa9a7748eed43104a967c05a9b0a4157804 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10491) * 8e4e109689b46f051e767306c55b298a28e6c388 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
the-other-tim-brown commented on code in PR #17691:
URL: https://github.com/apache/hudi/pull/17691#discussion_r2646030388
##
hudi-common/src/main/java/org/apache/hudi/common/util/CommitUtils.java:
##
@@ -49,7 +50,7 @@
public class CommitUtils {
private static final Logger LOG = LoggerFactory.getLogger(CommitUtils.class);
- private static final String NULL_SCHEMA_STR =
Schema.create(Schema.Type.NULL).toString();
+ private static final String NULL_SCHEMA_STR =
HoodieSchema.create(HoodieSchemaType.NULL).toString();
Review Comment:
Adding one now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
voonhous commented on code in PR #17691:
URL: https://github.com/apache/hudi/pull/17691#discussion_r2645721934
##
hudi-common/src/main/java/org/apache/hudi/common/util/CommitUtils.java:
##
@@ -49,7 +50,7 @@
public class CommitUtils {
private static final Logger LOG = LoggerFactory.getLogger(CommitUtils.class);
- private static final String NULL_SCHEMA_STR =
Schema.create(Schema.Type.NULL).toString();
+ private static final String NULL_SCHEMA_STR =
HoodieSchema.create(HoodieSchemaType.NULL).toString();
Review Comment:
`HoodieSchema.create(HoodieSchemaType.NULL)` seems to be used in quite a few
places, possible to just make this a public constant somewhere where we can
just import this to the other relevant classes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
voonhous commented on code in PR #17691:
URL: https://github.com/apache/hudi/pull/17691#discussion_r2645717983
##
hudi-common/src/test/java/org/apache/hudi/common/util/TestSortUtils.java:
##
@@ -77,7 +78,7 @@ record = new HoodieAvroRecord(new HoodieKey("record1",
"partition1"), payload);
record = new TestSparkRecord(new HoodieKey("record1", "partition1"),
payload);
}
String[] userSortColumns = new String[] {"non_pii_col", "timestamp"};
-FlatLists.ComparableList> comparableList =
SortUtils.getComparableSortColumns(record, userSortColumns,
Schema.parse(SCHEMA), suffixRecordKey, true);
+FlatLists.ComparableList> comparableList =
SortUtils.getComparableSortColumns(record, userSortColumns,
HoodieSchema.parse(SCHEMA), suffixRecordKey, true);
Review Comment:
Nit: We can use HoodieSchema to parse the `SCHEMA` string to a
`HoodieSchema`, after which, we can remove the importation of
`org.apache.avro.Schema`.
##
hudi-common/src/main/java/org/apache/hudi/common/util/CommitUtils.java:
##
@@ -49,7 +50,7 @@
public class CommitUtils {
private static final Logger LOG = LoggerFactory.getLogger(CommitUtils.class);
- private static final String NULL_SCHEMA_STR =
Schema.create(Schema.Type.NULL).toString();
+ private static final String NULL_SCHEMA_STR =
HoodieSchema.create(HoodieSchemaType.NULL).toString();
Review Comment:
`HoodieSchema.create(HoodieSchemaType.NULL)` seems to be used in quite a few
places, possible for us to just make this a public constant somewhere where we
can just import this to the other relevant classes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3688238041 ## CI report: * a9cdaaa9a7748eed43104a967c05a9b0a4157804 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10491) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3687897303 ## CI report: * a9cdaaa9a7748eed43104a967c05a9b0a4157804 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10491) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(schema): Migrate clustering operations to use HoodieSchema [hudi]
hudi-bot commented on PR #17691: URL: https://github.com/apache/hudi/pull/17691#issuecomment-3687893563 ## CI report: * a9cdaaa9a7748eed43104a967c05a9b0a4157804 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
