[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 2b7aa552fe32001973f4b25de5d6184bf52c1dd8 UNKNOWN * 274faca1645961b704d58506fda68caa107fe718 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=394) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
codecov-commenter edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-866110119 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3134](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a71e70e) into [master](https://codecov.io/gh/apache/hudi/commit/7bd517a82f9da3a6b42f6786da1d8a23151bdd94?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (7bd517a) will **increase** coverage by `3.82%`. > The diff coverage is `n/a`. > :exclamation: Current head a71e70e differs from pull request most recent head 51d09c6. Consider uploading reports for the commit 51d09c6 to get more accurate results [](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3134 +/- ## + Coverage 46.08% 49.91% +3.82% + Complexity 5319 395-4924 Files 914 66 -848 Lines 39596 2919 -36677 Branches 4269 318-3951 - Hits 18249 1457 -16792 + Misses19480 1327 -18153 + Partials 1867 135-1732 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `?` | | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `?` | | | huditimelineservice | `?` | | | hudiutilities | `49.91% <ø> (-8.50%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...ies/exception/HoodieSnapshotExporterException.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2V4Y2VwdGlvbi9Ib29kaWVTbmFwc2hvdEV4cG9ydGVyRXhjZXB0aW9uLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../apache/hudi/utilities/HoodieSnapshotExporter.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVNuYXBzaG90RXhwb3J0ZXIuamF2YQ==) | `5.17% <0.00%> (-83.63%)` | :arrow_down: | | [...hudi/utilities/schema/JdbcbasedSchemaProvider.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9KZGJjYmFzZWRTY2hlbWFQcm92aWRlci5qYXZh) | `0.00% <0.00%> (-72.23%)` | :arrow_down: | | [...org/apache/hudi/utilities/HDFSParquetImporter.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hERlNQYXJxdWV0SW1wb3J0ZXIuamF2YQ==) | `0.00% <0.00%> (-71.82%)` | :arrow_down: | | [...he/hudi/utilities/transform/AWSDmsTransformer.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3RyYW5zZm9ybS9BV1NEbXNUcmFuc2Zvcm1lci5qYXZh) | `0.00% <0.00%> (-66.67%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3133](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2b7aa55) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. > :exclamation: Current head 2b7aa55 differs from pull request most recent head 274faca. Consider uploading reports for the commit 274faca to get more accurate results [](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@Coverage Diff@@ ## master#3133 +/- ## = Coverage 46.13% 46.14% Complexity 5325 5325 = Files 915 915 Lines 3962239628+6 Branches 4272 4273+1 = + Hits 1828118285+4 - Misses1947019471+1 - Partials 1871 1872+1 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (ø)` | | | hudiclient | `30.45% <ø> (ø)` | | | hudicommon | `47.56% <ø> (-0.02%)` | :arrow_down: | | hudiflink | `61.26% <ø> (ø)` | | | hudihadoopmr | `51.29% <ø> (ø)` | | | hudisparkdatasource | `67.11% <100.00%> (+0.05%)` | :arrow_up: | | hudisync | `51.73% <ø> (ø)` | | | huditimelineservice | `64.36% <ø> (ø)` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...ala/org/apache/spark/sql/hudi/HoodieSqlUtils.scala](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9Ib29kaWVTcWxVdGlscy5zY2FsYQ==) | `69.64% <100.00%> (+0.55%)` | :arrow_up: | | [...ql/hudi/command/InsertIntoHoodieTableCommand.scala](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9jb21tYW5kL0luc2VydEludG9Ib29kaWVUYWJsZUNvbW1hbmQuc2NhbGE=) | `88.61% <100.00%> (+0.48%)` | :arrow_up: | | [...e/hudi/common/table/log/HoodieLogFormatWriter.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL2xvZy9Ib29kaWVMb2dGb3JtYXRXcml0ZXIuamF2YQ==) | `78.12% <0.00%> (-1.57%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=footer_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). Last update
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * a71e70e23cda06ba22cf012946575530566947b3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=396) * 51d09c6f734edb58c4f0736989b4f10eededa8d4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[hudi] branch master updated: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table (#3137)
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new 7e50f9a [HUDI-2061] Incorrect Schema Inference For Schema Evolved
Table (#3137)
7e50f9a is described below
commit 7e50f9a5a6be8d68b13a587a07a9af81819540ca
Author: pengzhiwei
AuthorDate: Thu Jun 24 13:48:01 2021 +0800
[HUDI-2061] Incorrect Schema Inference For Schema Evolved Table (#3137)
---
.../main/scala/org/apache/hudi/DefaultSource.scala| 19 ++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git
a/hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
b/hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
index 32bd9a4..d9f64ba 100644
---
a/hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
+++
b/hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
@@ -27,6 +27,7 @@ import org.apache.hudi.common.table.{HoodieTableMetaClient,
TableSchemaResolver}
import org.apache.hudi.exception.HoodieException
import org.apache.hudi.hadoop.HoodieROTablePathFilter
import org.apache.log4j.LogManager
+import org.apache.spark.sql.avro.SchemaConverters
import org.apache.spark.sql.execution.datasources.{DataSource,
FileStatusCache, HadoopFsRelation}
import org.apache.spark.sql.execution.datasources.orc.OrcFileFormat
import org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val tableSchemaResolver = new TableSchemaResolver(metaClient)
+try {
+
Some(SchemaConverters.toSqlType(tableSchemaResolver.getTableAvroSchema)
+.dataType.asInstanceOf[StructType])
+} catch {
+ case _: Throwable =>
+None // If there is no commit in the table, we can not get the
schema
+ // with tableSchemaResolver, return None here.
+}
+ } else {
+Some(schema)
+ }
// simply return as a regular relation
DataSource.apply(
sparkSession = sqlContext.sparkSession,
paths = extraReadPaths,
-userSpecifiedSchema = Option(schema),
+userSpecifiedSchema = specifySchema,
className = formatClassName,
options = optParams)
.resolveRelation()
[GitHub] [hudi] vinothchandar merged pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
vinothchandar merged pull request #3137: URL: https://github.com/apache/hudi/pull/3137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * 6658f31fa576e72437650235f3970f74415cf2d1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=391) * a71e70e23cda06ba22cf012946575530566947b3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=396) * 51d09c6f734edb58c4f0736989b4f10eededa8d4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] yuzhaojing commented on a change in pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
yuzhaojing commented on a change in pull request #3134:
URL: https://github.com/apache/hudi/pull/3134#discussion_r657638827
##
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/bootstrap/BootstrapFunction.java
##
@@ -168,33 +178,74 @@ private HoodieFlinkTable getTable() {
* @param partitionPath The partition path
*/
@SuppressWarnings("unchecked")
- private void loadRecords(String partitionPath, Collector out) {
+ private void loadRecords(String partitionPath, Collector out) throws
Exception {
long start = System.currentTimeMillis();
+
BaseFileUtils fileUtils =
BaseFileUtils.getInstance(this.hoodieTable.getBaseFileFormat());
-List latestBaseFiles =
-HoodieIndexUtils.getLatestBaseFilesForPartition(partitionPath,
this.hoodieTable);
-LOG.info("All baseFile in partition {} size = {}", partitionPath,
latestBaseFiles.size());
+Schema schema = new
TableSchemaResolver(this.hoodieTable.getMetaClient()).getTableAvroSchema();
final int parallelism = getRuntimeContext().getNumberOfParallelSubtasks();
final int maxParallelism =
getRuntimeContext().getMaxNumberOfParallelSubtasks();
final int taskID = getRuntimeContext().getIndexOfThisSubtask();
-for (HoodieBaseFile baseFile : latestBaseFiles) {
- boolean shouldLoad = KeyGroupRangeAssignment.assignKeyToParallelOperator(
- baseFile.getFileId(), maxParallelism, parallelism) == taskID;
-
- if (shouldLoad) {
-LOG.info("Load records from file {}.", baseFile);
-final List hoodieKeys;
-try {
- hoodieKeys =
- fileUtils.fetchRecordKeyPartitionPath(this.hadoopConf, new
Path(baseFile.getPath()));
-} catch (Exception e) {
- throw new HoodieException(String.format("Error when loading record
keys from file: %s", baseFile), e);
-}
-for (HoodieKey hoodieKey : hoodieKeys) {
- out.collect((O) new IndexRecord(generateHoodieRecord(hoodieKey,
baseFile)));
-}
+Option latestCommitTime =
this.hoodieTable.getMetaClient().getCommitsTimeline()
+.filterCompletedInstants().lastInstant();
+
+if (latestCommitTime.isPresent()) {
+ List fileSlices = this.hoodieTable.getSliceView()
+ .getLatestFileSlicesBeforeOrOn(partitionPath,
latestCommitTime.get().getTimestamp(), true)
+ .collect(toList());
+
+ for (FileSlice fileSlice : fileSlices) {
+// load parquet records
+fileSlice.getBaseFile().ifPresent(baseFile -> {
+ // filter out crushed files
+ if (baseFile.getFileSize() <= 0) {
+return;
+ }
+
+ if (shouldLoadFileId(baseFile.getFileId(), maxParallelism,
parallelism, taskID)) {
+LOG.info("Load records from file {}.", baseFile);
+final List hoodieKeys;
+try {
+ hoodieKeys =
+ fileUtils.fetchRecordKeyPartitionPath(this.hadoopConf, new
Path(baseFile.getPath()));
+} catch (Exception e) {
+ throw new HoodieException(String.format("Error when loading
record keys from file: %s", baseFile), e);
+}
+
+for (HoodieKey hoodieKey : hoodieKeys) {
+ out.collect((O) new IndexRecord(generateHoodieRecord(hoodieKey,
baseFile)));
+}
+ }
+});
+
+// load avro log records
+fileSlice.getLogFiles().forEach(logFile -> {
+ // filter out crushed files
Review comment:
ok, I already modify this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3092: [HUDI-1910] Commit Offset to Kafka after successful Hudi commit
codecov-commenter edited a comment on pull request #3092: URL: https://github.com/apache/hudi/pull/3092#issuecomment-864517429 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3133](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2b7aa55) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `29.61%`. > The diff coverage is `n/a`. > :exclamation: Current head 2b7aa55 differs from pull request most recent head 274faca. Consider uploading reports for the commit 274faca to get more accurate results [](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3133 +/- ## = - Coverage 46.13% 16.52% -29.62% + Complexity 5325 467 -4858 = Files 915 274 -641 Lines 3962210649-28973 Branches 4272 1088 -3184 = - Hits 18281 1760-16521 + Misses19470 8730-10740 + Partials 1871 159 -1712 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0676b60) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `0.00%`. > The diff coverage is `50.00%`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3137 +/- ## - Coverage 46.13% 46.13% -0.01% Complexity 5325 5325 Files 915 915 Lines 3962239629 +7 Branches 4272 4273 +1 + Hits 1828118282 +1 - Misses1947019474 +4 - Partials 1871 1873 +2 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (ø)` | | | hudiclient | `30.45% <ø> (ø)` | | | hudicommon | `47.56% <ø> (-0.02%)` | :arrow_down: | | hudiflink | `61.26% <ø> (ø)` | | | hudihadoopmr | `51.29% <ø> (ø)` | | | hudisparkdatasource | `67.00% <50.00%> (-0.05%)` | :arrow_down: | | hudisync | `51.73% <ø> (ø)` | | | huditimelineservice | `64.36% <ø> (ø)` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...src/main/scala/org/apache/hudi/DefaultSource.scala](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0RlZmF1bHRTb3VyY2Uuc2NhbGE=) | `75.22% <50.00%> (-2.23%)` | :arrow_down: | | [...e/hudi/common/table/log/HoodieLogFormatWriter.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL2xvZy9Ib29kaWVMb2dGb3JtYXRXcml0ZXIuamF2YQ==) | `78.12% <0.00%> (-1.57%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=footer_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). Last update [e039e0f...0676b60](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=lastupdated_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0676b60) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `3.44%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3137 +/- ## - Coverage 46.13% 42.68% -3.45% + Complexity 5325 4737 -588 Files 915 837 -78 Lines 3962235273-4349 Branches 4272 3604 -668 - Hits 1828115058-3223 + Misses1947018818 -652 + Partials 1871 1397 -474 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (ø)` | | | hudiclient | `30.45% <ø> (ø)` | | | hudicommon | `47.56% <ø> (-0.02%)` | :arrow_down: | | hudiflink | `61.26% <ø> (ø)` | | | hudihadoopmr | `51.29% <ø> (ø)` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...he/hudi/hive/HiveStylePartitionValueExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN0eWxlUGFydGl0aW9uVmFsdWVFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...main/java/org/apache/hudi/hive/HiveSyncConfig.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNDb25maWcuamF2YQ==) | `0.00% <0.00%> (-97.83%)` | :arrow_down: | | [.../apache/hudi/hive/MultiPartKeysValueExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTXVsdGlQYXJ0S2V5c1ZhbHVlRXh0cmFjdG9yLmphdmE=) | `0.00% <0.00%> (-90.91%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/SchemaDifference.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvU2NoZW1hRGlmZmVyZW5jZS5qYXZh) | `0.00% <0.00%> (-84.85%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3092: [HUDI-1910] Commit Offset to Kafka after successful Hudi commit
codecov-commenter edited a comment on pull request #3092: URL: https://github.com/apache/hudi/pull/3092#issuecomment-864517429 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3092](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1ae0313) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `17.23%`. > The diff coverage is `81.81%`. [](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3092 +/- ## = - Coverage 46.13% 28.90% -17.24% + Complexity 5325 1259 -4066 = Files 915 370 -545 Lines 3962214116-25506 Branches 4272 1441 -2831 = - Hits 18281 4080-14201 + Misses19470 9740 -9730 + Partials 1871 296 -1575 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `22.29% <ø> (-8.17%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.50% <81.81%> (+0.12%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `70.70% <0.00%> (-0.15%)` | :arrow_down: | | [...hudi/utilities/sources/helpers/KafkaOffsetGen.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvaGVscGVycy9LYWZrYU9mZnNldEdlbi5qYXZh) | `87.09% <90.00%> (-0.18%)` | :arrow_down: | | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0676b60) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `17.29%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3137 +/- ## = - Coverage 46.13% 28.84% -17.30% + Complexity 5325 1255 -4070 = Files 915 370 -545 Lines 3962214100-25522 Branches 4272 1440 -2832 = - Hits 18281 4067-14214 + Misses19470 9737 -9733 + Partials 1871 296 -1575 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `22.29% <ø> (-8.17%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3092: [HUDI-1910] Commit Offset to Kafka after successful Hudi commit
hudi-bot edited a comment on pull request #3092: URL: https://github.com/apache/hudi/pull/3092#issuecomment-862136673 ## CI report: * 1ae031303ebfa92fd153b8b6c0a404d1aac39a0a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=393) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * 6658f31fa576e72437650235f3970f74415cf2d1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=391) * a71e70e23cda06ba22cf012946575530566947b3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=396) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 263d5ee8e39e5c1e159311a56032b5ba5ce139de Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=387) * 2b7aa552fe32001973f4b25de5d6184bf52c1dd8 UNKNOWN * 274faca1645961b704d58506fda68caa107fe718 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=394) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * 6658f31fa576e72437650235f3970f74415cf2d1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=391) * a71e70e23cda06ba22cf012946575530566947b3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3092: [HUDI-1910] Commit Offset to Kafka after successful Hudi commit
codecov-commenter edited a comment on pull request #3092: URL: https://github.com/apache/hudi/pull/3092#issuecomment-864517429 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3092](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1ae0313) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `29.51%`. > The diff coverage is `81.81%`. [](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3092 +/- ## = - Coverage 46.13% 16.62% -29.52% + Complexity 5325 471 -4854 = Files 915 274 -641 Lines 3962210665-28957 Branches 4272 1089 -3183 = - Hits 18281 1773-16508 + Misses19470 8733-10737 + Partials 1871 159 -1712 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.50% <81.81%> (+0.12%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `70.70% <0.00%> (-0.15%)` | :arrow_down: | | [...hudi/utilities/sources/helpers/KafkaOffsetGen.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvaGVscGVycy9LYWZrYU9mZnNldEdlbi5qYXZh) | `87.09% <90.00%> (-0.18%)` | :arrow_down: | | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3133](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2b7aa55) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `43.00%`. > The diff coverage is `n/a`. > :exclamation: Current head 2b7aa55 differs from pull request most recent head 274faca. Consider uploading reports for the commit 274faca to get more accurate results [](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3133 +/- ## - Coverage 46.13% 3.13% -43.01% + Complexity 5325 82 -5243 Files 915 274 -641 Lines 39622 10649-28973 Branches 42721088 -3184 - Hits 18281 334-17947 + Misses19470 10289 -9181 + Partials 1871 26 -1845 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `9.52% <ø> (-48.86%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * b927fe06c9de3942da07d44a91f2848bd875ddb8 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=392) * cdf677ec65776afcad061ba83166c16073ca82c3 UNKNOWN * 0676b60e140439c65ff2f9cf4f36db72fa97ab9c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=395) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 263d5ee8e39e5c1e159311a56032b5ba5ce139de Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=387) * 2b7aa552fe32001973f4b25de5d6184bf52c1dd8 UNKNOWN * 274faca1645961b704d58506fda68caa107fe718 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0676b60) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `29.61%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3137 +/- ## = - Coverage 46.13% 16.52% -29.62% + Complexity 5325 467 -4858 = Files 915 274 -641 Lines 3962210649-28973 Branches 4272 1088 -3184 = - Hits 18281 1760-16521 + Misses19470 8730-10740 + Partials 1871 159 -1712 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 263d5ee8e39e5c1e159311a56032b5ba5ce139de Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=387) * 2b7aa552fe32001973f4b25de5d6184bf52c1dd8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3092: [HUDI-1910] Commit Offset to Kafka after successful Hudi commit
codecov-commenter edited a comment on pull request #3092: URL: https://github.com/apache/hudi/pull/3092#issuecomment-864517429 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3092](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (1ae0313) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `43.00%`. > The diff coverage is `0.00%`. [](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3092 +/- ## - Coverage 46.13% 3.13% -43.01% + Complexity 5325 82 -5243 Files 915 274 -641 Lines 39622 10665-28957 Branches 42721089 -3183 - Hits 18281 334-17947 + Misses19470 10305 -9165 + Partials 1871 26 -1845 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `9.47% <0.00%> (-48.91%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3092?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...apache/hudi/utilities/deltastreamer/DeltaSync.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL2RlbHRhc3RyZWFtZXIvRGVsdGFTeW5jLmphdmE=) | `0.00% <0.00%> (-70.85%)` | :arrow_down: | | [...hudi/utilities/sources/helpers/KafkaOffsetGen.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvaGVscGVycy9LYWZrYU9mZnNldEdlbi5qYXZh) | `0.00% <0.00%> (-87.28%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3092/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0676b60) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `43.00%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3137 +/- ## - Coverage 46.13% 3.13% -43.01% + Complexity 5325 82 -5243 Files 915 274 -641 Lines 39622 10649-28973 Branches 42721088 -3184 - Hits 18281 334-17947 + Misses19470 10289 -9181 + Partials 1871 26 -1845 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `9.52% <ø> (-48.86%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3092: [HUDI-1910] Commit Offset to Kafka after successful Hudi commit
hudi-bot edited a comment on pull request #3092: URL: https://github.com/apache/hudi/pull/3092#issuecomment-862136673 ## CI report: * 2cb652554386030acaf569a73f9e074cbc72ffa3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=324) * 1ae031303ebfa92fd153b8b6c0a404d1aac39a0a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=393) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] veenaypatil commented on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen
veenaypatil commented on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-867330097 @vinothchandar make sense, will add those tests as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3092: [HUDI-1910] Commit Offset to Kafka after successful Hudi commit
hudi-bot edited a comment on pull request #3092: URL: https://github.com/apache/hudi/pull/3092#issuecomment-862136673 ## CI report: * 2cb652554386030acaf569a73f9e074cbc72ffa3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=324) * 1ae031303ebfa92fd153b8b6c0a404d1aac39a0a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * 6658f31fa576e72437650235f3970f74415cf2d1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=391) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
codecov-commenter edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-866110119 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3134](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (6658f31) into [master](https://codecov.io/gh/apache/hudi/commit/7bd517a82f9da3a6b42f6786da1d8a23151bdd94?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (7bd517a) will **decrease** coverage by `17.24%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3134 +/- ## = - Coverage 46.08% 28.84% -17.25% + Complexity 5319 1255 -4064 = Files 914 370 -544 Lines 3959614100-25496 Branches 4269 1440 -2829 = - Hits 18249 4067-14182 + Misses19480 9737 -9743 + Partials 1867 296 -1571 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `22.29% <ø> (-8.15%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] vinothchandar commented on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
vinothchandar commented on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-867322948 Per conversation between you both, I ll land this once CI passes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * b927fe06c9de3942da07d44a91f2848bd875ddb8 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=392) * cdf677ec65776afcad061ba83166c16073ca82c3 UNKNOWN * 0676b60e140439c65ff2f9cf4f36db72fa97ab9c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * d5606e08da3a088b43df6c224d4843996285aea2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=388) * b927fe06c9de3942da07d44a91f2848bd875ddb8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=392) * cdf677ec65776afcad061ba83166c16073ca82c3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
pengzhiwei2018 commented on a change in pull request #3137:
URL: https://github.com/apache/hudi/pull/3137#discussion_r657607681
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
+try {
+ Some(SchemaConverters.toSqlType(schemaUtil.getTableAvroSchema)
+.dataType.asInstanceOf[StructType])
+} catch {
+ case _: Throwable =>
+None // If there is no commit in the table, we can not get the
schema
+ // with schemaUtil, return None here.
+}
+ } else {
+None
Review comment:
Thanks for remind me this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
pengzhiwei2018 commented on a change in pull request #3137:
URL: https://github.com/apache/hudi/pull/3137#discussion_r657605824
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
+try {
+ Some(SchemaConverters.toSqlType(schemaUtil.getTableAvroSchema)
+.dataType.asInstanceOf[StructType])
+} catch {
+ case _: Throwable =>
+None // If there is no commit in the table, we can not get the
schema
+ // with schemaUtil, return None here.
+}
+ } else {
+None
Review comment:
The schema is null at this if-branch, so it should be None.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * d5606e08da3a088b43df6c224d4843996285aea2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=388) * b927fe06c9de3942da07d44a91f2848bd875ddb8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
codecov-commenter edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-866110119 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3134](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (6658f31) into [master](https://codecov.io/gh/apache/hudi/commit/7bd517a82f9da3a6b42f6786da1d8a23151bdd94?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (7bd517a) will **decrease** coverage by `29.56%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3134 +/- ## = - Coverage 46.08% 16.52% -29.57% + Complexity 5319 467 -4852 = Files 914 274 -640 Lines 3959610649-28947 Branches 4269 1088 -3181 = - Hits 18249 1760-16489 + Misses19480 8730-10750 + Partials 1867 159 -1708 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.45%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] zhangyue19921010 opened a new pull request #3145: [Docs] Improving Hudi Configurations docs
zhangyue19921010 opened a new pull request #3145: URL: https://github.com/apache/hudi/pull/3145 ## What is the purpose of the pull request 1. Show the configurations in the form of a table making it easier to read and find | Option Name | Property | Required | Default | Remarks | | --- | | | --- | --- | | xxx | xxx | xxx | xxx | xxx | 2. Fix some document presentation problems: - Missing default value of `hoodie.hbase.index.update.partition.path` false - Duplicate configs of `hoodie.parquet.compression.codec` - Wrong property of `withBootstrapModeForRegexMatch(modeForRegexMatch = org.apache.hudi.client.bootstrap.METADATA_ONLY)` - Wrong property of `withDatadogApiTimeoutSeconds(timeout = 3)` - Wrong unit for `hoodie.parquet.max.file.size` default value `120MB` which should be `125829120` in bytes and similar errors - etc ... ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc changes done or have another open PR - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
pengzhiwei2018 commented on a change in pull request #3137:
URL: https://github.com/apache/hudi/pull/3137#discussion_r657605824
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
+try {
+ Some(SchemaConverters.toSqlType(schemaUtil.getTableAvroSchema)
+.dataType.asInstanceOf[StructType])
+} catch {
+ case _: Throwable =>
+None // If there is no commit in the table, we can not get the
schema
+ // with schemaUtil, return None here.
+}
+ } else {
+None
Review comment:
The schema is null at this if-branch, so it should be None.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
pengzhiwei2018 commented on a change in pull request #3137:
URL: https://github.com/apache/hudi/pull/3137#discussion_r657605651
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
+try {
+ Some(SchemaConverters.toSqlType(schemaUtil.getTableAvroSchema)
Review comment:
Yes, we can refactor this code in another PR.
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
Review comment:
done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3138](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (8d126fb) into [master](https://codecov.io/gh/apache/hudi/commit/2687eab8f0b49f410c49fac7771a8409722574cf?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2687eab) will **increase** coverage by `12.31%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3138 +/- ## = + Coverage 16.53% 28.85% +12.31% - Complexity 468 1256 +788 = Files 274 370 +96 Lines 1064914100 +3451 Branches 1088 1440 +352 = + Hits 1761 4068 +2307 - Misses 8730 9737 +1007 - Partials158 295 +137 ``` | Flag | Coverage Δ | | |---|---|---| | hudiclient | `22.29% <ø> (+22.29%)` | :arrow_up: | | hudisync | `6.72% <ø> (ø)` | | | hudiutilities | `58.41% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/client/SparkRDDWriteClient.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L1NwYXJrUkREV3JpdGVDbGllbnQuamF2YQ==) | `72.05% <0.00%> (ø)` | | | [...commit/SparkUpsertPreppedCommitActionExecutor.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdGFibGUvYWN0aW9uL2NvbW1pdC9TcGFya1Vwc2VydFByZXBwZWRDb21taXRBY3Rpb25FeGVjdXRvci5qYXZh) | `100.00% <0.00%> (ø)` | | | [...t/SparkInsertPreppedDeltaCommitActionExecutor.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdGFibGUvYWN0aW9uL2RlbHRhY29tbWl0L1NwYXJrSW5zZXJ0UHJlcHBlZERlbHRhQ29tbWl0QWN0aW9uRXhlY3V0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...a/org/apache/hudi/client/HoodieSparkCompactor.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L0hvb2RpZVNwYXJrQ29tcGFjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...he/hudi/table/action/commit/UpsertPartitioner.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdGFibGUvYWN0aW9uL2NvbW1pdC9VcHNlcnRQYXJ0aXRpb25lci5qYXZh) | `96.05% <0.00%> (ø)` | | | [...e/action/commit/BaseSparkCommitActionExecutor.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdGFibGUvYWN0aW9uL2NvbW1pdC9CYXNlU3BhcmtDb21taXRBY3Rpb25FeGVjdXRvci5qYXZh) | `82.51% <0.00%> (ø)` | | |
[jira] [Assigned] (HUDI-2067) Sync all the options of FlinkOptions to FlinkStreamerConfig
[
https://issues.apache.org/jira/browse/HUDI-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinay reassigned HUDI-2067:
---
Assignee: Vinay
> Sync all the options of FlinkOptions to FlinkStreamerConfig
> ---
>
> Key: HUDI-2067
> URL: https://issues.apache.org/jira/browse/HUDI-2067
> Project: Apache Hudi
> Issue Type: Task
> Components: Flink Integration
>Reporter: Danny Chen
>Assignee: Vinay
>Priority: Major
> Fix For: 0.9.0
>
>
> Sync the options so that the {{HoodieFlinkStreamer}} can have more config
> options.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] nsivabalan commented on a change in pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
nsivabalan commented on a change in pull request #3137:
URL: https://github.com/apache/hudi/pull/3137#discussion_r657602819
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
+try {
+ Some(SchemaConverters.toSqlType(schemaUtil.getTableAvroSchema)
+.dataType.asInstanceOf[StructType])
+} catch {
+ case _: Throwable =>
+None // If there is no commit in the table, we can not get the
schema
+ // with schemaUtil, return None here.
+}
+ } else {
+None
Review comment:
basically can we have a test which hits this else block(231). I assume
all tests hit line 223. schema evol tests hits 227.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (HUDI-2067) Sync all the options of FlinkOptions to FlinkStreamerConfig
Danny Chen created HUDI-2067:
Summary: Sync all the options of FlinkOptions to
FlinkStreamerConfig
Key: HUDI-2067
URL: https://issues.apache.org/jira/browse/HUDI-2067
Project: Apache Hudi
Issue Type: Task
Components: Flink Integration
Reporter: Danny Chen
Fix For: 0.9.0
Sync the options so that the {{HoodieFlinkStreamer}} can have more config
options.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] nsivabalan closed pull request #3144: [MINOR] disabling flaky tests to unblock PRs
nsivabalan closed pull request #3144: URL: https://github.com/apache/hudi/pull/3144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * 886976b7067dc3062b6d465f6d962c3b7db012a0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=363) * 6658f31fa576e72437650235f3970f74415cf2d1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=391) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3144: [MINOR] disabling flaky tests to unblock PRs
nsivabalan commented on pull request #3144: URL: https://github.com/apache/hudi/pull/3144#issuecomment-867310110 oh, awesome. thanks for the update. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
nsivabalan commented on a change in pull request #3137:
URL: https://github.com/apache/hudi/pull/3137#discussion_r657598480
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
Review comment:
can we name the variable as "tableSchemaResolver".
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
+try {
+ Some(SchemaConverters.toSqlType(schemaUtil.getTableAvroSchema)
+.dataType.asInstanceOf[StructType])
+} catch {
+ case _: Throwable =>
+None // If there is no commit in the table, we can not get the
schema
+ // with schemaUtil, return None here.
+}
+ } else {
+None
Review comment:
shouldn't this be Option.of(schema) ?
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -213,11 +214,27 @@ class DefaultSource extends RelationProvider
classOf[HoodieROTablePathFilter],
classOf[org.apache.hadoop.fs.PathFilter])
+ val specifySchema = if (schema == null) {
+// Load the schema from the commit meta data.
+// Here we should specify the schema to the latest commit schema since
+// the table schema evolution.
+val schemaUtil = new TableSchemaResolver(metaClient)
+try {
+ Some(SchemaConverters.toSqlType(schemaUtil.getTableAvroSchema)
Review comment:
Ideally getTableAvroSchema should return Option and callers
should check if present and proceed accordingly. But in the interest of
unblocking others, I am ok with this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a change in pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
danny0405 commented on a change in pull request #3134:
URL: https://github.com/apache/hudi/pull/3134#discussion_r657601079
##
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/bootstrap/BootstrapFunction.java
##
@@ -168,33 +178,74 @@ private HoodieFlinkTable getTable() {
* @param partitionPath The partition path
*/
@SuppressWarnings("unchecked")
- private void loadRecords(String partitionPath, Collector out) {
+ private void loadRecords(String partitionPath, Collector out) throws
Exception {
long start = System.currentTimeMillis();
+
BaseFileUtils fileUtils =
BaseFileUtils.getInstance(this.hoodieTable.getBaseFileFormat());
-List latestBaseFiles =
-HoodieIndexUtils.getLatestBaseFilesForPartition(partitionPath,
this.hoodieTable);
-LOG.info("All baseFile in partition {} size = {}", partitionPath,
latestBaseFiles.size());
+Schema schema = new
TableSchemaResolver(this.hoodieTable.getMetaClient()).getTableAvroSchema();
final int parallelism = getRuntimeContext().getNumberOfParallelSubtasks();
final int maxParallelism =
getRuntimeContext().getMaxNumberOfParallelSubtasks();
final int taskID = getRuntimeContext().getIndexOfThisSubtask();
-for (HoodieBaseFile baseFile : latestBaseFiles) {
- boolean shouldLoad = KeyGroupRangeAssignment.assignKeyToParallelOperator(
- baseFile.getFileId(), maxParallelism, parallelism) == taskID;
-
- if (shouldLoad) {
-LOG.info("Load records from file {}.", baseFile);
-final List hoodieKeys;
-try {
- hoodieKeys =
- fileUtils.fetchRecordKeyPartitionPath(this.hadoopConf, new
Path(baseFile.getPath()));
-} catch (Exception e) {
- throw new HoodieException(String.format("Error when loading record
keys from file: %s", baseFile), e);
-}
-for (HoodieKey hoodieKey : hoodieKeys) {
- out.collect((O) new IndexRecord(generateHoodieRecord(hoodieKey,
baseFile)));
-}
+Option latestCommitTime =
this.hoodieTable.getMetaClient().getCommitsTimeline()
+.filterCompletedInstants().lastInstant();
+
+if (latestCommitTime.isPresent()) {
+ List fileSlices = this.hoodieTable.getSliceView()
+ .getLatestFileSlicesBeforeOrOn(partitionPath,
latestCommitTime.get().getTimestamp(), true)
+ .collect(toList());
+
+ for (FileSlice fileSlice : fileSlices) {
+// load parquet records
+fileSlice.getBaseFile().ifPresent(baseFile -> {
+ // filter out crushed files
+ if (baseFile.getFileSize() <= 0) {
+return;
+ }
+
+ if (shouldLoadFileId(baseFile.getFileId(), maxParallelism,
parallelism, taskID)) {
+LOG.info("Load records from file {}.", baseFile);
+final List hoodieKeys;
+try {
+ hoodieKeys =
+ fileUtils.fetchRecordKeyPartitionPath(this.hadoopConf, new
Path(baseFile.getPath()));
+} catch (Exception e) {
+ throw new HoodieException(String.format("Error when loading
record keys from file: %s", baseFile), e);
+}
+
+for (HoodieKey hoodieKey : hoodieKeys) {
+ out.collect((O) new IndexRecord(generateHoodieRecord(hoodieKey,
baseFile)));
+}
+ }
+});
+
+// load avro log records
+fileSlice.getLogFiles().forEach(logFile -> {
+ // filter out crushed files
Review comment:
You can read the logs all together with the scanner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
codecov-commenter edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-866110119 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3134](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (6658f31) into [master](https://codecov.io/gh/apache/hudi/commit/7bd517a82f9da3a6b42f6786da1d8a23151bdd94?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (7bd517a) will **decrease** coverage by `42.95%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3134 +/- ## - Coverage 46.08% 3.13% -42.96% + Complexity 5319 82 -5237 Files 914 274 -640 Lines 39596 10649-28947 Branches 42691088 -3181 - Hits 18249 334-17915 + Misses19480 10289 -9191 + Partials 1867 26 -1841 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.45%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `9.52% <ø> (-48.89%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3134?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/3134/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter commented on pull request #3144: [MINOR] disabling flaky tests to unblock PRs
codecov-commenter commented on pull request #3144: URL: https://github.com/apache/hudi/pull/3144#issuecomment-867308845 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3144?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3144](https://codecov.io/gh/apache/hudi/pull/3144?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (7b1eb80) into [master](https://codecov.io/gh/apache/hudi/commit/380518e232b883bb12579b3e98283659b464285a?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (380518e) will **decrease** coverage by `27.62%`. > The diff coverage is `n/a`. > :exclamation: Current head 7b1eb80 differs from pull request most recent head bb834ee. Consider uploading reports for the commit bb834ee to get more accurate results [](https://codecov.io/gh/apache/hudi/pull/3144?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3144 +/- ## = - Coverage 44.15% 16.52% -27.63% + Complexity 4537 467 -4070 = Files 819 274 -545 Lines 3617110649-25522 Branches 3920 1088 -2832 = - Hits 15970 1760-14210 + Misses18467 8730 -9737 + Partials 1734 159 -1575 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-16.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3144?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3144/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3144/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3144/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3144/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3144/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3144: [MINOR] disabling flaky tests to unblock PRs
hudi-bot edited a comment on pull request #3144: URL: https://github.com/apache/hudi/pull/3144#issuecomment-867295856 ## CI report: * 7b1eb80f604bd694f28357ca2ed4f1c54231bc06 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=390) * bb834eec0bf2d89eaaf31dfbe8f52792325becc1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
hudi-bot edited a comment on pull request #3134: URL: https://github.com/apache/hudi/pull/3134#issuecomment-865885733 ## CI report: * 886976b7067dc3062b6d465f6d962c3b7db012a0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=363) * 6658f31fa576e72437650235f3970f74415cf2d1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely
hudi-bot edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-87254 ## CI report: * 4346314677d2969e441545f796aff2a37126fa49 UNKNOWN * 8d126fbff62054936249ed6f5b0bcc46416a4b02 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=389) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3138](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (8d126fb) into [master](https://codecov.io/gh/apache/hudi/commit/2687eab8f0b49f410c49fac7771a8409722574cf?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2687eab) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@Coverage Diff@@ ## master#3138 +/- ## = Coverage 16.53% 16.53% Complexity 468 468 = Files 274 274 Lines 1064910649 Branches 1088 1088 = Hits 1761 1761 Misses 8730 8730 Partials158 158 ``` | Flag | Coverage Δ | | |---|---|---| | hudiclient | `0.00% <ø> (ø)` | | | hudisync | `6.72% <ø> (ø)` | | | hudiutilities | `58.41% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=footer_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). Last update [2687eab...8d126fb](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=lastupdated_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3144: [MINOR] disabling flaky tests to unblock PRs
hudi-bot edited a comment on pull request #3144: URL: https://github.com/apache/hudi/pull/3144#issuecomment-867295856 ## CI report: * 7b1eb80f604bd694f28357ca2ed4f1c54231bc06 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=390) * bb834eec0bf2d89eaaf31dfbe8f52792325becc1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] leesf commented on pull request #3144: [MINOR] disabling flaky tests to unblock PRs
leesf commented on pull request #3144: URL: https://github.com/apache/hudi/pull/3144#issuecomment-867304916 @nsivabalan TestHoodieBackedMetadata#testOnlyValidPartitionsAdded has been fixed https://github.com/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09 and master branch passed now, and there is also a fix to another test, see https://github.com/apache/hudi/pull/3137 so we would just close the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3138](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (8d126fb) into [master](https://codecov.io/gh/apache/hudi/commit/2687eab8f0b49f410c49fac7771a8409722574cf?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2687eab) will **decrease** coverage by `13.40%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3138 +/- ## - Coverage 16.53% 3.13% -13.41% + Complexity 468 82 -386 Files 274 274 Lines 10649 10649 Branches 10881088 - Hits 1761 334 -1427 - Misses 8730 10289 +1559 + Partials158 26 -132 ``` | Flag | Coverage Δ | | |---|---|---| | hudiclient | `0.00% <ø> (ø)` | | | hudisync | `6.72% <ø> (ø)` | | | hudiutilities | `9.52% <ø> (-48.89%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/utilities/sources/CsvDFSSource.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQ3N2REZTU291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3144: [MINOR] disabling flaky tests to unblock PRs
hudi-bot edited a comment on pull request #3144: URL: https://github.com/apache/hudi/pull/3144#issuecomment-867295856 ## CI report: * 7b1eb80f604bd694f28357ca2ed4f1c54231bc06 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=390) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d5606e0) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `0.00%`. > The diff coverage is `50.00%`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3137 +/- ## - Coverage 46.13% 46.13% -0.01% + Complexity 5325 5324 -1 Files 915 915 Lines 3962239629 +7 Branches 4272 4273 +1 Hits 1828118281 - Misses1947019476 +6 - Partials 1871 1872 +1 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (ø)` | | | hudiclient | `30.45% <ø> (ø)` | | | hudicommon | `47.55% <ø> (-0.03%)` | :arrow_down: | | hudiflink | `61.26% <ø> (ø)` | | | hudihadoopmr | `51.29% <ø> (ø)` | | | hudisparkdatasource | `67.00% <50.00%> (-0.05%)` | :arrow_down: | | hudisync | `51.73% <ø> (ø)` | | | huditimelineservice | `64.36% <ø> (ø)` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...src/main/scala/org/apache/hudi/DefaultSource.scala](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9odWRpL0RlZmF1bHRTb3VyY2Uuc2NhbGE=) | `75.22% <50.00%> (-2.23%)` | :arrow_down: | | [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `79.31% <0.00%> (-10.35%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=footer_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). Last update [e039e0f...d5606e0](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=lastupdated_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3144: [MINOR] disabling flaky tests to unblock PRs
hudi-bot commented on pull request #3144: URL: https://github.com/apache/hudi/pull/3144#issuecomment-867295856 ## CI report: * 7b1eb80f604bd694f28357ca2ed4f1c54231bc06 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely
hudi-bot edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-87254 ## CI report: * 0b3283067503c07bd1168cd88cae6d4bf8f215ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=375) * 4346314677d2969e441545f796aff2a37126fa49 UNKNOWN * 8d126fbff62054936249ed6f5b0bcc46416a4b02 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=389) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * d5606e08da3a088b43df6c224d4843996285aea2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=388) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 263d5ee8e39e5c1e159311a56032b5ba5ce139de Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=387) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d5606e0) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `3.45%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3137 +/- ## - Coverage 46.13% 42.68% -3.46% + Complexity 5325 4736 -589 Files 915 837 -78 Lines 3962235273-4349 Branches 4272 3604 -668 - Hits 1828115057-3224 + Misses1947018820 -650 + Partials 1871 1396 -475 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (ø)` | | | hudiclient | `30.45% <ø> (ø)` | | | hudicommon | `47.55% <ø> (-0.03%)` | :arrow_down: | | hudiflink | `61.26% <ø> (ø)` | | | hudihadoopmr | `51.29% <ø> (ø)` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...he/hudi/hive/HiveStylePartitionValueExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN0eWxlUGFydGl0aW9uVmFsdWVFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...main/java/org/apache/hudi/hive/HiveSyncConfig.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNDb25maWcuamF2YQ==) | `0.00% <0.00%> (-97.83%)` | :arrow_down: | | [.../apache/hudi/hive/MultiPartKeysValueExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTXVsdGlQYXJ0S2V5c1ZhbHVlRXh0cmFjdG9yLmphdmE=) | `0.00% <0.00%> (-90.91%)` | :arrow_down: | | [...in/java/org/apache/hudi/hive/SchemaDifference.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvU2NoZW1hRGlmZmVyZW5jZS5qYXZh) | `0.00% <0.00%> (-84.85%)` | :arrow_down: | |
[GitHub] [hudi] nsivabalan opened a new pull request #3144: [MINOR] disabling flaky tests to unblock PRs
nsivabalan opened a new pull request #3144: URL: https://github.com/apache/hudi/pull/3144 ## What is the purpose of the pull request Disabling flaky tests to unblock other PRs. have filed separate jiras to track each flaky tests being disabled. https://issues.apache.org/jira/browse/HUDI-2065 https://issues.apache.org/jira/browse/HUDI-2066 ## Brief change log *(for example:)* - *Modify AnnotationLocation checkstyle rule in checkstyle.xml* ## Verify this pull request *(Please pick either of the following options)* This pull request is a trivial rework / code cleanup without any test coverage. *(or)* This pull request is already covered by existing tests, such as *(please describe tests)*. (or) This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end.* - *Added HoodieClientWriteTest to verify the change.* - *Manually verified the change by running a job locally.* ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc changes done or have another open PR - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d5606e0) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `17.29%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3137 +/- ## = - Coverage 46.13% 28.84% -17.30% + Complexity 5325 1255 -4070 = Files 915 370 -545 Lines 3962214100-25522 Branches 4272 1440 -2832 = - Hits 18281 4067-14214 + Misses19470 9737 -9733 + Partials 1871 296 -1575 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `22.29% <ø> (-8.17%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely
hudi-bot edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-87254 ## CI report: * 0b3283067503c07bd1168cd88cae6d4bf8f215ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=375) * 4346314677d2969e441545f796aff2a37126fa49 UNKNOWN * 8d126fbff62054936249ed6f5b0bcc46416a4b02 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3133](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (263d5ee) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `17.29%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3133 +/- ## = - Coverage 46.13% 28.84% -17.30% + Complexity 5325 1255 -4070 = Files 915 370 -545 Lines 3962214100-25522 Branches 4272 1440 -2832 = - Hits 18281 4067-14214 + Misses19470 9737 -9733 + Partials 1871 296 -1575 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `22.29% <ø> (-8.17%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[jira] [Updated] (HUDI-2062) Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely
[ https://issues.apache.org/jira/browse/HUDI-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuzhaojing updated HUDI-2062: - Summary: Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely (was: Catch IOException in WriteProfiles #getCommitMetadataSafely) > Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely > - > > Key: HUDI-2062 > URL: https://issues.apache.org/jira/browse/HUDI-2062 > Project: Apache Hudi > Issue Type: Bug > Components: Flink Integration >Reporter: yuzhaojing >Assignee: yuzhaojing >Priority: Major > Labels: pull-request-available > > The function WriteProfiles #getCommitMetadataSafely expect get instant > safely, if instant deleted by cleaner that ignore it. > But in case, FileNotFoundException will throw out of try catch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HUDI-2066) Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution
[
https://issues.apache.org/jira/browse/HUDI-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan reassigned HUDI-2066:
-
Assignee: sivabalan narayanan
> Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution
> ---
>
> Key: HUDI-2066
> URL: https://issues.apache.org/jira/browse/HUDI-2066
> Project: Apache Hudi
> Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: sivabalan narayanan
>Assignee: sivabalan narayanan
>Priority: Major
>
> {quote}
> [2] [https://travis-ci.com/github/apache/hudi/jobs/518067393]{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] hudi-bot edited a comment on pull request #3138: [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadataSafely
hudi-bot edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-87254 ## CI report: * 0b3283067503c07bd1168cd88cae6d4bf8f215ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=375) * 4346314677d2969e441545f796aff2a37126fa49 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HUDI-2066) Fix flaky test: HoodieSparkSqlWriterSuite.test schema evolution
sivabalan narayanan created HUDI-2066:
-
Summary: Fix flaky test: HoodieSparkSqlWriterSuite.test schema
evolution
Key: HUDI-2066
URL: https://issues.apache.org/jira/browse/HUDI-2066
Project: Apache Hudi
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: sivabalan narayanan
{quote}
[2] [https://travis-ci.com/github/apache/hudi/jobs/518067393]{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HUDI-2065) Fix flaky test: TestHoodieBackedMetadata#testOnlyValidPartitionsAdded
[
https://issues.apache.org/jira/browse/HUDI-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan reassigned HUDI-2065:
-
Assignee: Prashant Wason
> Fix flaky test: TestHoodieBackedMetadata#testOnlyValidPartitionsAdded
> -
>
> Key: HUDI-2065
> URL: https://issues.apache.org/jira/browse/HUDI-2065
> Project: Apache Hudi
> Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: sivabalan narayanan
>Assignee: Prashant Wason
>Priority: Major
>
> Fix flaky test: TestHoodieBackedMetadata#testOnlyValidPartitionsAdded
>
> {quote}
> [1] [https://travis-ci.com/github/apache/hudi/jobs/518067391]{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HUDI-2065) Fix flaky test: TestHoodieBackedMetadata#testOnlyValidPartitionsAdded
sivabalan narayanan created HUDI-2065:
-
Summary: Fix flaky test:
TestHoodieBackedMetadata#testOnlyValidPartitionsAdded
Key: HUDI-2065
URL: https://issues.apache.org/jira/browse/HUDI-2065
Project: Apache Hudi
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: sivabalan narayanan
Fix flaky test: TestHoodieBackedMetadata#testOnlyValidPartitionsAdded
{quote}
[1] [https://travis-ci.com/github/apache/hudi/jobs/518067391]{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] yuzhaojing commented on a change in pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
yuzhaojing commented on a change in pull request #3134:
URL: https://github.com/apache/hudi/pull/3134#discussion_r657583870
##
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/bootstrap/BootstrapFunction.java
##
@@ -197,10 +219,70 @@ private void loadRecords(String partitionPath,
Collector out) {
}
}
}
+ }
-long cost = System.currentTimeMillis() - start;
-LOG.info("Task [{}}:{}}] finish loading the index under partition {} and
sending them to downstream, time cost: {} milliseconds.",
-this.getClass().getSimpleName(), taskID, partitionPath, cost);
+ @SuppressWarnings("unchecked")
+ private void loadLogFiles(String partitionPath, Collector out) throws
Exception {
+Option latestCommitTime =
this.hoodieTable.getMetaClient().getCommitsTimeline()
+.filterCompletedInstants().lastInstant();
+if (!latestCommitTime.isPresent()) {
+ return;
+}
+
+List logFiles = this.hoodieTable.getSliceView()
+.getLatestFileSlicesBeforeOrOn(partitionPath,
latestCommitTime.get().getTimestamp(), true)
+.map(fileSlice -> fileSlice.getLogFiles().collect(toList()))
Review comment:
I think so, I will modify this logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d5606e0) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `29.61%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3137 +/- ## = - Coverage 46.13% 16.52% -29.62% + Complexity 5325 467 -4858 = Files 915 274 -641 Lines 3962210649-28973 Branches 4272 1088 -3184 = - Hits 18281 1760-16521 + Misses19470 8730-10740 + Partials 1871 159 -1712 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] danny0405 commented on a change in pull request #3134: [HUDI-2052] Support load logFile in BootstrapFunction
danny0405 commented on a change in pull request #3134:
URL: https://github.com/apache/hudi/pull/3134#discussion_r657581213
##
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/bootstrap/BootstrapFunction.java
##
@@ -197,10 +219,70 @@ private void loadRecords(String partitionPath,
Collector out) {
}
}
}
+ }
-long cost = System.currentTimeMillis() - start;
-LOG.info("Task [{}}:{}}] finish loading the index under partition {} and
sending them to downstream, time cost: {} milliseconds.",
-this.getClass().getSimpleName(), taskID, partitionPath, cost);
+ @SuppressWarnings("unchecked")
+ private void loadLogFiles(String partitionPath, Collector out) throws
Exception {
+Option latestCommitTime =
this.hoodieTable.getMetaClient().getCommitsTimeline()
+.filterCompletedInstants().lastInstant();
+if (!latestCommitTime.isPresent()) {
+ return;
+}
+
+List logFiles = this.hoodieTable.getSliceView()
+.getLatestFileSlicesBeforeOrOn(partitionPath,
latestCommitTime.get().getTimestamp(), true)
+.map(fileSlice -> fileSlice.getLogFiles().collect(toList()))
Review comment:
To simplify the logic, fetch the fileslice directly and send the parquet
and log record keys, the file group id is solid, which is recorded by the
fileslice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3133](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (263d5ee) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `29.61%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3133 +/- ## = - Coverage 46.13% 16.52% -29.62% + Complexity 5325 467 -4858 = Files 915 274 -641 Lines 3962210649-28973 Branches 4272 1088 -3184 = - Hits 18281 1760-16521 + Misses19470 8730-10740 + Partials 1871 159 -1712 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...main/java/org/apache/hudi/metrics/HoodieGauge.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvSG9vZGllR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/hive/NonPartitionedExtractor.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvTm9uUGFydGl0aW9uZWRFeHRyYWN0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../java/org/apache/hudi/metrics/MetricsReporter.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/metrics/MetricsReporterType.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL21ldHJpY3MvTWV0cmljc1JlcG9ydGVyVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...rg/apache/hudi/client/bootstrap/BootstrapMode.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9ib290c3RyYXAvQm9vdHN0cmFwTW9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3138](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0b32830) into [master](https://codecov.io/gh/apache/hudi/commit/2687eab8f0b49f410c49fac7771a8409722574cf?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2687eab) will **increase** coverage by `29.58%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3138 +/- ## = + Coverage 16.53% 46.12% +29.58% - Complexity 468 5323 +4855 = Files 274 915 +641 Lines 1064939617+28968 Branches 1088 4272 +3184 = + Hits 176118273+16512 - Misses 873019475+10745 - Partials158 1869 +1711 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (?)` | | | hudiclient | `30.45% <ø> (+30.45%)` | :arrow_up: | | hudicommon | `47.58% <ø> (?)` | | | hudiflink | `61.11% <100.00%> (?)` | | | hudihadoopmr | `51.29% <ø> (?)` | | | hudisparkdatasource | `67.05% <ø> (?)` | | | hudisync | `51.73% <ø> (+45.00%)` | :arrow_up: | | huditimelineservice | `64.36% <ø> (?)` | | | hudiutilities | `58.41% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...e/hudi/sink/partitioner/profile/WriteProfiles.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL3Byb2ZpbGUvV3JpdGVQcm9maWxlcy5qYXZh) | `50.00% <100.00%> (ø)` | | | [...pache/hudi/common/config/HoodieMetadataConfig.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2NvbmZpZy9Ib29kaWVNZXRhZGF0YUNvbmZpZy5qYXZh) | `0.00% <0.00%> (ø)` | | | [.../apache/hudi/common/model/HoodieRecordPayload.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL21vZGVsL0hvb2RpZVJlY29yZFBheWxvYWQuamF2YQ==) | `25.00% <0.00%> (ø)` | | | [...hudi/common/config/DFSPropertiesConfiguration.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2NvbmZpZy9ERlNQcm9wZXJ0aWVzQ29uZmlndXJhdGlvbi5qYXZh) | `78.04% <0.00%> (ø)` | | | [...di/hadoop/hive/HoodieCombineRealtimeHiveSplit.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL2hpdmUvSG9vZGllQ29tYmluZVJlYWx0aW1lSGl2ZVNwbGl0LmphdmE=) | `50.00% <0.00%> (ø)` | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3137](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d5606e0) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `43.00%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3137 +/- ## - Coverage 46.13% 3.13% -43.01% + Complexity 5325 82 -5243 Files 915 274 -641 Lines 39622 10649-28973 Branches 42721088 -3184 - Hits 18281 334-17947 + Misses19470 10289 -9181 + Partials 1871 26 -1845 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `9.52% <ø> (-48.86%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3137?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/3137/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * 0de8f0708cee42f6c5272762c4749b52493e76e3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=381) * d5606e08da3a088b43df6c224d4843996285aea2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=388) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * b705768fe421e0dcfb71834ce2045a8104d1dbf6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=380) * 263d5ee8e39e5c1e159311a56032b5ba5ce139de Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=387) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3133](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (263d5ee) into [master](https://codecov.io/gh/apache/hudi/commit/e039e0ff6d04b9b0684dbf29452af8a8dae94b09?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (e039e0f) will **decrease** coverage by `43.00%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #3133 +/- ## - Coverage 46.13% 3.13% -43.01% + Complexity 5325 82 -5243 Files 915 274 -641 Lines 39622 10649-28973 Branches 42721088 -3184 - Hits 18281 334-17947 + Misses19470 10289 -9181 + Partials 1871 26 -1845 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `?` | | | hudiclient | `0.00% <ø> (-30.46%)` | :arrow_down: | | hudicommon | `?` | | | hudiflink | `?` | | | hudihadoopmr | `?` | | | hudisparkdatasource | `?` | | | hudisync | `6.72% <ø> (-45.01%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `9.52% <ø> (-48.86%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3133?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/3133/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * b705768fe421e0dcfb71834ce2045a8104d1dbf6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=380) * 263d5ee8e39e5c1e159311a56032b5ba5ce139de UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * 0de8f0708cee42f6c5272762c4749b52493e76e3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=381) * d5606e08da3a088b43df6c224d4843996285aea2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3138](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0b32830) into [master](https://codecov.io/gh/apache/hudi/commit/2687eab8f0b49f410c49fac7771a8409722574cf?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2687eab) will **increase** coverage by `27.60%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3138 +/- ## = + Coverage 16.53% 44.14% +27.60% - Complexity 468 4535 +4067 = Files 274 819 +545 Lines 1064936166+25517 Branches 1088 3920 +2832 = + Hits 176115966+14205 - Misses 873018468 +9738 - Partials158 1732 +1574 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (?)` | | | hudiclient | `16.45% <ø> (+16.45%)` | :arrow_up: | | hudicommon | `47.58% <ø> (?)` | | | hudiflink | `61.11% <100.00%> (?)` | | | hudihadoopmr | `51.29% <ø> (?)` | | | hudisparkdatasource | `67.05% <ø> (?)` | | | hudisync | `51.73% <ø> (+45.00%)` | :arrow_up: | | huditimelineservice | `64.36% <ø> (?)` | | | hudiutilities | `58.41% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...e/hudi/sink/partitioner/profile/WriteProfiles.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL3Byb2ZpbGUvV3JpdGVQcm9maWxlcy5qYXZh) | `50.00% <100.00%> (ø)` | | | [...apache/hudi/sink/event/BatchWriteSuccessEvent.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL2V2ZW50L0JhdGNoV3JpdGVTdWNjZXNzRXZlbnQuamF2YQ==) | `92.30% <0.00%> (ø)` | | | [...ache/hudi/index/state/FlinkInMemoryStateIndex.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGllbnQvaHVkaS1mbGluay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaW5kZXgvc3RhdGUvRmxpbmtJbk1lbW9yeVN0YXRlSW5kZXguamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...i/common/table/view/FileSystemViewStorageType.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL3ZpZXcvRmlsZVN5c3RlbVZpZXdTdG9yYWdlVHlwZS5qYXZh) | `100.00% <0.00%> (ø)` | | | [...rg/apache/hudi/hadoop/HoodieROTablePathFilter.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL0hvb2RpZVJPVGFibGVQYXRoRmlsdGVyLmphdmE=) | `65.00% <0.00%> (ø)` | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely
codecov-commenter edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#3138](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0b32830) into [master](https://codecov.io/gh/apache/hudi/commit/2687eab8f0b49f410c49fac7771a8409722574cf?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (2687eab) will **increase** coverage by `23.66%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#3138 +/- ## = + Coverage 16.53% 40.20% +23.66% - Complexity 468 4149 +3681 = Files 274 819 +545 Lines 1064936166+25517 Branches 1088 3920 +2832 = + Hits 176114539+12778 - Misses 873020027+11297 - Partials158 1600 +1442 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (?)` | | | hudiclient | `16.45% <ø> (+16.45%)` | :arrow_up: | | hudicommon | `47.58% <ø> (?)` | | | hudiflink | `61.11% <100.00%> (?)` | | | hudihadoopmr | `51.29% <ø> (?)` | | | hudisparkdatasource | `67.05% <ø> (?)` | | | hudisync | `51.73% <ø> (+45.00%)` | :arrow_up: | | huditimelineservice | `64.36% <ø> (?)` | | | hudiutilities | `9.52% <ø> (-48.89%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...e/hudi/sink/partitioner/profile/WriteProfiles.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL3BhcnRpdGlvbmVyL3Byb2ZpbGUvV3JpdGVQcm9maWxlcy5qYXZh) | `50.00% <100.00%> (ø)` | | | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/3138/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | |
[GitHub] [hudi] yanghua closed pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely
yanghua closed pull request #3138: URL: https://github.com/apache/hudi/pull/3138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-1860) Add INSERT_OVERWRITE support to DeltaStreamer
[ https://issues.apache.org/jira/browse/HUDI-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368540#comment-17368540 ] Samrat Deb commented on HUDI-1860: -- Hi [~codope], Greetings! I want to pick this subtask. If you are not working on this sub-task actively, please help me with the context. > Add INSERT_OVERWRITE support to DeltaStreamer > - > > Key: HUDI-1860 > URL: https://issues.apache.org/jira/browse/HUDI-1860 > Project: Apache Hudi > Issue Type: Sub-task >Reporter: Sagar Sumit >Assignee: Sagar Sumit >Priority: Major > Original Estimate: 72h > Remaining Estimate: 72h > > As discussed in [this > RFC|https://cwiki.apache.org/confluence/display/HUDI/RFC+-+14+%3A+JDBC+incremental+puller], > having full fetch mode use the inser_overwrite to write to sync would be > better as it can handle schema changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] nsivabalan commented on a change in pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp
nsivabalan commented on a change in pull request #2438:
URL: https://github.com/apache/hudi/pull/2438#discussion_r657546921
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java
##
@@ -461,7 +465,7 @@ public void refreshTimeline() throws IOException {
if (!hasErrors || cfg.commitOnErrors) {
HashMap checkpointCommitMetadata = new HashMap<>();
checkpointCommitMetadata.put(CHECKPOINT_KEY, checkpointStr);
- if (cfg.checkpoint != null) {
+ if (cfg.checkpoint != null &&
!"timestamp".equals(props.getString("hoodie.deltastreamer.source.kafka.checkpoint.type")))
{
Review comment:
Can you help me understand why we need this ? My understanding is that,
user will set cfg.checkpoint during first batch and set the checkpoint type (to
timestamp) as well. but even for any checkpoint types, we should add the
checkpoint_reset_key here at the end of 1st batch. Am I missing something. can
you please help me understand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] leesf merged pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded
leesf merged pull request #3141: URL: https://github.com/apache/hudi/pull/3141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[hudi] branch master updated (380518e -> e039e0f)
This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 380518e [HUDI-2038] Support rollback inflight compaction instances for CompactionPlanOperator (#3105) add e039e0f [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded (#3141) No new revisions were added by this update. Summary of changes: .../java/org/apache/hudi/metadata/TestHoodieBackedMetadata.java | 9 - 1 file changed, 8 insertions(+), 1 deletion(-)
[GitHub] [hudi] leesf commented on pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded
leesf commented on pull request #3141: URL: https://github.com/apache/hudi/pull/3141#issuecomment-867226004 @prashantwason we should also add normal rollback commit (using HoodieFailedWritesCleaningPolicy#EAGER) unit tests backed by hoodie metadata table to solve metadata table exception, just merging this PR to make master branch happy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2542: Add ability to provide multi-region (global) data consistency across HMS in different regions
codecov-commenter edited a comment on pull request #2542: URL: https://github.com/apache/hudi/pull/2542#issuecomment-849630457 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2542?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2542](https://codecov.io/gh/apache/hudi/pull/2542?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (77e1ba1) into [master](https://codecov.io/gh/apache/hudi/commit/f8d9242372e9c682ea2573dde8b308d5895357f9?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (f8d9242) will **increase** coverage by `0.09%`. > The diff coverage is `65.11%`. [](https://codecov.io/gh/apache/hudi/pull/2542?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2542 +/- ## + Coverage 46.00% 46.09% +0.09% - Complexity 5306 5361 +55 Files 911 920 +9 Lines 3948839957 +469 Branches 4255 4309 +54 + Hits 1816518418 +253 - Misses1946819659 +191 - Partials 1855 1880 +25 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (ø)` | | | hudiclient | `30.12% <ø> (-0.33%)` | :arrow_down: | | hudicommon | `47.58% <66.66%> (+0.02%)` | :arrow_up: | | hudiflink | `61.26% <ø> (+0.14%)` | :arrow_up: | | hudihadoopmr | `51.29% <ø> (ø)` | | | hudisparkdatasource | `67.05% <ø> (+0.52%)` | :arrow_up: | | hudisync | `54.05% <65.09%> (+2.31%)` | :arrow_up: | | huditimelineservice | `64.36% <ø> (ø)` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2542?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...a/org/apache/hudi/common/util/ReflectionUtils.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvUmVmbGVjdGlvblV0aWxzLmphdmE=) | `22.64% <0.00%> (ø)` | | | [.../java/org/apache/hudi/hadoop/InputPathHandler.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL0lucHV0UGF0aEhhbmRsZXIuamF2YQ==) | `100.00% <ø> (ø)` | | | [.../org/apache/hudi/hadoop/utils/HoodieHiveUtils.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL3V0aWxzL0hvb2RpZUhpdmVVdGlscy5qYXZh) | `88.09% <ø> (ø)` | | | [...ache/hudi/hadoop/utils/HoodieInputFormatUtils.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL3V0aWxzL0hvb2RpZUlucHV0Rm9ybWF0VXRpbHMuamF2YQ==) | `54.35% <ø> (ø)` | | | [...main/java/org/apache/hudi/hive/HiveSyncConfig.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNDb25maWcuamF2YQ==) | `97.82% <ø> (ø)` | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2542: Add ability to provide multi-region (global) data consistency across HMS in different regions
codecov-commenter edited a comment on pull request #2542: URL: https://github.com/apache/hudi/pull/2542#issuecomment-849630457 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2542?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2542](https://codecov.io/gh/apache/hudi/pull/2542?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (77e1ba1) into [master](https://codecov.io/gh/apache/hudi/commit/f8d9242372e9c682ea2573dde8b308d5895357f9?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (f8d9242) will **decrease** coverage by `1.48%`. > The diff coverage is `0.93%`. [](https://codecov.io/gh/apache/hudi/pull/2542?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2542 +/- ## - Coverage 46.00% 44.51% -1.49% + Complexity 5306 5116 -190 Files 911 910 -1 Lines 3948839129 -359 Branches 4255 4256 +1 - Hits 1816517418 -747 - Misses1946819924 +456 + Partials 1855 1787 -68 ``` | Flag | Coverage Δ | | |---|---|---| | hudicli | `39.95% <ø> (ø)` | | | hudiclient | `30.12% <ø> (-0.33%)` | :arrow_down: | | hudicommon | `47.58% <66.66%> (+0.02%)` | :arrow_up: | | hudiflink | `61.26% <ø> (+0.14%)` | :arrow_up: | | hudihadoopmr | `51.29% <ø> (ø)` | | | hudisparkdatasource | `67.05% <ø> (+0.52%)` | :arrow_up: | | hudisync | `5.38% <0.00%> (-46.35%)` | :arrow_down: | | huditimelineservice | `?` | | | hudiutilities | `58.37% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2542?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...a/org/apache/hudi/common/util/ReflectionUtils.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvUmVmbGVjdGlvblV0aWxzLmphdmE=) | `22.64% <0.00%> (ø)` | | | [.../java/org/apache/hudi/hadoop/InputPathHandler.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL0lucHV0UGF0aEhhbmRsZXIuamF2YQ==) | `100.00% <ø> (ø)` | | | [.../org/apache/hudi/hadoop/utils/HoodieHiveUtils.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL3V0aWxzL0hvb2RpZUhpdmVVdGlscy5qYXZh) | `88.09% <ø> (ø)` | | | [...ache/hudi/hadoop/utils/HoodieInputFormatUtils.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1oYWRvb3AtbXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvaGFkb29wL3V0aWxzL0hvb2RpZUlucHV0Rm9ybWF0VXRpbHMuamF2YQ==) | `54.35% <ø> (ø)` | | | [...main/java/org/apache/hudi/hive/HiveSyncConfig.java](https://codecov.io/gh/apache/hudi/pull/2542/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNDb25maWcuamF2YQ==) | `0.00% <ø> (-97.83%)` | :arrow_down: | |
[GitHub] [hudi] hudi-bot commented on pull request #2487: [WIP HUDI-53] Adding Record Level Index based on hoodie backed table
hudi-bot commented on pull request #2487: URL: https://github.com/apache/hudi/pull/2487#issuecomment-867212403 ## CI report: * 8b07157d222a415db4f0d12fabb720cb4a37e28c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
