[GitHub] [hudi] pengzhiwei2018 edited a comment on pull request #2645: [HUDI-1659] Basic Implementation Of Spark Sql Support
pengzhiwei2018 edited a comment on pull request #2645: URL: https://github.com/apache/hudi/pull/2645#issuecomment-835134388 Hi @vinothchandar @umehrot2 , The PR has updated, mainly with the follow changes: - Support atomic for CTAS - Support use timestamp type as partition field - Fix exception when partition column is not in the rightest of select list for CTAS. - Add `TestSqlStatement` which does a sequence of statements. see - Add more test case for CTAS & partitioned table. - Change the `SparkSqlAdpater` to `SparkAdapter` For other issues your have mentioned above, I have filed a JIRA for each. - Support Truncate Command For Hoodie [1883](https://issues.apache.org/jira/browse/HUDI-1883) - Support Partial Update For MergeInto [1884](https://issues.apache.org/jira/browse/HUDI-1884) - Support Delete/Update Non-pk table [1885](https://issues.apache.org/jira/browse/HUDI-1885) After this first pr has merged, we can continue to solve these JIRAs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #2893: [HUDI-1371] Support metadata based listing for Spark DataSource and Spark SQL
pengzhiwei2018 commented on a change in pull request #2893:
URL: https://github.com/apache/hudi/pull/2893#discussion_r629013054
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java
##
@@ -141,6 +143,31 @@ protected BaseTableMetadata(HoodieEngineContext
engineContext, HoodieMetadataCon
.getAllFilesInPartition(partitionPath);
}
+ @Override
+ public Map getAllFilesInPartitions(List
partitionPaths)
+ throws IOException {
+if (enabled) {
+ Map partitionsFilesMap = new HashMap<>();
+
+ try {
+for (String partitionPath : partitionPaths) {
+ partitionsFilesMap.put(partitionPath, fetchAllFilesInPartition(new
Path(partitionPath)));
+}
+ } catch (Exception e) {
+if (metadataConfig.enableFallback()) {
+ LOG.error("Failed to retrieve files in partitions from metadata", e);
Review comment:
If enable the fallback here, an empty `partitionsFilesMap` will return
if there is an Exception happened, is it right?
##
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java
##
@@ -105,6 +106,20 @@ public FileSystemBackedTableMetadata(HoodieEngineContext
engineContext, Serializ
return partitionPaths;
}
+ @Override
+ public Map getAllFilesInPartitions(List
partitionPaths)
+ throws IOException {
+int parallelism = Math.min(DEFAULT_LISTING_PARALLELISM,
partitionPaths.size());
Review comment:
If the `partitionPaths` is empty, the `parallelism` will be 0, there may
be an Exception ("Positive number of partitions required") throw out for the
`sparkContext.parallelize(seq, parallelism)`,
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieFileIndex.scala
##
@@ -227,7 +227,17 @@ case class HoodieFileIndex(
*/
private def loadPartitionPathFiles(): Map[PartitionRowPath,
Array[FileStatus]] = {
val sparkEngine = new HoodieSparkEngineContext(new
JavaSparkContext(spark.sparkContext))
+val serializableConf = new
SerializableConfiguration(spark.sessionState.newHadoopConf())
+
val properties = new Properties()
+// To support metadata listing via Spark SQL we allow users to pass the
config via Hadoop Conf. Spark SQL does not
Review comment:
Should we get these `configurations` from the `spark.sessionState.conf`
for spark?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Closed] (HUDI-1886) Avoid to generates corrupted files for flink sink
[ https://issues.apache.org/jira/browse/HUDI-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-1886. -- Resolution: Done c1b331bcffaffde7661b75a9b846b117e9df63dc > Avoid to generates corrupted files for flink sink > - > > Key: HUDI-1886 > URL: https://issues.apache.org/jira/browse/HUDI-1886 > Project: Apache Hudi > Issue Type: Improvement > Components: Flink Integration >Reporter: Danny Chen >Assignee: Danny Chen >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (HUDI-1886) Avoid to generates corrupted files for flink sink
[ https://issues.apache.org/jira/browse/HUDI-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang updated HUDI-1886: --- Comment: was deleted (was: c1b331bcffaffde7661b75a9b846b117e9df63dc) > Avoid to generates corrupted files for flink sink > - > > Key: HUDI-1886 > URL: https://issues.apache.org/jira/browse/HUDI-1886 > Project: Apache Hudi > Issue Type: Improvement > Components: Flink Integration >Reporter: Danny Chen >Assignee: Danny Chen >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HUDI-1886) Avoid to generates corrupted files for flink sink
[ https://issues.apache.org/jira/browse/HUDI-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reopened HUDI-1886: > Avoid to generates corrupted files for flink sink > - > > Key: HUDI-1886 > URL: https://issues.apache.org/jira/browse/HUDI-1886 > Project: Apache Hudi > Issue Type: Improvement > Components: Flink Integration >Reporter: Danny Chen >Assignee: Danny Chen >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (HUDI-1886) Avoid to generates corrupted files for flink sink
[ https://issues.apache.org/jira/browse/HUDI-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-1886. -- Resolution: Fixed c1b331bcffaffde7661b75a9b846b117e9df63dc > Avoid to generates corrupted files for flink sink > - > > Key: HUDI-1886 > URL: https://issues.apache.org/jira/browse/HUDI-1886 > Project: Apache Hudi > Issue Type: Improvement > Components: Flink Integration >Reporter: Danny Chen >Assignee: Danny Chen >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hudi] yanghua merged pull request #2929: [HUDI-1886] Avoid to generates corrupted files for flink sink
yanghua merged pull request #2929: URL: https://github.com/apache/hudi/pull/2929 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[hudi] branch master updated (bfbf993 -> c1b331b)
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from bfbf993 [HUDI-1878] Add max memory option for flink writer task (#2920) add c1b331b [HUDI-1886] Avoid to generates corrupted files for flink sink (#2929) No new revisions were added by this update. Summary of changes: .../java/org/apache/hudi/io/FlinkCreateHandle.java | 4 ++- .../org/apache/hudi/sink/StreamWriteFunction.java | 35 ++ .../java/org/apache/hudi/util/StreamerUtil.java| 17 +++ .../sink/utils/StreamWriteFunctionWrapper.java | 1 - 4 files changed, 49 insertions(+), 8 deletions(-)
[jira] [Updated] (HUDI-1420) HoodieTableMetaClient.getMarkerFolderPath works incorrectly on windows client with hdfs server for wrong file seperator
[
https://issues.apache.org/jira/browse/HUDI-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 updated HUDI-1420:
-
Fix Version/s: (was: 0.8.0)
0.9.0
> HoodieTableMetaClient.getMarkerFolderPath works incorrectly on windows client
> with hdfs server for wrong file seperator
> ---
>
> Key: HUDI-1420
> URL: https://issues.apache.org/jira/browse/HUDI-1420
> Project: Apache Hudi
> Issue Type: Bug
> Components: Common Core
>Reporter: Sean Zhang
>Assignee: 谢波
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> The method assembly market dir with {{File.separator}}, here {{File}}
> references {{java.io.File}}. But when a hudi app run on a windows system (for
> example, spark local app on windows jdk), the constant is "\", but if the
> hudi directory locates on HDFS, the seperator should be "/".
> It cause the market is generated under directories like
> {{.hdfs://localhost:29820/HUDO_BASE/HUDI_TABLE_DIR\\.hoodie\\.temp\\20201127161504}},
> here {{HUDI_TABLE_DIR\\.hoodie\\.temp\\20201127161504}} is a whole directory
> name. And hudi app report some out market directory exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HUDI-1420) HoodieTableMetaClient.getMarkerFolderPath works incorrectly on windows client with hdfs server for wrong file seperator
[
https://issues.apache.org/jira/browse/HUDI-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 reopened HUDI-1420:
--
Assignee: 谢波
> HoodieTableMetaClient.getMarkerFolderPath works incorrectly on windows client
> with hdfs server for wrong file seperator
> ---
>
> Key: HUDI-1420
> URL: https://issues.apache.org/jira/browse/HUDI-1420
> Project: Apache Hudi
> Issue Type: Bug
> Components: Common Core
>Reporter: Sean Zhang
>Assignee: 谢波
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.8.0
>
>
> The method assembly market dir with {{File.separator}}, here {{File}}
> references {{java.io.File}}. But when a hudi app run on a windows system (for
> example, spark local app on windows jdk), the constant is "\", but if the
> hudi directory locates on HDFS, the seperator should be "/".
> It cause the market is generated under directories like
> {{.hdfs://localhost:29820/HUDO_BASE/HUDI_TABLE_DIR\\.hoodie\\.temp\\20201127161504}},
> here {{HUDI_TABLE_DIR\\.hoodie\\.temp\\20201127161504}} is a whole directory
> name. And hudi app report some out market directory exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hudi] danny0405 commented on a change in pull request #2929: [HUDI-1886] Avoid to generates corrupted files for flink sink
danny0405 commented on a change in pull request #2929: URL: https://github.com/apache/hudi/pull/2929#discussion_r629011753 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteFunction.java ## @@ -142,6 +142,22 @@ */ private transient TotalSizeTracer tracer; + /** + * Flag saying whether the write task is waiting for the checkpoint success notification + * after it finished a checkpoint. + * + * The flag is needed because the write task does not block during the waiting time interval, + * some data buckets still flush out with old instant time. There are two cases that the flush may produce + * corrupted files if the old instant is committed successfully: + * 1) the write handle was writing data but interrupted, left a corrupted parquet file; + * 2) the write handle finished the write but was not closed, left an empty parquet file. + * + * To solve, when this flag was set to true, we flush the data buffer with a new instant time = old instant time + 1ms, + * the new instant time would affect the write file name. The filesystem view does not recognize the file as committed because + * it always filters the data files based on successful commit time. Review comment: It is not invalid, if the write success and there is no retry, the write data is still valid and the reader can see the data, for current instant. If there is a retry for the cases i list here, the data would be seen as invalid and thus does not cause confuse to the filesystem view, we then can go on with the write. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] garyli1019 commented on a change in pull request #2930: [HUDI-1818] Validate and check required option for HoodieTable (Azure…
garyli1019 commented on a change in pull request #2930:
URL: https://github.com/apache/hudi/pull/2930#discussion_r629008470
##
File path:
hudi-flink/src/test/java/org/apache/hudi/table/TestHoodieTableFactory.java
##
@@ -297,4 +343,4 @@ public boolean isTemporary() {
return false;
}
}
-}
+}
Review comment:
nit: new line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] garyli1019 commented on a change in pull request #2930: [HUDI-1818] Validate and check required option for HoodieTable (Azure…
garyli1019 commented on a change in pull request #2930:
URL: https://github.com/apache/hudi/pull/2930#discussion_r629008388
##
File path:
hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java
##
@@ -98,6 +102,33 @@ public String factoryIdentifier() {
// Utilities
// -
+ /** Validate required options. e.g record key and pre combine key.
+ *
+ * @param conf The table options
+ * @param schema The table schema
+ */
+ private void validateRequiredOptions(Configuration conf, TableSchema schema)
{
Review comment:
sounds like `validateRequiredFields`? should we handle the partition
field as well if it's a partitioned table?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] garyli1019 commented on a change in pull request #2931: [MINOR] optimize FilePathUtils
garyli1019 commented on a change in pull request #2931:
URL: https://github.com/apache/hudi/pull/2931#discussion_r629007133
##
File path:
hudi-flink/src/main/java/org/apache/hudi/table/format/FilePathUtils.java
##
@@ -260,12 +245,12 @@ private static void listStatusRecursively(
int level,
int expectLevel,
List results) throws IOException {
-if (expectLevel == level && !isHiddenFile(fileStatus)) {
+if (expectLevel == level && isHiddenFile(fileStatus)) {
Review comment:
why should we load hidden files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] garyli1019 commented on a change in pull request #2929: [HUDI-1886] Avoid to generates corrupted files for flink sink
garyli1019 commented on a change in pull request #2929: URL: https://github.com/apache/hudi/pull/2929#discussion_r629006340 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteFunction.java ## @@ -142,6 +142,22 @@ */ private transient TotalSizeTracer tracer; + /** + * Flag saying whether the write task is waiting for the checkpoint success notification + * after it finished a checkpoint. + * + * The flag is needed because the write task does not block during the waiting time interval, + * some data buckets still flush out with old instant time. There are two cases that the flush may produce + * corrupted files if the old instant is committed successfully: + * 1) the write handle was writing data but interrupted, left a corrupted parquet file; + * 2) the write handle finished the write but was not closed, left an empty parquet file. + * + * To solve, when this flag was set to true, we flush the data buffer with a new instant time = old instant time + 1ms, + * the new instant time would affect the write file name. The filesystem view does not recognize the file as committed because + * it always filters the data files based on successful commit time. Review comment: Do you mean we write the corrupted file with an invalid instant time(+1ms) then retry? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #2906: [WIP] [HUDI-393] remove travis (Azure CI)
hudi-bot edited a comment on pull request #2906: URL: https://github.com/apache/hudi/pull/2906#issuecomment-830929050 ## CI report: * e96f2c03ab0f4fd6deb6803479fa6624eb21ed73 UNKNOWN * 0985b9b4a64b8015257eae8d85dfd899acf7a910 UNKNOWN * 7d6525d83a0fb964feb121e44fc617342778aff5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis` re-run the last Travis build - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2931: [MINOR] optimize FilePathUtils
codecov-commenter edited a comment on pull request #2931: URL: https://github.com/apache/hudi/pull/2931#issuecomment-835768056 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2931](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (7c84ec1) into [master](https://codecov.io/gh/apache/hudi/commit/bfbf993cbe3f1e3fab93095f4342ed17423efab5?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bfbf993) will **increase** coverage by `14.80%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2931 +/- ## = + Coverage 54.77% 69.58% +14.80% + Complexity 3796 375 -3421 = Files 481 54 -427 Lines 23284 2002-21282 Branches 2478 237 -2241 = - Hits 12753 1393-11360 + Misses 9385 478 -8907 + Partials 1146 131 -1015 ``` | Flag | Coverage Δ | Complexity Δ | | |---|---|---|---| | hudicli | `?` | `?` | | | hudiclient | `∅ <ø> (∅)` | `0.00 <ø> (ø)` | | | hudicommon | `?` | `?` | | | hudiflink | `?` | `?` | | | hudihadoopmr | `?` | `?` | | | hudisparkdatasource | `?` | `?` | | | hudisync | `?` | `?` | | | huditimelineservice | `?` | `?` | | | hudiutilities | `69.58% <ø> (ø)` | `375.00 <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...rg/apache/hudi/common/model/HoodieAvroPayload.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL21vZGVsL0hvb2RpZUF2cm9QYXlsb2FkLmphdmE=) | | | | | [...rc/main/java/org/apache/hudi/ApiMaturityLevel.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvQXBpTWF0dXJpdHlMZXZlbC5qYXZh) | | | | | [.../apache/hudi/common/bootstrap/FileStatusUtils.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2Jvb3RzdHJhcC9GaWxlU3RhdHVzVXRpbHMuamF2YQ==) | | | | | [...e/hudi/metadata/FileSystemBackedTableMetadata.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0YWRhdGEvRmlsZVN5c3RlbUJhY2tlZFRhYmxlTWV0YWRhdGEuamF2YQ==) | | | | | [...apache/hudi/common/util/collection/ArrayUtils.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvY29sbGVjdGlvbi9BcnJheVV0aWxzLmphdmE=) | | | | | [...ache/hudi/hive/HiveMetastoreBasedLockProvider.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZU1ldGFzdG9yZUJhc2VkTG9ja1Byb3ZpZGVyLmphdmE=) | | | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2931: [MINOR] optimize FilePathUtils
codecov-commenter edited a comment on pull request #2931: URL: https://github.com/apache/hudi/pull/2931#issuecomment-835768056 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2931](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (7c84ec1) into [master](https://codecov.io/gh/apache/hudi/commit/bfbf993cbe3f1e3fab93095f4342ed17423efab5?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bfbf993) will **increase** coverage by `14.80%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2931 +/- ## = + Coverage 54.77% 69.58% +14.80% + Complexity 3796 375 -3421 = Files 481 54 -427 Lines 23284 2002-21282 Branches 2478 237 -2241 = - Hits 12753 1393-11360 + Misses 9385 478 -8907 + Partials 1146 131 -1015 ``` | Flag | Coverage Δ | Complexity Δ | | |---|---|---|---| | hudicli | `?` | `?` | | | hudiclient | `?` | `?` | | | hudicommon | `?` | `?` | | | hudiflink | `?` | `?` | | | hudihadoopmr | `?` | `?` | | | hudisparkdatasource | `?` | `?` | | | hudisync | `?` | `?` | | | huditimelineservice | `?` | `?` | | | hudiutilities | `69.58% <ø> (ø)` | `375.00 <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...e/hudi/common/table/log/HoodieLogFormatReader.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL2xvZy9Ib29kaWVMb2dGb3JtYXRSZWFkZXIuamF2YQ==) | | | | | [...he/hudi/common/table/timeline/dto/FilePathDTO.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL3RpbWVsaW5lL2R0by9GaWxlUGF0aERUTy5qYXZh) | | | | | [...i/common/util/collection/ExternalSpillableMap.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvY29sbGVjdGlvbi9FeHRlcm5hbFNwaWxsYWJsZU1hcC5qYXZh) | | | | | [...apache/hudi/cli/HoodieHistoryFileNameProvider.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpL0hvb2RpZUhpc3RvcnlGaWxlTmFtZVByb3ZpZGVyLmphdmE=) | | | | | [...apache/hudi/cli/commands/HoodieLogFileCommand.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jbGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpL2NvbW1hbmRzL0hvb2RpZUxvZ0ZpbGVDb21tYW5kLmphdmE=) | | | | | [...ava/org/apache/hudi/payload/AWSDmsAvroPayload.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvcGF5bG9hZC9BV1NEbXNBdnJvUGF5bG9hZC5qYXZh) | | | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2931: [MINOR] optimize FilePathUtils
codecov-commenter edited a comment on pull request #2931: URL: https://github.com/apache/hudi/pull/2931#issuecomment-835768056 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2931](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (7c84ec1) into [master](https://codecov.io/gh/apache/hudi/commit/bfbf993cbe3f1e3fab93095f4342ed17423efab5?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bfbf993) will **decrease** coverage by `45.43%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #2931 +/- ## - Coverage 54.77% 9.34% -45.44% + Complexity 3796 48 -3748 Files 481 54 -427 Lines 232842002-21282 Branches 2478 237 -2241 - Hits 12753 187-12566 + Misses 93851802 -7583 + Partials 1146 13 -1133 ``` | Flag | Coverage Δ | Complexity Δ | | |---|---|---|---| | hudicli | `?` | `?` | | | hudiclient | `?` | `?` | | | hudicommon | `?` | `?` | | | hudiflink | `?` | `?` | | | hudihadoopmr | `?` | `?` | | | hudisparkdatasource | `?` | `?` | | | hudisync | `?` | `?` | | | huditimelineservice | `?` | `?` | | | hudiutilities | `9.34% <ø> (-60.24%)` | `48.00 <ø> (-327.00)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-2.00%)` | | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-3.00%)` | | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-4.00%)` | | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2929: [HUDI-1886] Avoid to generates corrupted files for flink sink
codecov-commenter edited a comment on pull request #2929: URL: https://github.com/apache/hudi/pull/2929#issuecomment-835640212 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2929](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5cdff47) into [master](https://codecov.io/gh/apache/hudi/commit/bfbf993cbe3f1e3fab93095f4342ed17423efab5?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bfbf993) will **decrease** coverage by `0.00%`. > The diff coverage is `46.15%`. [](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2929 +/- ## - Coverage 54.77% 54.76% -0.01% - Complexity 3796 3798 +2 Files 481 481 Lines 2328423291 +7 Branches 2478 2480 +2 + Hits 1275312756 +3 - Misses 9385 9386 +1 - Partials 1146 1149 +3 ``` | Flag | Coverage Δ | Complexity Δ | | |---|---|---|---| | hudicli | `39.53% <ø> (ø)` | `220.00 <ø> (ø)` | | | hudiclient | `∅ <ø> (∅)` | `0.00 <ø> (ø)` | | | hudicommon | `50.41% <ø> (+0.02%)` | `1976.00 <ø> (+1.00)` | | | hudiflink | `62.88% <46.15%> (-0.13%)` | `525.00 <4.00> (+1.00)` | :arrow_down: | | hudihadoopmr | `50.93% <ø> (ø)` | `259.00 <ø> (ø)` | | | hudisparkdatasource | `73.33% <ø> (ø)` | `237.00 <ø> (ø)` | | | hudisync | `46.52% <ø> (ø)` | `144.00 <ø> (ø)` | | | huditimelineservice | `64.36% <ø> (ø)` | `62.00 <ø> (ø)` | | | hudiutilities | `69.58% <ø> (ø)` | `375.00 <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `53.65% <0.00%> (-1.35%)` | `17.00 <1.00> (+1.00)` | :arrow_down: | | [...java/org/apache/hudi/sink/StreamWriteFunction.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL1N0cmVhbVdyaXRlRnVuY3Rpb24uamF2YQ==) | `78.98% <60.00%> (-2.07%)` | `23.00 <3.00> (ø)` | | | [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `89.65% <0.00%> (+10.34%)` | `16.00% <0.00%> (+1.00%)` | | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter commented on pull request #2931: [MINOR] optimize FilePathUtils
codecov-commenter commented on pull request #2931: URL: https://github.com/apache/hudi/pull/2931#issuecomment-835768056 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2931](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (a76d931) into [master](https://codecov.io/gh/apache/hudi/commit/bfbf993cbe3f1e3fab93095f4342ed17423efab5?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bfbf993) will **decrease** coverage by `45.43%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #2931 +/- ## - Coverage 54.77% 9.34% -45.44% + Complexity 3796 48 -3748 Files 481 54 -427 Lines 232842002-21282 Branches 2478 237 -2241 - Hits 12753 187-12566 + Misses 93851802 -7583 + Partials 1146 13 -1133 ``` | Flag | Coverage Δ | Complexity Δ | | |---|---|---|---| | hudicli | `?` | `?` | | | hudiclient | `?` | `?` | | | hudicommon | `?` | `?` | | | hudiflink | `?` | `?` | | | hudihadoopmr | `?` | `?` | | | hudisparkdatasource | `?` | `?` | | | hudisync | `?` | `?` | | | huditimelineservice | `?` | `?` | | | hudiutilities | `9.34% <ø> (-60.24%)` | `48.00 <ø> (-327.00)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2931?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...va/org/apache/hudi/utilities/IdentitySplitter.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0lkZW50aXR5U3BsaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-2.00%)` | | | [...va/org/apache/hudi/utilities/schema/SchemaSet.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NjaGVtYS9TY2hlbWFTZXQuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-3.00%)` | | | [...a/org/apache/hudi/utilities/sources/RowSource.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvUm93U291cmNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-4.00%)` | | | [.../org/apache/hudi/utilities/sources/AvroSource.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvQXZyb1NvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | | | [.../org/apache/hudi/utilities/sources/JsonSource.java](https://codecov.io/gh/apache/hudi/pull/2931/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3NvdXJjZXMvSnNvblNvdXJjZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | | |
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2929: [HUDI-1886] Avoid to generates corrupted files for flink sink
codecov-commenter edited a comment on pull request #2929: URL: https://github.com/apache/hudi/pull/2929#issuecomment-835640212 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2929](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5cdff47) into [master](https://codecov.io/gh/apache/hudi/commit/bfbf993cbe3f1e3fab93095f4342ed17423efab5?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bfbf993) will **decrease** coverage by `0.00%`. > The diff coverage is `46.15%`. [](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2929 +/- ## - Coverage 54.77% 54.76% -0.01% - Complexity 3796 3798 +2 Files 481 481 Lines 2328423291 +7 Branches 2478 2480 +2 + Hits 1275312756 +3 - Misses 9385 9386 +1 - Partials 1146 1149 +3 ``` | Flag | Coverage Δ | Complexity Δ | | |---|---|---|---| | hudicli | `39.53% <ø> (ø)` | `220.00 <ø> (ø)` | | | hudiclient | `∅ <ø> (∅)` | `0.00 <ø> (ø)` | | | hudicommon | `50.41% <ø> (+0.02%)` | `1976.00 <ø> (+1.00)` | | | hudiflink | `62.88% <46.15%> (-0.13%)` | `525.00 <4.00> (+1.00)` | :arrow_down: | | hudihadoopmr | `50.93% <ø> (ø)` | `259.00 <ø> (ø)` | | | hudisparkdatasource | `73.33% <ø> (ø)` | `237.00 <ø> (ø)` | | | hudisync | `46.52% <ø> (ø)` | `144.00 <ø> (ø)` | | | huditimelineservice | `64.36% <ø> (ø)` | `62.00 <ø> (ø)` | | | hudiutilities | `69.58% <ø> (ø)` | `375.00 <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `53.65% <0.00%> (-1.35%)` | `17.00 <1.00> (+1.00)` | :arrow_down: | | [...java/org/apache/hudi/sink/StreamWriteFunction.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL1N0cmVhbVdyaXRlRnVuY3Rpb24uamF2YQ==) | `78.98% <60.00%> (-2.07%)` | `23.00 <3.00> (ø)` | | | [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `89.65% <0.00%> (+10.34%)` | `16.00% <0.00%> (+1.00%)` | | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] MyLanPangzi opened a new pull request #2931: [MINOR] optimize FilePathUtils
MyLanPangzi opened a new pull request #2931: URL: https://github.com/apache/hudi/pull/2931 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull request remove unused method optimize code logic ## Brief change log *(for example:)* - *Modify AnnotationLocation checkstyle rule in checkstyle.xml* ## Verify this pull request *(Please pick either of the following options)* This pull request is a trivial rework / code cleanup without any test coverage. *(or)* This pull request is already covered by existing tests, such as *(please describe tests)*. (or) This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end.* - *Added HoodieClientWriteTest to verify the change.* - *Manually verified the change by running a job locally.* ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc changes done or have another open PR - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2929: [HUDI-1886] Avoid to generates corrupted files for flink sink
codecov-commenter edited a comment on pull request #2929: URL: https://github.com/apache/hudi/pull/2929#issuecomment-835640212 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2929](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5cdff47) into [master](https://codecov.io/gh/apache/hudi/commit/bfbf993cbe3f1e3fab93095f4342ed17423efab5?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bfbf993) will **decrease** coverage by `0.01%`. > The diff coverage is `46.15%`. [](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2929 +/- ## - Coverage 54.77% 54.75% -0.02% - Complexity 3796 3797 +1 Files 481 481 Lines 2328423291 +7 Branches 2478 2480 +2 Hits 1275312753 - Misses 9385 9389 +4 - Partials 1146 1149 +3 ``` | Flag | Coverage Δ | Complexity Δ | | |---|---|---|---| | hudicli | `39.53% <ø> (ø)` | `220.00 <ø> (ø)` | | | hudiclient | `∅ <ø> (∅)` | `0.00 <ø> (ø)` | | | hudicommon | `50.38% <ø> (ø)` | `1975.00 <ø> (ø)` | | | hudiflink | `62.88% <46.15%> (-0.13%)` | `525.00 <4.00> (+1.00)` | :arrow_down: | | hudihadoopmr | `50.93% <ø> (ø)` | `259.00 <ø> (ø)` | | | hudisparkdatasource | `73.33% <ø> (ø)` | `237.00 <ø> (ø)` | | | hudisync | `46.52% <ø> (ø)` | `144.00 <ø> (ø)` | | | huditimelineservice | `64.36% <ø> (ø)` | `62.00 <ø> (ø)` | | | hudiutilities | `69.58% <ø> (ø)` | `375.00 <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `53.65% <0.00%> (-1.35%)` | `17.00 <1.00> (+1.00)` | :arrow_down: | | [...java/org/apache/hudi/sink/StreamWriteFunction.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL1N0cmVhbVdyaXRlRnVuY3Rpb24uamF2YQ==) | `78.98% <60.00%> (-2.07%)` | `23.00 <3.00> (ø)` | | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] codecov-commenter edited a comment on pull request #2929: [HUDI-1886] Avoid to generates corrupted files for flink sink
codecov-commenter edited a comment on pull request #2929: URL: https://github.com/apache/hudi/pull/2929#issuecomment-835640212 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#2929](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (5cdff47) into [master](https://codecov.io/gh/apache/hudi/commit/bfbf993cbe3f1e3fab93095f4342ed17423efab5?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (bfbf993) will **decrease** coverage by `0.01%`. > The diff coverage is `46.15%`. [](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#2929 +/- ## - Coverage 54.77% 54.75% -0.02% - Complexity 3796 3797 +1 Files 481 481 Lines 2328423291 +7 Branches 2478 2480 +2 Hits 1275312753 - Misses 9385 9389 +4 - Partials 1146 1149 +3 ``` | Flag | Coverage Δ | Complexity Δ | | |---|---|---|---| | hudicli | `39.53% <ø> (ø)` | `220.00 <ø> (ø)` | | | hudiclient | `∅ <ø> (∅)` | `0.00 <ø> (ø)` | | | hudicommon | `50.38% <ø> (ø)` | `1975.00 <ø> (ø)` | | | hudiflink | `62.88% <46.15%> (-0.13%)` | `525.00 <4.00> (+1.00)` | :arrow_down: | | hudihadoopmr | `50.93% <ø> (ø)` | `259.00 <ø> (ø)` | | | hudisparkdatasource | `73.33% <ø> (ø)` | `237.00 <ø> (ø)` | | | hudisync | `46.52% <ø> (ø)` | `144.00 <ø> (ø)` | | | huditimelineservice | `64.36% <ø> (ø)` | `62.00 <ø> (ø)` | | | hudiutilities | `69.58% <ø> (ø)` | `375.00 <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/apache/hudi/pull/2929?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS91dGlsL1N0cmVhbWVyVXRpbC5qYXZh) | `53.65% <0.00%> (-1.35%)` | `17.00 <1.00> (+1.00)` | :arrow_down: | | [...java/org/apache/hudi/sink/StreamWriteFunction.java](https://codecov.io/gh/apache/hudi/pull/2929/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-aHVkaS1mbGluay9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVkaS9zaW5rL1N0cmVhbVdyaXRlRnVuY3Rpb24uamF2YQ==) | `78.98% <60.00%> (-2.07%)` | `23.00 <3.00> (ø)` | | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 closed pull request #2929: [HUDI-1886] Avoid to generates corrupted files for flink sink
danny0405 closed pull request #2929: URL: https://github.com/apache/hudi/pull/2929 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
