[
https://issues.apache.org/jira/browse/HUDI-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176115#comment-17176115
]
Balaji Varadarajan commented on HUDI-1182:
------------------------------------------
[~shivnarayan] [~vinoth] : FYI, I am listing the issues I am finding here and
debugging. Out of many runs today, only integ test was flaky. So, just
iterating on that. I am seeing 2 patterns of failures which I have listed in
the description. This is the testing PR :
[https://github.com/apache/hudi/pull/1952]
> Stabilize CI
> -------------
>
> Key: HUDI-1182
> URL: https://issues.apache.org/jira/browse/HUDI-1182
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Testing
> Reporter: Balaji Varadarajan
> Assignee: Balaji Varadarajan
> Priority: Blocker
>
> Documenting all failure cases here :
>
> Integration Tests:
>
> 1. ITTestHoodieSanity : Command
> ([/var/hoodie/ws/hudi-spark/run_hoodie_streaming_app.sh, --hive-sync,
> --table-path, hdfs://namenode/docker_hoodie_single_partition_key_cow_test,
> --hive-url, jdbc:hive2://hiveserver:10000, --table-type, COPY_ON_WRITE,
> --hive-table, docker_hoodie_single_partition_key_cow_test])
> {code:java}
> 05:17:48.384 [main] ERROR HoodieJavaStreamingApp - Got error running app
> java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException:
> Expecting 100 records, Got 50 at
> java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_212] at
> java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_212] at
> HoodieJavaStreamingApp.run(HoodieJavaStreamingApp.java:193)
> ~[test-classes/:?] at
> HoodieJavaStreamingApp.main(HoodieJavaStreamingApp.java:126)
> [test-classes/:?] Caused by: java.lang.IllegalArgumentException: Expecting
> 100 records, Got 50 at
> org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:40)
> ~[hudi-spark-bundle_2.11-0.6.0-SNAPSHOT.jar:0.6.0-SNAPSHOT] at
> HoodieJavaStreamingApp.addInputAndValidateIngestion(HoodieJavaStreamingApp.java:352)
> ~[test-classes/:?] at
> HoodieJavaStreamingApp.lambda$run$1(HoodieJavaStreamingApp.java:186)
> ~[test-classes/:?] at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_212] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_212] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_212] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_212]
> {code}
>
> Comments: From the context, it looks like out of 100 records in 1st batch,
> only 50 records were updated but all 100 records were expected to be upddated.
>
> 2. ITTestHoodieSanity : Command
> ([/var/hoodie/ws/hudi-spark/run_hoodie_streaming_app.sh, --hive-sync, --t
> able-path, hdfs://namenode/docker_hoodie_single_partition_key_mor_test,
> --hive-url, jdbc:hive2://hiveserver:10000, --t
> able-type, MERGE_ON_READ, --hive-table,
> docker_hoodie_single_partition_key_mor_test]) expected to succeed. Exit (255)
> ==> expected: <0> but was: <255>
>
> {code:java}
> Instants :[[20200812012618__deltacommit__COMPLETED],
> [20200812012629__deltacommit__COMPLETED]]
> Instants :[[20200812012618__deltacommit__COMPLETED],
> [20200812012629__deltacommit__COMPLETED]]
> 01:29:35.754 [main] ERROR HoodieJavaStreamingApp - Got error running app
> java.util.concurrent.ExecutionException: java.lang.IllegalStateException:
> Timedout waiting for 3 commits to appear in
> hdfs://namenode/docker_hoodie_single_partition_key_mor_test
> at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_212]
> at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_212]
> at HoodieJavaStreamingApp.run(HoodieJavaStreamingApp.java:185)
> ~[test-classes/:?]
> at HoodieJavaStreamingApp.main(HoodieJavaStreamingApp.java:118)
> [test-classes/:?]
> Caused by: java.lang.IllegalStateException: Timedout waiting for 3 commits
> to appear in hdfs://namenode/docker_hoodie_
> single_partition_key_mor_test
> at HoodieJavaStreamingApp.waitTillNCommits(HoodieJavaStreamingApp.java:261)
> ~[test-classes/:?]
> at
> HoodieJavaStreamingApp.addInputAndValidateIngestion(HoodieJavaStreamingApp.java:298)
> ~[test-classes/:?]
> at HoodieJavaStreamingApp.lambda$run$1(HoodieJavaStreamingApp.java:178)
> ~[test-classes/:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_212]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_212]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_212]
> at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_212]{code}
>
> From the context: Compaction should have been scheduled and executed but that
> did not happen here.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
