Re: [HELP] unstable tests in the travis CI
Thanks Raymond. Will chat more on the JIRA. I am looking into some of these. I vote for stabilizing things before we land any involved PRs. On Sat, Jun 26, 2021 at 4:45 PM Raymond Xu wrote: > I did some backlog grooming; putting flaky tests by class in subtasks there > in > https://issues.apache.org/jira/browse/HUDI-1248 > > If you're working on any of those, please set the assignee. Let's > parallelize the efforts :) > > Also, the Azure CI builds for master/release versions can be found here > > https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build?definitionId=3&_a=summary > > Looks like some flakiness only happened in Travis CI. Let's also keep > observing how it goes in Azure. > > On Wed, Jun 23, 2021 at 12:48 PM Vinoth Chandar wrote: > > > yes. CI is pretty flaky atm. There is a compiled list here > > https://issues.apache.org/jira/browse/HUDI-1248 > > > > Siva and I are looking into some of this and try and get everything back > to > > normal again > > > > That schema evolution test, I have tried reproducing a few times, without > > luck. :/ > > > > On Wed, Jun 23, 2021 at 10:17 AM Prashant Wason > > > wrote: > > > > > Sure. I will take a look today. I wonder how the CI passed during the > > > merge. > > > > > > > > > On Wed, Jun 23, 2021 at 7:57 AM pzwpzw > .invalid> > > > wrote: > > > > > > > Hi @Prashant Wason, I found that after the [HUDI-1717]( commit hash: > > > > 11e64b2db0ddf8f816561f8442b373de15a26d71) has merged yesterday， the > > test > > > > case TestHoodieBackedMetadata#testOnlyValidPartitionsAdded will > always > > > > crash: > > > > > > > > org.apache.hudi.exception.HoodieMetadataException: Failed to retrieve > > > > files in partition > > > > > > > > > > /var/folders/my/841b2c052038ppns0csrf8g8gn/T/junit3095347769583879437/dataset/p1 > > > > from metadata > > > > > > > > at > > > > > > > > > > org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:129) > > > > at > > > > > > > > > > org.apache.hudi.metadata.TestHoodieBackedMetadata.testOnlyValidPartitionsAdded(TestHoodieBackedMetadata.java:210) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > at > > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > at > > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > at java.lang.reflect.Method.invoke(Method.java:498) > > > > > > > > Can you take a look at this, Thanks~ > > > > > > > > > > > > > > > > 2021年6月23日 下午1:49，Danny Chan 写道： > > > > > > > > Hi, fellows, there are two test cases in the travis CI that fails > very > > > > often, which blocks our coding too many times, please, if these tests > > are > > > > not stable, can we disable them first ? > > > > They are annoying ~ > > > > > > > > > > > > TestHoodieBackedMetadata.testOnlyValidPartitionsAdded[1] > > > > HoodieSparkSqlWriterSuite: schema evolution for ... [2] > > > > > > > > [1] https://travis-ci.com/github/apache/hudi/jobs/518067391 > > > > [2] https://travis-ci.com/github/apache/hudi/jobs/518067393 > > > > > > > > Best, > > > > Danny Chan > > > > > > > > > > > > > >
Re: [HELP] unstable tests in the travis CI
I did some backlog grooming; putting flaky tests by class in subtasks there in https://issues.apache.org/jira/browse/HUDI-1248 If you're working on any of those, please set the assignee. Let's parallelize the efforts :) Also, the Azure CI builds for master/release versions can be found here https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build?definitionId=3&_a=summary Looks like some flakiness only happened in Travis CI. Let's also keep observing how it goes in Azure. On Wed, Jun 23, 2021 at 12:48 PM Vinoth Chandar wrote: > yes. CI is pretty flaky atm. There is a compiled list here > https://issues.apache.org/jira/browse/HUDI-1248 > > Siva and I are looking into some of this and try and get everything back to > normal again > > That schema evolution test, I have tried reproducing a few times, without > luck. :/ > > On Wed, Jun 23, 2021 at 10:17 AM Prashant Wason > wrote: > > > Sure. I will take a look today. I wonder how the CI passed during the > > merge. > > > > > > On Wed, Jun 23, 2021 at 7:57 AM pzwpzw .invalid> > > wrote: > > > > > Hi @Prashant Wason, I found that after the [HUDI-1717]( commit hash: > > > 11e64b2db0ddf8f816561f8442b373de15a26d71) has merged yesterday， the > test > > > case TestHoodieBackedMetadata#testOnlyValidPartitionsAdded will always > > > crash: > > > > > > org.apache.hudi.exception.HoodieMetadataException: Failed to retrieve > > > files in partition > > > > > > /var/folders/my/841b2c052038ppns0csrf8g8gn/T/junit3095347769583879437/dataset/p1 > > > from metadata > > > > > > at > > > > > > org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:129) > > > at > > > > > > org.apache.hudi.metadata.TestHoodieBackedMetadata.testOnlyValidPartitionsAdded(TestHoodieBackedMetadata.java:210) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:498) > > > > > > Can you take a look at this, Thanks~ > > > > > > > > > > > > 2021年6月23日 下午1:49，Danny Chan 写道： > > > > > > Hi, fellows, there are two test cases in the travis CI that fails very > > > often, which blocks our coding too many times, please, if these tests > are > > > not stable, can we disable them first ? > > > They are annoying ~ > > > > > > > > > TestHoodieBackedMetadata.testOnlyValidPartitionsAdded[1] > > > HoodieSparkSqlWriterSuite: schema evolution for ... [2] > > > > > > [1] https://travis-ci.com/github/apache/hudi/jobs/518067391 > > > [2] https://travis-ci.com/github/apache/hudi/jobs/518067393 > > > > > > Best, > > > Danny Chan > > > > > > > > >
Re: [HELP] unstable tests in the travis CI
yes. CI is pretty flaky atm. There is a compiled list here https://issues.apache.org/jira/browse/HUDI-1248 Siva and I are looking into some of this and try and get everything back to normal again That schema evolution test, I have tried reproducing a few times, without luck. :/ On Wed, Jun 23, 2021 at 10:17 AM Prashant Wason wrote: > Sure. I will take a look today. I wonder how the CI passed during the > merge. > > > On Wed, Jun 23, 2021 at 7:57 AM pzwpzw > wrote: > > > Hi @Prashant Wason, I found that after the [HUDI-1717]( commit hash: > > 11e64b2db0ddf8f816561f8442b373de15a26d71) has merged yesterday， the test > > case TestHoodieBackedMetadata#testOnlyValidPartitionsAdded will always > > crash: > > > > org.apache.hudi.exception.HoodieMetadataException: Failed to retrieve > > files in partition > > > /var/folders/my/841b2c052038ppns0csrf8g8gn/T/junit3095347769583879437/dataset/p1 > > from metadata > > > > at > > > org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:129) > > at > > > org.apache.hudi.metadata.TestHoodieBackedMetadata.testOnlyValidPartitionsAdded(TestHoodieBackedMetadata.java:210) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > > > Can you take a look at this, Thanks~ > > > > > > > > 2021年6月23日 下午1:49，Danny Chan 写道： > > > > Hi, fellows, there are two test cases in the travis CI that fails very > > often, which blocks our coding too many times, please, if these tests are > > not stable, can we disable them first ? > > They are annoying ~ > > > > > > TestHoodieBackedMetadata.testOnlyValidPartitionsAdded[1] > > HoodieSparkSqlWriterSuite: schema evolution for ... [2] > > > > [1] https://travis-ci.com/github/apache/hudi/jobs/518067391 > > [2] https://travis-ci.com/github/apache/hudi/jobs/518067393 > > > > Best, > > Danny Chan > > > > >
Re: [HELP] unstable tests in the travis CI
Sure. I will take a look today. I wonder how the CI passed during the merge. On Wed, Jun 23, 2021 at 7:57 AM pzwpzw wrote: > Hi @Prashant Wason, I found that after the [HUDI-1717]( commit hash: > 11e64b2db0ddf8f816561f8442b373de15a26d71) has merged yesterday， the test > case TestHoodieBackedMetadata#testOnlyValidPartitionsAdded will always > crash: > > org.apache.hudi.exception.HoodieMetadataException: Failed to retrieve > files in partition > /var/folders/my/841b2c052038ppns0csrf8g8gn/T/junit3095347769583879437/dataset/p1 > from metadata > > at > org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:129) > at > org.apache.hudi.metadata.TestHoodieBackedMetadata.testOnlyValidPartitionsAdded(TestHoodieBackedMetadata.java:210) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > > Can you take a look at this, Thanks~ > > > > 2021年6月23日 下午1:49，Danny Chan 写道： > > Hi, fellows, there are two test cases in the travis CI that fails very > often, which blocks our coding too many times, please, if these tests are > not stable, can we disable them first ? > They are annoying ~ > > > TestHoodieBackedMetadata.testOnlyValidPartitionsAdded[1] > HoodieSparkSqlWriterSuite: schema evolution for ... [2] > > [1] https://travis-ci.com/github/apache/hudi/jobs/518067391 > [2] https://travis-ci.com/github/apache/hudi/jobs/518067393 > > Best, > Danny Chan > >
Re: [HELP] unstable tests in the travis CI
Hi @Prashant Wason, I found that after the [HUDI-1717]( commit hash: 11e64b2db0ddf8f816561f8442b373de15a26d71) has merged yesterday， the test case TestHoodieBackedMetadata#testOnlyValidPartitionsAdded will always crash: org.apache.hudi.exception.HoodieMetadataException: Failed to retrieve files in partition /var/folders/my/841b2c052038ppns0csrf8g8gn/T/junit3095347769583879437/dataset/p1 from metadata at org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:129) at org.apache.hudi.metadata.TestHoodieBackedMetadata.testOnlyValidPartitionsAdded(TestHoodieBackedMetadata.java:210) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) Can you take a look at this, Thanks~ 2021年6月23日 下午1:49，Danny Chan 写道： Hi, fellows, there are two test cases in the travis CI that fails very often, which blocks our coding too many times, please, if these tests are not stable, can we disable them first ? They are annoying ~ TestHoodieBackedMetadata.testOnlyValidPartitionsAdded[1] HoodieSparkSqlWriterSuite: schema evolution for ... [2] [1] https://travis-ci.com/github/apache/hudi/jobs/518067391 [2] https://travis-ci.com/github/apache/hudi/jobs/518067393 Best, Danny Chan
Re: [HELP] unstable tests in the travis CI
That's cool, thanks ~ Best, Danny Chan pzwpzw 于2021年6月23日周三 下午1:58写道： > Hi Danny, There is a bug in schema resolve in DefaultSource which lead > the test case 2 crash. I will submit a PR to solve this later. > > 2021年6月23日 下午1:49，Danny Chan 写道： > > Hi, fellows, there are two test cases in the travis CI that fails very > often, which blocks our coding too many times, please, if these tests are > not stable, can we disable them first ? > They are annoying ~ > > > TestHoodieBackedMetadata.testOnlyValidPartitionsAdded[1] > HoodieSparkSqlWriterSuite: schema evolution for ... [2] > > [1] https://travis-ci.com/github/apache/hudi/jobs/518067391 > [2] https://travis-ci.com/github/apache/hudi/jobs/518067393 > > Best, > Danny Chan > >
Re: [HELP] unstable tests in the travis CI
Hi Danny, There is a bug in schema resolve in DefaultSource which lead the test case 2 crash. I will submit a PR to solve this later. 2021年6月23日 下午1:49，Danny Chan 写道： Hi, fellows, there are two test cases in the travis CI that fails very often, which blocks our coding too many times, please, if these tests are not stable, can we disable them first ? They are annoying ~ TestHoodieBackedMetadata.testOnlyValidPartitionsAdded[1] HoodieSparkSqlWriterSuite: schema evolution for ... [2] [1] https://travis-ci.com/github/apache/hudi/jobs/518067391 [2] https://travis-ci.com/github/apache/hudi/jobs/518067393 Best, Danny Chan
