That test uses many concurrent writes and I’ve seen cases when it led to deadlocks in our test HMS. I think HMS is capable of recovering on its own but that process can be slow in highly concurrent environments. There is a 2 min timeout in that test so it can potentially fail. I’ve seen a deadlock but 2 min was always enough for that test in my local env and internal/upstream build pipelines. If there is an environment that constantly or frequently hits this problem, it would be great to check debug logs.
I am +1 on releasing RC2. I checked it locally. - Anton > On 30 Apr 2020, at 02:52, Mass Dosage <massdos...@gmail.com> wrote: > > The build for RC2 worked fine for me, I didn't get a failure on > "TestHiveTableConcurrency". Perhaps there is some kind of race condition in > the test? I have seen timeout errors like that when I ran tests on an > overloaded machine, could that have been the case? > > On Thu, 30 Apr 2020 at 08:32, OpenInx <open...@gmail.com > <mailto:open...@gmail.com>> wrote: > I checked the rc2, seems the TestHiveTableConcurrency is broken, may need to > fix it. > > 1. Download the tarball and check the signature & checksum: OK > 2. license checking: RAT checks passed. > 3. Build and test the project (java8): > org.apache.iceberg.hive.TestHiveTableConcurrency > testConcurrentConnections > FAILED > java.lang.AssertionError: Timeout > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.assertTrue(Assert.java:41) > at > org.apache.iceberg.hive.TestHiveTableConcurrency.testConcurrentConnections(TestHiveTableConcurrency.java:106) > > On Thu, Apr 30, 2020 at 9:29 AM Ryan Blue <b...@apache.org > <mailto:b...@apache.org>> wrote: > Hi everyone, > > I propose the following candidate to be released as the official Apache > Iceberg 0.8.0-incubating release. > > The commit id is 8c05a2f5f1c8b111c049d43cf15cd8a51920dda1 > * This corresponds to the tag: apache-iceberg-0.8.0-incubating-rc2 > * > https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc2 > > <https://github.com/apache/incubator-iceberg/commits/apache-iceberg-0.8.0-incubating-rc2> > * https://github.com/apache/incubator-iceberg/tree/8c05a2f5 > <https://github.com/apache/incubator-iceberg/tree/8c05a2f5> > > The release tarball, signature, and checksums are here: > * > https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc2/ > > <https://dist.apache.org/repos/dist/dev/incubator/iceberg/apache-iceberg-0.8.0-incubating-rc2/> > > You can find the KEYS file here: > * https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS > <https://dist.apache.org/repos/dist/dev/incubator/iceberg/KEYS> > > Convenience binary artifacts are staged in Nexus. The Maven repository URL is: > * https://repository.apache.org/content/repositories/orgapacheiceberg-1006/ > <https://repository.apache.org/content/repositories/orgapacheiceberg-1006/> > > This release contains many bug fixes and several new features: > * Actions to remove orphaned files and to optimize metadata for query > performance > * Support for ORC data files > * Snapshot cherry-picking > * Incremental scan planning based on table history > * In and notIn expressions > * An InputFormat for writing MR jobs > > Please download, verify, and test. > > Please vote in the next 72 hours. > > [ ] +1 Release this as Apache Iceberg 0.8.0-incubating > [ ] +0 > [ ] -1 Do not release this because... > > -- > Ryan Blue
