Correction: 3/12 or 3/13, as in March 12 or 13th. -C On Tue, Feb 20, 2018 at 3:59 PM, Chris Douglas <cdoug...@apache.org> wrote: > Looking at the poll [1], it looks like 12/12 or 12/13 would be the > best day to do this. If you haven't posted your availability and would > like to participate, please fill this out soon. > > We'll work on some common space in the bay area, but we'll also open a > channel for folks who can't attend in-person. -C > > [1]: https://doodle.com/poll/r22znitzae9apfbf > > > On Fri, Feb 9, 2018 at 1:45 PM, Chris Douglas <cdoug...@apache.org> wrote: >> On Thu, Feb 8, 2018 at 5:47 PM, 郑锴(铁杰) <zhengkai...@alibaba-inc.com> wrote: >>>>>I'm looking at you, TestDFSStripedOutputStreamWithFailure ... >>> AFAIK and IMO, it's pretty hard to get all the test cases stably running >>> given the limitation of MiniDFSCluster, and if we'd agree on that, we could >>> remove these cases as unit tests and cover them in integration tests instead >>> using a true cluster, like based on k8s infra. >> >> Since its inception, the Mini*Clusters have made it much easier to >> write unit tests (good!), but these tests take more time and resources >> than fine-grained tests (unfortunate) and they're often unreliable >> (bad). As we see in issues like HDFS-12711, the resource problem seems >> to harm reliability across the entire suite of tests (dire). >> >> Running integration tests with our CI is helpful, but perhaps we >> should separate the Mini*Cluster tests and/or press contributors to >> write lighter unit tests for new functionality. -C >> >>> We're lacking basic facility >>> infra env and tools to get most of the complicated functionalities well >>> tested and covered, so let's avoid too much complicated tests. Fixing of >>> such tests should definitely help and be appreciated. >>> >>> Regards, >>> Kai >>> >>> ------------------------------------------------------------------ >>> 发件人：Chris Douglas <cdoug...@apache.org> >>> 发送时间：2018年2月8日(星期四) 08:39 >>> 收件人：Hdfs-dev <hdfs-dev@hadoop.apache.org> >>> 主 题：Re: [DISCUSS] Meetup for HDFS tests and build infra >>> >>> Created a poll [1] to inform scheduling. -C >>> >>> [1]: https://doodle.com/poll/r22znitzae9apfbf >>> >>> On Tue, Feb 6, 2018 at 3:09 PM, Chris Douglas <cdoug...@apache.org> wrote: >>>> The HDFS build is not healthy. Many of the unit tests aren't actually >>>> run in Jenkins due to resource exhaustion, haven't been updated since >>>> build/test/data was the test temp dir, or are chronically unstable >>>> (I'm looking at you, TestDFSStripedOutputStreamWithFailure). The >>>> situation has deteriorated slowly, but we can't confidently merge >>>> patches, let alone significant features, when our CI infra is in this >>>> state. >>>> >>>> How would folks feel about a half to full-day meetup to work through >>>> patches improving this, specifically? We can improve tests, >>>> troubleshoot the build, and rev/commit existing patches. It would >>>> require some preparation, so the simultaneous attention is productive >>>> and not a coordination bottleneck. I started a wiki page for this [1], >>>> please add to it. >>>> >>>> If enough people can make time for this, say in 2-3 weeks, the project >>>> would certainly benefit. -C >>>> >>>> [1]: https://s.apache.org/ng3C >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org >>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
--------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
