On Mon, Apr 4, 2022 at 1:11 PM Dave Marion <dmario...@gmail.com> wrote: > > I understand the desire to see less coupling for the optional features, but > getting to that point for ScanServers (and less so for ExternalCompactions) > would be a ton of work I think.
The likelihood of it being a lot of work doesn't mean it shouldn't be done. It's much more likely you could get help doing the work, if needed, after the 2.1 release, when some of the time that people are putting in 2.1 quality controls / testing, is freed up to work on other tasks. > The concern that I brought up in the "2.1 > Release TODOs" thread regarding planning has not been addressed. If there > was a defined path forward, then that might make it easier to see how this > feature gets added in the near-future in whatever form it takes. LOL, I forked the mailing list conversation because Keith started discussing the feature merits in the release planning thread, and it seemed like a separate topic. Now, you're talking about release planning in this thread. I'm not sure where to reply! 🤦 I guess the lines are blurred. I'll reply here. If we decide not to merge it into 2.1, what would be your preferred path forward? What's your contingency plan for this feature if the community doesn't include it in 2.1? Here's some possibilities: * Could release in non-LTM 2.2 (release right away) * Could release in non-LTM 3.0 (to include with other 3.0 changes) * Could do some work to decouple, and create a separate optional add-on jar for 2.1 By default, I would expect it to go into the next release (probably named 3.0), unless there's another path proposed that we can get consensus around. The more decoupled it is from Accumulo's internals, though, makes it more likely to be usable with more versions, even 2.1. If completely decoupled, it could be released at any time on its own schedule and be made to work with any version you like. It wouldn't need to be released coupled to an Accumulo release. > > Regarding the concern about the readiness of the feature branch, Keith is > doing a last pass review on the draft and then I believe we are ready to > take it out of draft state. I think it will be before the end of this week. > We have added six new integration tests and we have done some local and > cluster testing. Taking it out of draft state means that some people will begin to look at it. But, others, whose focus is on polishing existing features and testing for 2.1 won't have time, unless they prioritize your PR over the existing prep for 2.1. I'm not sure how long that would take. > Regarding the concern mentioned above, "availability of time to review/test > such a big feature without delaying 2.1," I didn't realize that we had a > schedule. We don't have a strict schedule. We never have. It's informal. It's just that 2.1 has been under development for so long already. I've been feeling the pressure build to get it released. I think a lot of our community has already started wanting a 2.x LTM release once we started discussing the LTM concept, and they've been stuck with 2.0 as the only 2.x version to work with. We only reluctantly did a CVE patch for 2.0, when it was non-LTM, only because we were not able to get 2.1 out at that time. Mike also suggested in Slack months ago (January 27th), that we should consider branching a 2.2 for new features, so 2.1 doesn't get slowed down. So, I know I'm not the only one who has felt 2.1 is getting a little fat with feature changes and overdue for a release. You commented on that thread, so I know you had at least a peripheral awareness at some point, that there was interest in wrapping up 2.1 before you even started the ScanServer work, way back when we were still cleaning up the tests from the breakages caused by the big external compactions and thread pool changes. > Does it matter if it takes 2/4/6/8 weeks to test the 1000+ > completed issues in this release? Yes. I think so. First, most of those changes are trivial, and have already been thoroughly tested and proven during development. While we do overall testing near the end of development near a release, most of the testing is done continuously throughout development, even for the non-trivial changes. We often test features thoroughly as they are added before we move on to the next feature to work on. So, it's misleading to suggest that there are 1000+ untested features queued up to be tested and that this is merely one more. Second, while the specific duration 2 weeks vs. 8 weeks doesn't necessarily matter, it does matter whether we're testing a moving target or not. At this point, our testing target is still moving, but it should be moving much more slowly than it was at the beginning of the development cycle, and it should be slowing down, not speeding up with new features. Many of us have already been thinking about a 2.1 release for several months and have been working on fixing tests, making test improvements, wrapping up existing features, and doing code quality checks, instead of doing new features, checking the open tickets, and closing open PRs, specifically because we've moved informally into a "wrapping up" phase. I myself only looked into cleaning up ZooKeeper recently, because we had several tickets open marked for 2.1 for singleton manager cleanup, and I wanted to see if we could get them in or if they needed to be bumped... because I'm focused on wrapping up 2.1. I suspect others are in a similar mindset for 2.1. > I know that we want to finish up the > 2.1.0 release, but is there a target date? No, there isn't a specific target date. We've never had a specific target date. What we've had is a general need to have a reasonable cut off point once momentum has started building, and consensus has grown towards the conclusion that we're feature-sufficient for a release. I think that momentum has already been building for a while now. We don't need a specific target date, but we do need development to slow down as we polish up, so we don't have a rapidly moving target to chase. Even if I understood the rush to get it released in 2.1, which I don't, I still think it would likely delay 2.1 a lot, because we don't have unlimited developer time and expertise. So we'd have to have consensus on accepting that delay as a trade-off for including it. Speaking only for myself, I would prefer we release without it, rather than delay, and then focus on more big features for the next release, and release more frequently... something this community has discussed on the mailing list many times, but never managed to accomplish. Part of the goal of LTM releases was so we could have more frequent non-LTM releases. If we put so much importance into getting all our features into LTM releases, regardless of how long it delays, or where we're at in the development cycle, we're never going to do non-LTM releases, because we just won't have the energy or motivation to do it.
