I have committed PHOENIX-7191 (thanks for the review, Viraj). I have also put up a PR <https://github.com/apache/phoenix/pull/1810> for PHOENIX-7193.
I was not able to test them comprehensively, as I am still struggling with some issues with HBase 3 where simple GETs initiated from a coprocessor hang until they time out, but I wanted to get the known fixes in to unblock 5.2. Istvan On Tue, Feb 13, 2024 at 6:24 AM Istvan Toth <st...@apache.org> wrote: > We didn't really have such branches for past releases (that I have > followed), but we could change the practice. > > When I acted as an RM, I didn't feel the need to branch early, but if it > helps your planned workflow, then sure. > At least it would remind us to actually complete the release in a > reasonable amount of time. > > Istvan > > On Tue, Feb 13, 2024 at 4:30 AM Viraj Jasani <vjas...@apache.org> wrote: > >> No worries, that’s fine, both 5.2.0 and 5.1.4 can wait for non-zk registry >> fixes for a week or so. >> >> On the other hand, how about we still cut 5.2 branch now and keep >> backporting changes landing on master branch to 5.2 if necessary. Once >> non-zk registries are fixed, I can start with 5.2.0 release preparation. >> >> >> On Sun, Feb 11, 2024 at 11:28 PM Istvan Toth <st...@apache.org> wrote: >> >> > Thank you, Viraj, >> > >> > 5.1.4 also has the non-ZK registries feature, and has the same bugs as >> 5.2 >> > in that code. >> > >> > >> > >> > On Sat, Feb 10, 2024 at 7:00 PM Viraj Jasani <vjas...@apache.org> >> wrote: >> > >> > > Sure no worries, we can wait a few more days. >> > > >> > > In the meantime, I have merged the backport PR on 5.1 branch for the >> data >> > > integrity fixes. Once omid dependency change is in, I believe we are >> good >> > > to start with 5.1.4. >> > > >> > > Thank you Rajeshbabu for volunteering to take it up. >> > > >> > > >> > > On Sat, Feb 10, 2024 at 1:28 AM Istvan Toth <st...@apache.org> wrote: >> > > >> > >> I appreciate the drive to get 5.2.0 out of the door, but I would >> prefer >> > to >> > >> have a few more days to fix the registry issues, >> > >> and run some tests on them before cutting the branch, Viraj. >> > >> The non-ZK registry support is one of the bigger new features, and >> I'd >> > >> prefer not to have known breaking bugs in the release. >> > >> Can we target Friday or the next Monday for the cut ? >> > >> >> > >> Istvan >> > >> >> > >> On Sat, Feb 10, 2024 at 10:07 AM rajeshb...@apache.org < >> > >> chrajeshbab...@gmail.com> wrote: >> > >> >> > >> > Yes Viraj, I can release 5.1.4 >> > >> > >> > >> > Thanks, >> > >> > Rajeshbabu. >> > >> > >> > >> > On Sat, Feb 10, 2024, 10:28 AM Viraj Jasani <vjas...@apache.org> >> > wrote: >> > >> > >> > >> >> I think we can also target 5.2.1 very soon, perhaps just next >> month, >> > >> with >> > >> >> more CVE fixes and any other fixes if ready. >> > >> >> >> > >> >> >> > >> >> On Fri, Feb 9, 2024 at 6:39 PM Viraj Jasani <vjas...@apache.org> >> > >> wrote: >> > >> >> >> > >> >> > For 5.2.0, it would be great to focus on the known data >> integrity >> > >> >> issues. >> > >> >> > We can fix non-zk registry, cover a few more CVEs by upgrading >> > third >> > >> >> party >> > >> >> > dependencies and stabilize tests. As for the tests, they don’t >> seem >> > >> >> broken, >> > >> >> > but are flaky. I have got multiple builds without any test >> failures >> > >> on >> > >> >> PR >> > >> >> > for PHOENIX-7106. >> > >> >> > >> > >> >> > If this looks good to you, I can start release preparation next >> > week. >> > >> >> What >> > >> >> > do you think, Istvan? >> > >> >> > >> > >> >> > In the meantime, I have 5.1 backport PR open, awaiting good >> build >> > >> >> results >> > >> >> > before committing it. >> > >> >> > Rajeshbabu, would you like to be RM for 5.1.4 once the PR is >> > merged? >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > On Fri, Feb 9, 2024 at 11:13 AM Istvan Toth <st...@apache.org> >> > >> wrote: >> > >> >> > >> > >> >> >> Yes, they basically make the non-ZK registries unusable. >> > >> >> >> (at least the connectionless problems should be fixed.) >> > >> >> >> >> > >> >> >> I hope to have the final fix for those sometime next week. >> > >> >> >> >> > >> >> >> Also have we looked at potential CVE issues on master recently >> ? >> > >> >> >> >> > >> >> >> I think we should also look at the most flakey tests I linked >> > above, >> > >> >> >> and fix them or at least make sure that they are test issues >> and >> > not >> > >> >> real >> > >> >> >> bugs. >> > >> >> >> >> > >> >> >> Istvan >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> On Fri, Feb 9, 2024 at 6:55 PM Viraj Jasani < >> vjas...@apache.org> >> > >> >> wrote: >> > >> >> >> >> > >> >> >> > Thanks Istvan. >> > >> >> >> > I would like to cut 5.2 branch from master. Do you see non-ZK >> > >> >> registry >> > >> >> >> for >> > >> >> >> > MapReduce jobs as blocker for 5.2.0? >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > On Fri, Feb 9, 2024 at 12:24 AM Istvan Toth < >> st...@apache.org> >> > >> >> wrote: >> > >> >> >> > >> > >> >> >> > > >> > >> ParallelPhoenixConnectionFailureTest.testExecuteQueryChainFailure >> > >> >> also >> > >> >> >> > > fails too often, especially when the test host is slow >> and/or >> > >> the >> > >> >> >> load is >> > >> >> >> > > high. >> > >> >> >> > > On my fast laptop, I can semi-reliably break it by running >> > >> >> >> > > mvn clean verify -am -pl phoenix-core -DnumForkedUT=20 >> > >> >> >> > > >> > >> >> >> > > On Fri, Feb 9, 2024 at 9:19 AM Istvan Toth < >> st...@apache.org> >> > >> >> wrote: >> > >> >> >> > > >> > >> >> >> > > > We're making progress. >> > >> >> >> > > > I can see that Viraj has just landed PHOENIX-7601, and >> > >> Rajeshbabu >> > >> >> >> has >> > >> >> >> > > > released Omid 1.1.1. >> > >> >> >> > > > Thank you! >> > >> >> >> > > > >> > >> >> >> > > > At the moment, the following outstanding issues are on my >> > >> radar: >> > >> >> >> > > > >> > >> >> >> > > > https://issues.apache.org/jira/browse/PHOENIX-7191 >> > >> >> >> > > > https://issues.apache.org/jira/browse/PHOENIX-7193 >> > >> >> >> > > > >> > >> >> >> > > > These are bugs in my non-ZK registry implementation, >> which >> > >> were >> > >> >> >> found >> > >> >> >> > > > during HBase 3 work. >> > >> >> >> > > > I have some PRs up, but they may not be complete. I will >> > push >> > >> for >> > >> >> >> > reviews >> > >> >> >> > > > once I have the HBase 3 tests passing, and possibly >> updated >> > >> them >> > >> >> >> based >> > >> >> >> > on >> > >> >> >> > > > that. >> > >> >> >> > > > >> > >> >> >> > > > We also have a number of very flakey tests, see: >> > >> >> >> > > > >> > >> >> >> > > > >> > >> >> >> > > > >> > >> >> >> > > >> > >> >> >> > >> > >> >> >> >> > >> >> >> > >> >> > >> https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/test_results_analyzer/ >> > >> >> >> > > > >> > >> >> >> > > > >> > >> >> >> > > > >> > >> >> >> > > > On Tue, Jan 30, 2024 at 7:09 AM Istvan Toth < >> > >> st...@cloudera.com> >> > >> >> >> > wrote: >> > >> >> >> > > > >> > >> >> >> > > >> As Viraj wrote, those are just plans. >> > >> >> >> > > >> If HBase 3 won't be released by the time the other >> features >> > >> are >> > >> >> >> ready, >> > >> >> >> > > >> then it won't make it into 5.3. >> > >> >> >> > > >> If other major features are ready by that time, then >> they >> > >> will >> > >> >> be >> > >> >> >> > > >> included. (though we are not aware of any now) >> > >> >> >> > > >> >> > >> >> >> > > >> As for the new major version, in the past Phoenix didn't >> > >> have a >> > >> >> >> > > >> compatibility module system, >> > >> >> >> > > >> so a new branch was required, which didn't support >> older >> > >> >> HBases. >> > >> >> >> > Also, >> > >> >> >> > > >> the API changes between HBase 1.x and 2.x were much >> larger, >> > >> >> >> > > >> The HBase 2 and 3 API are pretty similar, apart from the >> > >> >> removal of >> > >> >> >> > > >> deprecated 1.x APIs. (and the protobuf/protocol thing, >> > which >> > >> >> >> requires >> > >> >> >> > a >> > >> >> >> > > >> rather ugly hack). >> > >> >> >> > > >> >> > >> >> >> > > >> I will start the discussion on how we can add HBase 3 >> > >> support as >> > >> >> >> soon >> > >> >> >> > as >> > >> >> >> > > >> I have a working POC patch. >> > >> >> >> > > >> >> > >> >> >> > > >> We could call 5.3 6.0 instead, after all Phoenix isn't >> > using >> > >> a >> > >> >> >> strict >> > >> >> >> > > >> semantic versioning, but then 6.0 would also support >> HBase >> > 2. >> > >> >> >> > > >> If we do not come to a consensus on the version name, we >> > can >> > >> >> always >> > >> >> >> > have >> > >> >> >> > > >> a vote on it. >> > >> >> >> > > >> >> > >> >> >> > > >> I think that the main motivation is that the community >> > wants >> > >> to >> > >> >> >> > maintain >> > >> >> >> > > >> as few branches as possible. >> > >> >> >> > > >> >> > >> >> >> > > >> Istvan >> > >> >> >> > > >> >> > >> >> >> > > >> On Mon, Jan 29, 2024 at 10:02 PM Stephen Jiang < >> > >> >> >> > syuanjiang...@gmail.com >> > >> >> >> > > > >> > >> >> >> > > >> wrote: >> > >> >> >> > > >> >> > >> >> >> > > >>> I am not sure how close HBase 3.0 is. Even if it is >> only >> > >> less >> > >> >> >> than >> > >> >> >> > one >> > >> >> >> > > >>> year away, the adoption would be low at the >> beginning. I >> > >> don't >> > >> >> >> think >> > >> >> >> > > 5.3 >> > >> >> >> > > >>> should wait for that. And traditionally, Phoenix >> would >> > >> have a >> > >> >> >> major >> > >> >> >> > > >>> release to support the HBase major release (4.x for >> HBase >> > >> 1.x >> > >> >> and >> > >> >> >> 5.x >> > >> >> >> > > for >> > >> >> >> > > >>> HBase 2.x), in this case, we are talking about Phoenix >> 6.0 >> > >> for >> > >> >> >> HBase >> > >> >> >> > > 3.0. >> > >> >> >> > > >>> >> > >> >> >> > > >>> Maybe we should adopt the HBase release model: master >> > branch >> > >> >> for >> > >> >> >> next >> > >> >> >> > > >>> major >> > >> >> >> > > >>> release (6.0) and branch-5.x branch for next 5.x minor >> > >> release >> > >> >> and >> > >> >> >> > > >>> branch-5.2 for 5.2 minor release. >> > >> >> >> > > >>> >> > >> >> >> > > >>> Thanks >> > >> >> >> > > >>> Stephen >> > >> >> >> > > >>> >> > >> >> >> > > >>> >> > >> >> >> > > >>> >> > >> >> >> > > >>> On Thu, Jan 25, 2024 at 11:50 PM Viraj Jasani < >> > >> >> vjas...@apache.org >> > >> >> >> > >> > >> >> >> > > >>> wrote: >> > >> >> >> > > >>> >> > >> >> >> > > >>> > Sounds good. >> > >> >> >> > > >>> > >> > >> >> >> > > >>> > Planned major changes for 5.3.0: >> > >> >> >> > > >>> > >> > >> >> >> > > >>> > 1. JSON support. >> > >> >> >> > > >>> > 2. HBase 3.0 support. >> > >> >> >> > > >>> > 3. CDC feature (leveraging uncovered global index >> > >> framework >> > >> >> and >> > >> >> >> > JSON >> > >> >> >> > > >>> > support). >> > >> >> >> > > >>> > >> > >> >> >> > > >>> > >> > >> >> >> > > >>> > On Wed, Jan 24, 2024 at 8:22 PM Kadir Ozdemir >> > >> >> >> > > >>> > <kozde...@salesforce.com.invalid> wrote: >> > >> >> >> > > >>> > >> > >> >> >> > > >>> > > I suggest including another major change for 5.3, >> > >> Phoenix >> > >> >> CDC, >> > >> >> >> > > >>> > > https://issues.apache.org/jira/browse/PHOENIX-7001 >> . >> > >> The PR >> > >> >> >> for >> > >> >> >> > it >> > >> >> >> > > >>> will >> > >> >> >> > > >>> > be >> > >> >> >> > > >>> > > posted soon. >> > >> >> >> > > >>> > > >> > >> >> >> > > >>> > > On Thu, Jan 25, 2024 at 5:35 AM Viraj Jasani < >> > >> >> >> vjas...@apache.org >> > >> >> >> > > >> > >> >> >> > > >>> wrote: >> > >> >> >> > > >>> > > >> > >> >> >> > > >>> > > > Thanks Istvan! I agree with your points. It’s >> really >> > >> >> been a >> > >> >> >> > while >> > >> >> >> > > >>> we >> > >> >> >> > > >>> > are >> > >> >> >> > > >>> > > > talking about releasing 5.2.0 and yet due to >> > bandwidth >> > >> >> >> issues, >> > >> >> >> > > >>> unable >> > >> >> >> > > >>> > to >> > >> >> >> > > >>> > > do >> > >> >> >> > > >>> > > > so. >> > >> >> >> > > >>> > > > >> > >> >> >> > > >>> > > > I agree to getting these fixes out, cut 5.2 and >> > start >> > >> the >> > >> >> >> > release >> > >> >> >> > > >>> work >> > >> >> >> > > >>> > > and >> > >> >> >> > > >>> > > > in the meantime I also need to prepare 5.1 >> backport. >> > >> >> >> > > >>> > > > >> > >> >> >> > > >>> > > > For 5.3, let's plan HBase 3.0 support and JSON as >> > >> major >> > >> >> >> > changes. >> > >> >> >> > > >>> > > > >> > >> >> >> > > >>> > > > >> > >> >> >> > > >>> > > > On Mon, Jan 22, 2024 at 9:17 PM Istvan Toth < >> > >> >> >> st...@apache.org> >> > >> >> >> > > >>> wrote: >> > >> >> >> > > >>> > > > >> > >> >> >> > > >>> > > > > Hi! >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > In my opinion cutting 5.2 now only makes sense >> IF >> > >> we DO >> > >> >> >> NOT >> > >> >> >> > > plan >> > >> >> >> > > >>> to >> > >> >> >> > > >>> > > > release >> > >> >> >> > > >>> > > > > the outstanding big features (like JSON) in >> 5.2. , >> > >> >> >> otherwise >> > >> >> >> > > it's >> > >> >> >> > > >>> > just >> > >> >> >> > > >>> > > > > extra work to maintain more branches. >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > Having said that, releasing a 5.2 and 5.1.4 >> with >> > the >> > >> >> data >> > >> >> >> > > >>> integrity >> > >> >> >> > > >>> > > fixes >> > >> >> >> > > >>> > > > > real soon, and then releasing 5.3 in a few >> months >> > >> with >> > >> >> >> JSON, >> > >> >> >> > > and >> > >> >> >> > > >>> any >> > >> >> >> > > >>> > > > other >> > >> >> >> > > >>> > > > > outstanding big features >> > >> >> >> > > >>> > > > > that are close to being finished (and HBase 3.0 >> > >> >> support, >> > >> >> >> if >> > >> >> >> > > it's >> > >> >> >> > > >>> > ready >> > >> >> >> > > >>> > > by >> > >> >> >> > > >>> > > > > then) would not be a bad idea. >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > On the CLDR side the only outstanding big >> feature >> > >> which >> > >> >> >> could >> > >> >> >> > > >>> impact >> > >> >> >> > > >>> > > > > Viraj's integrity work is HBase 3.0 support, >> and >> > >> even >> > >> >> >> that is >> > >> >> >> > > >>> only >> > >> >> >> > > >>> > > > because >> > >> >> >> > > >>> > > > > it may require some larger refactors of >> existing >> > >> code, >> > >> >> not >> > >> >> >> > > >>> because it >> > >> >> >> > > >>> > > > would >> > >> >> >> > > >>> > > > > change the actual behaviour or algorithms. >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > Phoenix used to have several minor releases per >> > >> year, >> > >> >> the >> > >> >> >> > > current >> > >> >> >> > > >>> > state >> > >> >> >> > > >>> > > > of >> > >> >> >> > > >>> > > > > extreme longevity of 5.1 and several big new >> > >> features >> > >> >> >> being >> > >> >> >> > > >>> added to >> > >> >> >> > > >>> > it >> > >> >> >> > > >>> > > > > (like uncovered indexes) is not ideal. >> > >> >> >> > > >>> > > > > Releasing 5.2 and 5.3 relatively close together >> > >> could >> > >> >> be a >> > >> >> >> > > >>> return to >> > >> >> >> > > >>> > a >> > >> >> >> > > >>> > > > > quicker cadence for minor releases, which could >> > also >> > >> >> help >> > >> >> >> > with >> > >> >> >> > > >>> the >> > >> >> >> > > >>> > > public >> > >> >> >> > > >>> > > > > image and adoption of Phoenix. >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > We were talking about releasing 5.2 at least a >> > year >> > >> >> ago, >> > >> >> >> and >> > >> >> >> > I >> > >> >> >> > > >>> have >> > >> >> >> > > >>> > > > started >> > >> >> >> > > >>> > > > > working on that then, but then emergencies have >> > come >> > >> >> up at >> > >> >> >> > > >>> $dayjob, >> > >> >> >> > > >>> > > and I >> > >> >> >> > > >>> > > > > could not see that through. >> > >> >> >> > > >>> > > > > (So I am in part responsible for the lack of >> minor >> > >> >> >> releases) >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > regards >> > >> >> >> > > >>> > > > > Istvan >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > On Mon, Jan 22, 2024 at 11:35 PM Viraj Jasani < >> > >> >> >> > > >>> vjas...@apache.org> >> > >> >> >> > > >>> > > > wrote: >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > > > Sorry for the late reply. >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > > No problem with that. I can cut 5.2 branch by >> > the >> > >> >> end of >> > >> >> >> > this >> > >> >> >> > > >>> week >> > >> >> >> > > >>> > or >> > >> >> >> > > >>> > > > at >> > >> >> >> > > >>> > > > > > the start of next week. >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > > If there is any very big change or feature >> ready >> > >> for >> > >> >> >> merge >> > >> >> >> > to >> > >> >> >> > > >>> > master >> > >> >> >> > > >>> > > > > branch >> > >> >> >> > > >>> > > > > > with PR approvals already in place, please do >> > let >> > >> me >> > >> >> >> know >> > >> >> >> > so >> > >> >> >> > > >>> that I >> > >> >> >> > > >>> > > can >> > >> >> >> > > >>> > > > > > help collaborate on how best we can get it >> > merged >> > >> >> >> without >> > >> >> >> > > >>> impacting >> > >> >> >> > > >>> > > 5.2 >> > >> >> >> > > >>> > > > > > release if required. My main motivation was >> for >> > >> any >> > >> >> big >> > >> >> >> > > change >> > >> >> >> > > >>> to >> > >> >> >> > > >>> > go >> > >> >> >> > > >>> > > > > > through newly introduced tests so that we >> know >> > >> that >> > >> >> >> > anything >> > >> >> >> > > >>> > > additional >> > >> >> >> > > >>> > > > > is >> > >> >> >> > > >>> > > > > > not broken, and also to prioritize for >> upcoming >> > >> 5.2.0 >> > >> >> >> and >> > >> >> >> > > 5.1.4 >> > >> >> >> > > >>> > > > releases. >> > >> >> >> > > >>> > > > > > Moreover, there are several PRs getting >> merged >> > on >> > >> the >> > >> >> >> > master >> > >> >> >> > > >>> > branch, >> > >> >> >> > > >>> > > we >> > >> >> >> > > >>> > > > > can >> > >> >> >> > > >>> > > > > > continue that as long as they are not very >> big >> > >> >> changes, >> > >> >> >> > which >> > >> >> >> > > >>> might >> > >> >> >> > > >>> > > > > require >> > >> >> >> > > >>> > > > > > significant time to understand any >> correlation >> > >> with >> > >> >> data >> > >> >> >> > > >>> integrity >> > >> >> >> > > >>> > > > > issues. >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > > The PR is also ready for review with some >> > >> additional >> > >> >> >> cases >> > >> >> >> > > >>> fixed >> > >> >> >> > > >>> > last >> > >> >> >> > > >>> > > > > week: >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > >> > >> >> >> > > >>> > > >> > >> >> >> > > >>> > >> > >> >> >> > > >>> >> > >> >> >> > > >> > >> >> >> > >> > >> >> >> >> > >> >> >> > >> >> > >> https://urldefense.com/v3/__https://github.com/apache/phoenix/pull/1736__;!!DCbAVzZNrAf4!D4OVjUp2EWW2BqhGnBxsapDX_AHsibRphIpoFBWfgRsd3dsAikrFLo6PGxdTzGbSXJJ2fJ0j9mcz3asXMXo$ >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > > Depending on the review bandwidth, I am >> hopeful >> > we >> > >> >> >> should >> > >> >> >> > be >> > >> >> >> > > >>> good >> > >> >> >> > > >>> > to >> > >> >> >> > > >>> > > > land >> > >> >> >> > > >>> > > > > > them sooner. >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > > On Wed, Jan 17, 2024 at 11:31 AM Rushabh Shah >> > >> >> >> > > >>> > > > > > <rushabh.s...@salesforce.com.invalid> wrote: >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > > > Thank you Viraj for initiating this thread. >> > >> >> >> > > >>> > > > > > > >> > >> >> >> > > >>> > > > > > > > Given the critical nature of these >> issues, I >> > >> >> would >> > >> >> >> like >> > >> >> >> > > to >> > >> >> >> > > >>> > > propose >> > >> >> >> > > >>> > > > > that >> > >> >> >> > > >>> > > > > > > we >> > >> >> >> > > >>> > > > > > > treat this as a high priority for the >> upcoming >> > >> >> 5.2.0 >> > >> >> >> > > >>> release, and >> > >> >> >> > > >>> > > not >> > >> >> >> > > >>> > > > > > > include any other feature or big change to >> > >> master >> > >> >> >> branch >> > >> >> >> > > >>> until we >> > >> >> >> > > >>> > > > merge >> > >> >> >> > > >>> > > > > > > this. >> > >> >> >> > > >>> > > > > > > >> > >> >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? >> > This >> > >> >> will >> > >> >> >> > > enable >> > >> >> >> > > >>> > other >> > >> >> >> > > >>> > > > > > > developers to merge features into master >> > branch >> > >> >> >> (5.3.0) >> > >> >> >> > and >> > >> >> >> > > >>> you >> > >> >> >> > > >>> > can >> > >> >> >> > > >>> > > > > take >> > >> >> >> > > >>> > > > > > > some more time to make sure we cover all >> the >> > >> corner >> > >> >> >> cases >> > >> >> >> > > >>> for the >> > >> >> >> > > >>> > > > data >> > >> >> >> > > >>> > > > > > > integrity issues that you uncovered. >> > >> >> >> > > >>> > > > > > > >> > >> >> >> > > >>> > > > > > > >> > >> >> >> > > >>> > > > > > > On Fri, Jan 5, 2024 at 6:38 PM Viraj >> Jasani < >> > >> >> >> > > >>> vjas...@apache.org> >> > >> >> >> > > >>> > > > > wrote: >> > >> >> >> > > >>> > > > > > > >> > >> >> >> > > >>> > > > > > > > Sounds good, thanks Rajeshbabu. I will >> try >> > to >> > >> get >> > >> >> >> the >> > >> >> >> > > >>> first PR >> > >> >> >> > > >>> > > out >> > >> >> >> > > >>> > > > > next >> > >> >> >> > > >>> > > > > > > > week and while reviews happen in >> parallel, >> > >> will >> > >> >> try >> > >> >> >> to >> > >> >> >> > > get >> > >> >> >> > > >>> 5.1 >> > >> >> >> > > >>> > PR >> > >> >> >> > > >>> > > > > soon. >> > >> >> >> > > >>> > > > > > > > >> > >> >> >> > > >>> > > > > > > > >> > >> >> >> > > >>> > > > > > > > On Wed, Jan 3, 2024 at 8:49 PM >> > >> >> >> rajeshb...@apache.org < >> > >> >> >> > > >>> > > > > > > > chrajeshbab...@gmail.com> wrote: >> > >> >> >> > > >>> > > > > > > > >> > >> >> >> > > >>> > > > > > > > > Hi Viraj, >> > >> >> >> > > >>> > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > Would be better to include the changes >> in >> > >> 5.1.4 >> > >> >> >> as >> > >> >> >> > in >> > >> >> >> > > >>> any >> > >> >> >> > > >>> > way >> > >> >> >> > > >>> > > it >> > >> >> >> > > >>> > > > > > will >> > >> >> >> > > >>> > > > > > > > take >> > >> >> >> > > >>> > > > > > > > > at least 3-4 days to complete the omid >> > >> release. >> > >> >> >> > > >>> > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > Thanks, >> > >> >> >> > > >>> > > > > > > > > Rajeshbabu. >> > >> >> >> > > >>> > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > On Thu, Jan 4, 2024 at 5:06 AM Viraj >> > Jasani >> > >> < >> > >> >> >> > > >>> > > vjas...@apache.org> >> > >> >> >> > > >>> > > > > > > wrote: >> > >> >> >> > > >>> > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > Thank you Kadir and Geoffrey for your >> > >> >> replies!! >> > >> >> >> > > >>> > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, which >> is >> > >> also >> > >> >> >> listed >> > >> >> >> > as >> > >> >> >> > > >>> a Fix >> > >> >> >> > > >>> > > > > Version >> > >> >> >> > > >>> > > > > > > for >> > >> >> >> > > >>> > > > > > > > > > > PHOENIX-7106? >> > >> >> >> > > >>> > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > Yes, it also needs to be ported to >> 5.1. >> > >> Once >> > >> >> the >> > >> >> >> > > >>> master PR >> > >> >> >> > > >>> > is >> > >> >> >> > > >>> > > > up >> > >> >> >> > > >>> > > > > > for >> > >> >> >> > > >>> > > > > > > > > final >> > >> >> >> > > >>> > > > > > > > > > review, I would start working on the >> > >> backport >> > >> >> >> PR. >> > >> >> >> > > >>> > > > > > > > > > We just need some more additional >> > testing >> > >> to >> > >> >> >> ensure >> > >> >> >> > > old >> > >> >> >> > > >>> > > client >> > >> >> >> > > >>> > > > > > (e.g. >> > >> >> >> > > >>> > > > > > > > > 5.1.3) >> > >> >> >> > > >>> > > > > > > > > > is compatible with the new server >> with >> > the >> > >> >> >> changes. >> > >> >> >> > > >>> > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > Hence, yes it is now a blocker for >> > >> upcoming >> > >> >> >> 5.1.4 >> > >> >> >> > as >> > >> >> >> > > >>> well >> > >> >> >> > > >>> > > since >> > >> >> >> > > >>> > > > > > 5.1.4 >> > >> >> >> > > >>> > > > > > > > RC >> > >> >> >> > > >>> > > > > > > > > > preparation is still pending (while >> Omid >> > >> >> >> release is >> > >> >> >> > > in >> > >> >> >> > > >>> > > > progress). >> > >> >> >> > > >>> > > > > > > > > > Otherwise if 5.1.4 was ready for >> > release, >> > >> I >> > >> >> >> would >> > >> >> >> > > have >> > >> >> >> > > >>> > > proposed >> > >> >> >> > > >>> > > > > > > > immediate >> > >> >> >> > > >>> > > > > > > > > > 5.1.5 release to include the changes >> > >> proposed >> > >> >> >> with >> > >> >> >> > > >>> > > > PHOENIX-7106. >> > >> >> >> > > >>> > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > On Wed, Jan 3, 2024 at 3:08 PM >> Geoffrey >> > >> >> Jacoby < >> > >> >> >> > > >>> > > > > gjac...@apache.org >> > >> >> >> > > >>> > > > > > > >> > >> >> >> > > >>> > > > > > > > > wrote: >> > >> >> >> > > >>> > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > I agree that data integrity issues >> > are a >> > >> >> >> higher >> > >> >> >> > > >>> priority >> > >> >> >> > > >>> > > than >> > >> >> >> > > >>> > > > > > > feature >> > >> >> >> > > >>> > > > > > > > > > > development, so I also support the >> > >> >> decision. >> > >> >> >> The >> > >> >> >> > > fact >> > >> >> >> > > >>> > that >> > >> >> >> > > >>> > > > > > several >> > >> >> >> > > >>> > > > > > > of >> > >> >> >> > > >>> > > > > > > > > the >> > >> >> >> > > >>> > > > > > > > > > > major remaining 5.2 features are >> > >> currently >> > >> >> >> being >> > >> >> >> > > >>> > developed >> > >> >> >> > > >>> > > in >> > >> >> >> > > >>> > > > > > > > > > long-running >> > >> >> >> > > >>> > > > > > > > > > > feature branches also helps, as >> work >> > can >> > >> >> >> continue >> > >> >> >> > > >>> there >> > >> >> >> > > >>> > at >> > >> >> >> > > >>> > > > the >> > >> >> >> > > >>> > > > > > cost >> > >> >> >> > > >>> > > > > > > > of >> > >> >> >> > > >>> > > > > > > > > a >> > >> >> >> > > >>> > > > > > > > > > > rebase later. >> > >> >> >> > > >>> > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, which >> is >> > >> also >> > >> >> >> listed >> > >> >> >> > as >> > >> >> >> > > >>> a Fix >> > >> >> >> > > >>> > > > > Version >> > >> >> >> > > >>> > > > > > > for >> > >> >> >> > > >>> > > > > > > > > > > PHOENIX-7106? From the bug >> description >> > >> it >> > >> >> also >> > >> >> >> > > sounds >> > >> >> >> > > >>> > like >> > >> >> >> > > >>> > > > > 5.1.3 >> > >> >> >> > > >>> > > > > > > and >> > >> >> >> > > >>> > > > > > > > > the >> > >> >> >> > > >>> > > > > > > > > > > forthcoming .4 are affected, since >> we >> > >> have >> > >> >> >> > > >>> server-side >> > >> >> >> > > >>> > > paging >> > >> >> >> > > >>> > > > > in >> > >> >> >> > > >>> > > > > > > 5.1. >> > >> >> >> > > >>> > > > > > > > > > (Feel >> > >> >> >> > > >>> > > > > > > > > > > free to move that to a separate >> thread >> > >> if >> > >> >> you >> > >> >> >> > feel >> > >> >> >> > > it >> > >> >> >> > > >>> > > should >> > >> >> >> > > >>> > > > > be a >> > >> >> >> > > >>> > > > > > > > > > separate >> > >> >> >> > > >>> > > > > > > > > > > discussion.) Should this be a >> blocker >> > >> for >> > >> >> >> > releasing >> > >> >> >> > > >>> > 5.1.4? >> > >> >> >> > > >>> > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > Geoffrey >> > >> >> >> > > >>> > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > On Wed, Jan 3, 2024 at 5:06 PM >> Kadir >> > >> >> Ozdemir < >> > >> >> >> > > >>> > > > > > > > > > > ka...@gsuite.cloud.apache.org> >> > >> >> >> > > >>> > > > > > > > > > > wrote: >> > >> >> >> > > >>> > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > > Being a database, Phoenix has to >> > make >> > >> >> sure >> > >> >> >> that >> > >> >> >> > > the >> > >> >> >> > > >>> > data >> > >> >> >> > > >>> > > > > stays >> > >> >> >> > > >>> > > > > > on >> > >> >> >> > > >>> > > > > > > > > disk >> > >> >> >> > > >>> > > > > > > > > > > > intact and its queries return >> > correct >> > >> >> data. >> > >> >> >> In >> > >> >> >> > > this >> > >> >> >> > > >>> > case, >> > >> >> >> > > >>> > > > > > Phoenix >> > >> >> >> > > >>> > > > > > > > > fails >> > >> >> >> > > >>> > > > > > > > > > > to >> > >> >> >> > > >>> > > > > > > > > > > > return correct data for some >> queries >> > >> if >> > >> >> >> their >> > >> >> >> > > scans >> > >> >> >> > > >>> > > > > experience >> > >> >> >> > > >>> > > > > > > > region >> > >> >> >> > > >>> > > > > > > > > > > > movement. Now that we know these >> > data >> > >> >> >> integrity >> > >> >> >> > > >>> issues >> > >> >> >> > > >>> > > and >> > >> >> >> > > >>> > > > > how >> > >> >> >> > > >>> > > > > > to >> > >> >> >> > > >>> > > > > > > > > > > reproduce >> > >> >> >> > > >>> > > > > > > > > > > > them, fixing them should be our >> > first >> > >> >> >> priority. >> > >> >> >> > > >>> So, I >> > >> >> >> > > >>> > > fully >> > >> >> >> > > >>> > > > > > > support >> > >> >> >> > > >>> > > > > > > > > > this >> > >> >> >> > > >>> > > > > > > > > > > > proposal. >> > >> >> >> > > >>> > > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > > On Wed, Jan 3, 2024 at 10:58 PM >> > Viraj >> > >> >> >> Jasani < >> > >> >> >> > > >>> > > > > > vjas...@apache.org >> > >> >> >> > > >>> > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > wrote: >> > >> >> >> > > >>> > > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > > > Hello, >> > >> >> >> > > >>> > > > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > > > I would like to bring >> PHOENIX-7106 >> > >> >> >> > > >>> > > > > > > > > > > > > < >> > >> >> >> > > >>> > > > > > > > >> > >> >> >> > > >>> > > > > > > >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > >> > >> >> >> > > >>> > > >> > >> >> >> > > >>> > >> > >> >> >> > > >>> >> > >> >> >> > > >> > >> >> >> > >> > >> >> >> >> > >> >> >> > >> >> > >> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/PHOENIX-7106__;!!DCbAVzZNrAf4!FZG5sv55IC1NqItQLY7GKWgUG2Do0gSta01gOiSdd36Dx3XHGtQx4M3c9visVXIt9DctPQzS-orob9vhzrCfVA$ >> > >> >> >> > > >>> > > > > > > > > to everyone's >> > >> >> >> > > >>> > > > > > > > > > > > > attention here and brief about >> the >> > >> data >> > >> >> >> > > integrity >> > >> >> >> > > >>> > > issues >> > >> >> >> > > >>> > > > > that >> > >> >> >> > > >>> > > > > > > we >> > >> >> >> > > >>> > > > > > > > > have >> > >> >> >> > > >>> > > > > > > > > > > in >> > >> >> >> > > >>> > > > > > > > > > > > > various coprocessors. Majority >> of >> > >> the >> > >> >> >> issues >> > >> >> >> > > are >> > >> >> >> > > >>> > > related >> > >> >> >> > > >>> > > > to >> > >> >> >> > > >>> > > > > > the >> > >> >> >> > > >>> > > > > > > > > fact >> > >> >> >> > > >>> > > > > > > > > > > that >> > >> >> >> > > >>> > > > > > > > > > > > > we do not return valid rowkey >> for >> > >> >> certain >> > >> >> >> > > >>> queries. If >> > >> >> >> > > >>> > > any >> > >> >> >> > > >>> > > > > > > region >> > >> >> >> > > >>> > > > > > > > > > moves >> > >> >> >> > > >>> > > > > > > > > > > in >> > >> >> >> > > >>> > > > > > > > > > > > > the middle of the scan, the >> HBase >> > >> >> client >> > >> >> >> > relies >> > >> >> >> > > >>> on >> > >> >> >> > > >>> > the >> > >> >> >> > > >>> > > > last >> > >> >> >> > > >>> > > > > > > > > returned >> > >> >> >> > > >>> > > > > > > > > > > > rowkey >> > >> >> >> > > >>> > > > > > > > > > > > > and accordingly changes the >> scan >> > >> >> >> boundaries >> > >> >> >> > > >>> while the >> > >> >> >> > > >>> > > > > scanner >> > >> >> >> > > >>> > > > > > > is >> > >> >> >> > > >>> > > > > > > > > > > getting >> > >> >> >> > > >>> > > > > > > > > > > > > reset to continue the scan >> > >> operation. >> > >> >> If >> > >> >> >> the >> > >> >> >> > > >>> region >> > >> >> >> > > >>> > > does >> > >> >> >> > > >>> > > > > not >> > >> >> >> > > >>> > > > > > > > move, >> > >> >> >> > > >>> > > > > > > > > > scan >> > >> >> >> > > >>> > > > > > > > > > > > is >> > >> >> >> > > >>> > > > > > > > > > > > > not expected to return invalid >> > data, >> > >> >> >> however >> > >> >> >> > if >> > >> >> >> > > >>> the >> > >> >> >> > > >>> > > > region >> > >> >> >> > > >>> > > > > > > moves >> > >> >> >> > > >>> > > > > > > > in >> > >> >> >> > > >>> > > > > > > > > > the >> > >> >> >> > > >>> > > > > > > > > > > > > middle of ongoing scan >> operation, >> > >> scan >> > >> >> >> would >> > >> >> >> > > >>> return >> > >> >> >> > > >>> > > > > > > > > invalid/incorrect >> > >> >> >> > > >>> > > > > > > > > > > > data >> > >> >> >> > > >>> > > > > > > > > > > > > causing data integrity issues. >> > >> >> >> > > >>> > > > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > > > Given the critical nature of >> these >> > >> >> >> issues, I >> > >> >> >> > > >>> would >> > >> >> >> > > >>> > like >> > >> >> >> > > >>> > > > to >> > >> >> >> > > >>> > > > > > > > propose >> > >> >> >> > > >>> > > > > > > > > > that >> > >> >> >> > > >>> > > > > > > > > > > > we >> > >> >> >> > > >>> > > > > > > > > > > > > treat this as a high priority >> for >> > >> the >> > >> >> >> > upcoming >> > >> >> >> > > >>> 5.2.0 >> > >> >> >> > > >>> > > > > release, >> > >> >> >> > > >>> > > > > > > and >> > >> >> >> > > >>> > > > > > > > > not >> > >> >> >> > > >>> > > > > > > > > > > > > include any other feature or >> big >> > >> >> change to >> > >> >> >> > > master >> > >> >> >> > > >>> > > branch >> > >> >> >> > > >>> > > > > > until >> > >> >> >> > > >>> > > > > > > we >> > >> >> >> > > >>> > > > > > > > > > merge >> > >> >> >> > > >>> > > > > > > > > > > > > this. The PR is still not >> ready as >> > >> >> >> additional >> > >> >> >> > > >>> changes >> > >> >> >> > > >>> > > are >> > >> >> >> > > >>> > > > > > still >> > >> >> >> > > >>> > > > > > > > in >> > >> >> >> > > >>> > > > > > > > > my >> > >> >> >> > > >>> > > > > > > > > > > > > local, requiring rebase with >> the >> > >> >> current >> > >> >> >> > > master. >> > >> >> >> > > >>> > > > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > > > I would get back to this >> discuss >> > >> >> thread as >> > >> >> >> > soon >> > >> >> >> > > >>> as >> > >> >> >> > > >>> > the >> > >> >> >> > > >>> > > PR >> > >> >> >> > > >>> > > > > and >> > >> >> >> > > >>> > > > > > > the >> > >> >> >> > > >>> > > > > > > > > doc >> > >> >> >> > > >>> > > > > > > > > > > are >> > >> >> >> > > >>> > > > > > > > > > > > > updated with the latest >> findings >> > so >> > >> >> far. >> > >> >> >> The >> > >> >> >> > > >>> changes >> > >> >> >> > > >>> > > > > include >> > >> >> >> > > >>> > > > > > > many >> > >> >> >> > > >>> > > > > > > > > of >> > >> >> >> > > >>> > > > > > > > > > > our >> > >> >> >> > > >>> > > > > > > > > > > > > coproc scanner implementations >> and >> > >> >> hence >> > >> >> >> it >> > >> >> >> > > would >> > >> >> >> > > >>> > > require >> > >> >> >> > > >>> > > > > > > > > significant >> > >> >> >> > > >>> > > > > > > > > > > > > review as well. >> > >> >> >> > > >>> > > > > > > > > > > > > It would be great if we can >> hold >> > on >> > >> to >> > >> >> >> > merging >> > >> >> >> > > >>> any >> > >> >> >> > > >>> > > > feature >> > >> >> >> > > >>> > > > > or >> > >> >> >> > > >>> > > > > > > big >> > >> >> >> > > >>> > > > > > > > > > > change >> > >> >> >> > > >>> > > > > > > > > > > > to >> > >> >> >> > > >>> > > > > > > > > > > > > master branch until this gets >> in >> > so >> > >> as >> > >> >> to >> > >> >> >> not >> > >> >> >> > > >>> > > complicate >> > >> >> >> > > >>> > > > > > > > > > > > merging/rebasing. >> > >> >> >> > > >>> > > > > > > > > > > > > Once this is merged to the >> master >> > >> >> branch, >> > >> >> >> I >> > >> >> >> > > would >> > >> >> >> > > >>> > like >> > >> >> >> > > >>> > > to >> > >> >> >> > > >>> > > > > cut >> > >> >> >> > > >>> > > > > > > 5.2 >> > >> >> >> > > >>> > > > > > > > > > > branch >> > >> >> >> > > >>> > > > > > > > > > > > > from master and we can move >> > forward >> > >> >> with >> > >> >> >> > 5.2.0 >> > >> >> >> > > >>> > release. >> > >> >> >> > > >>> > > > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > > > Please let me know if this >> looks >> > >> good >> > >> >> or >> > >> >> >> if >> > >> >> >> > you >> > >> >> >> > > >>> have >> > >> >> >> > > >>> > > any >> > >> >> >> > > >>> > > > > > other >> > >> >> >> > > >>> > > > > > > > high >> > >> >> >> > > >>> > > > > > > > > > > > > priority work for 5.2.0. >> > >> >> >> > > >>> > > > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > > >> > >> >> >> > > >>> > > > > > > > > >> > >> >> >> > > >>> > > > > > > > >> > >> >> >> > > >>> > > > > > > >> > >> >> >> > > >>> > > > > > >> > >> >> >> > > >>> > > > > >> > >> >> >> > > >>> > > > >> > >> >> >> > > >>> > > >> > >> >> >> > > >>> > >> > >> >> >> > > >>> >> > >> >> >> > > >> >> > >> >> >> > > >> >> > >> >> >> > > >> -- >> > >> >> >> > > >> *István Tóth* | Sr. Staff Software Engineer >> > >> >> >> > > >> *Email*: st...@cloudera.com >> > >> >> >> > > >> cloudera.com <https://www.cloudera.com> >> > >> >> >> > > >> [image: Cloudera] <https://www.cloudera.com/> >> > >> >> >> > > >> [image: Cloudera on Twitter] < >> https://twitter.com/cloudera >> > > >> > >> >> >> [image: >> > >> >> >> > > >> Cloudera on Facebook] < >> https://www.facebook.com/cloudera> >> > >> >> [image: >> > >> >> >> > > >> Cloudera on LinkedIn] < >> > >> >> https://www.linkedin.com/company/cloudera> >> > >> >> >> > > >> ------------------------------ >> > >> >> >> > > >> ------------------------------ >> > >> >> >> > > >> >> > >> >> >> > > > >> > >> >> >> > > >> > >> >> >> > >> > >> >> >> >> > >> >> > >> > >> >> >> > >> > >> > >> >> > > >> > >> >
