Thank you, Viraj, 5.1.4 also has the non-ZK registries feature, and has the same bugs as 5.2 in that code.
On Sat, Feb 10, 2024 at 7:00 PM Viraj Jasani <vjas...@apache.org> wrote: > Sure no worries, we can wait a few more days. > > In the meantime, I have merged the backport PR on 5.1 branch for the data > integrity fixes. Once omid dependency change is in, I believe we are good > to start with 5.1.4. > > Thank you Rajeshbabu for volunteering to take it up. > > > On Sat, Feb 10, 2024 at 1:28 AM Istvan Toth <st...@apache.org> wrote: > >> I appreciate the drive to get 5.2.0 out of the door, but I would prefer to >> have a few more days to fix the registry issues, >> and run some tests on them before cutting the branch, Viraj. >> The non-ZK registry support is one of the bigger new features, and I'd >> prefer not to have known breaking bugs in the release. >> Can we target Friday or the next Monday for the cut ? >> >> Istvan >> >> On Sat, Feb 10, 2024 at 10:07 AM rajeshb...@apache.org < >> chrajeshbab...@gmail.com> wrote: >> >> > Yes Viraj, I can release 5.1.4 >> > >> > Thanks, >> > Rajeshbabu. >> > >> > On Sat, Feb 10, 2024, 10:28 AM Viraj Jasani <vjas...@apache.org> wrote: >> > >> >> I think we can also target 5.2.1 very soon, perhaps just next month, >> with >> >> more CVE fixes and any other fixes if ready. >> >> >> >> >> >> On Fri, Feb 9, 2024 at 6:39 PM Viraj Jasani <vjas...@apache.org> >> wrote: >> >> >> >> > For 5.2.0, it would be great to focus on the known data integrity >> >> issues. >> >> > We can fix non-zk registry, cover a few more CVEs by upgrading third >> >> party >> >> > dependencies and stabilize tests. As for the tests, they don’t seem >> >> broken, >> >> > but are flaky. I have got multiple builds without any test failures >> on >> >> PR >> >> > for PHOENIX-7106. >> >> > >> >> > If this looks good to you, I can start release preparation next week. >> >> What >> >> > do you think, Istvan? >> >> > >> >> > In the meantime, I have 5.1 backport PR open, awaiting good build >> >> results >> >> > before committing it. >> >> > Rajeshbabu, would you like to be RM for 5.1.4 once the PR is merged? >> >> > >> >> > >> >> > >> >> > On Fri, Feb 9, 2024 at 11:13 AM Istvan Toth <st...@apache.org> >> wrote: >> >> > >> >> >> Yes, they basically make the non-ZK registries unusable. >> >> >> (at least the connectionless problems should be fixed.) >> >> >> >> >> >> I hope to have the final fix for those sometime next week. >> >> >> >> >> >> Also have we looked at potential CVE issues on master recently ? >> >> >> >> >> >> I think we should also look at the most flakey tests I linked above, >> >> >> and fix them or at least make sure that they are test issues and not >> >> real >> >> >> bugs. >> >> >> >> >> >> Istvan >> >> >> >> >> >> >> >> >> >> >> >> On Fri, Feb 9, 2024 at 6:55 PM Viraj Jasani <vjas...@apache.org> >> >> wrote: >> >> >> >> >> >> > Thanks Istvan. >> >> >> > I would like to cut 5.2 branch from master. Do you see non-ZK >> >> registry >> >> >> for >> >> >> > MapReduce jobs as blocker for 5.2.0? >> >> >> > >> >> >> > >> >> >> > On Fri, Feb 9, 2024 at 12:24 AM Istvan Toth <st...@apache.org> >> >> wrote: >> >> >> > >> >> >> > > >> ParallelPhoenixConnectionFailureTest.testExecuteQueryChainFailure >> >> also >> >> >> > > fails too often, especially when the test host is slow and/or >> the >> >> >> load is >> >> >> > > high. >> >> >> > > On my fast laptop, I can semi-reliably break it by running >> >> >> > > mvn clean verify -am -pl phoenix-core -DnumForkedUT=20 >> >> >> > > >> >> >> > > On Fri, Feb 9, 2024 at 9:19 AM Istvan Toth <st...@apache.org> >> >> wrote: >> >> >> > > >> >> >> > > > We're making progress. >> >> >> > > > I can see that Viraj has just landed PHOENIX-7601, and >> Rajeshbabu >> >> >> has >> >> >> > > > released Omid 1.1.1. >> >> >> > > > Thank you! >> >> >> > > > >> >> >> > > > At the moment, the following outstanding issues are on my >> radar: >> >> >> > > > >> >> >> > > > https://issues.apache.org/jira/browse/PHOENIX-7191 >> >> >> > > > https://issues.apache.org/jira/browse/PHOENIX-7193 >> >> >> > > > >> >> >> > > > These are bugs in my non-ZK registry implementation, which >> were >> >> >> found >> >> >> > > > during HBase 3 work. >> >> >> > > > I have some PRs up, but they may not be complete. I will push >> for >> >> >> > reviews >> >> >> > > > once I have the HBase 3 tests passing, and possibly updated >> them >> >> >> based >> >> >> > on >> >> >> > > > that. >> >> >> > > > >> >> >> > > > We also have a number of very flakey tests, see: >> >> >> > > > >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/test_results_analyzer/ >> >> >> > > > >> >> >> > > > >> >> >> > > > >> >> >> > > > On Tue, Jan 30, 2024 at 7:09 AM Istvan Toth < >> st...@cloudera.com> >> >> >> > wrote: >> >> >> > > > >> >> >> > > >> As Viraj wrote, those are just plans. >> >> >> > > >> If HBase 3 won't be released by the time the other features >> are >> >> >> ready, >> >> >> > > >> then it won't make it into 5.3. >> >> >> > > >> If other major features are ready by that time, then they >> will >> >> be >> >> >> > > >> included. (though we are not aware of any now) >> >> >> > > >> >> >> >> > > >> As for the new major version, in the past Phoenix didn't >> have a >> >> >> > > >> compatibility module system, >> >> >> > > >> so a new branch was required, which didn't support older >> >> HBases. >> >> >> > Also, >> >> >> > > >> the API changes between HBase 1.x and 2.x were much larger, >> >> >> > > >> The HBase 2 and 3 API are pretty similar, apart from the >> >> removal of >> >> >> > > >> deprecated 1.x APIs. (and the protobuf/protocol thing, which >> >> >> requires >> >> >> > a >> >> >> > > >> rather ugly hack). >> >> >> > > >> >> >> >> > > >> I will start the discussion on how we can add HBase 3 >> support as >> >> >> soon >> >> >> > as >> >> >> > > >> I have a working POC patch. >> >> >> > > >> >> >> >> > > >> We could call 5.3 6.0 instead, after all Phoenix isn't using >> a >> >> >> strict >> >> >> > > >> semantic versioning, but then 6.0 would also support HBase 2. >> >> >> > > >> If we do not come to a consensus on the version name, we can >> >> always >> >> >> > have >> >> >> > > >> a vote on it. >> >> >> > > >> >> >> >> > > >> I think that the main motivation is that the community wants >> to >> >> >> > maintain >> >> >> > > >> as few branches as possible. >> >> >> > > >> >> >> >> > > >> Istvan >> >> >> > > >> >> >> >> > > >> On Mon, Jan 29, 2024 at 10:02 PM Stephen Jiang < >> >> >> > syuanjiang...@gmail.com >> >> >> > > > >> >> >> > > >> wrote: >> >> >> > > >> >> >> >> > > >>> I am not sure how close HBase 3.0 is. Even if it is only >> less >> >> >> than >> >> >> > one >> >> >> > > >>> year away, the adoption would be low at the beginning. I >> don't >> >> >> think >> >> >> > > 5.3 >> >> >> > > >>> should wait for that. And traditionally, Phoenix would >> have a >> >> >> major >> >> >> > > >>> release to support the HBase major release (4.x for HBase >> 1.x >> >> and >> >> >> 5.x >> >> >> > > for >> >> >> > > >>> HBase 2.x), in this case, we are talking about Phoenix 6.0 >> for >> >> >> HBase >> >> >> > > 3.0. >> >> >> > > >>> >> >> >> > > >>> Maybe we should adopt the HBase release model: master branch >> >> for >> >> >> next >> >> >> > > >>> major >> >> >> > > >>> release (6.0) and branch-5.x branch for next 5.x minor >> release >> >> and >> >> >> > > >>> branch-5.2 for 5.2 minor release. >> >> >> > > >>> >> >> >> > > >>> Thanks >> >> >> > > >>> Stephen >> >> >> > > >>> >> >> >> > > >>> >> >> >> > > >>> >> >> >> > > >>> On Thu, Jan 25, 2024 at 11:50 PM Viraj Jasani < >> >> vjas...@apache.org >> >> >> > >> >> >> > > >>> wrote: >> >> >> > > >>> >> >> >> > > >>> > Sounds good. >> >> >> > > >>> > >> >> >> > > >>> > Planned major changes for 5.3.0: >> >> >> > > >>> > >> >> >> > > >>> > 1. JSON support. >> >> >> > > >>> > 2. HBase 3.0 support. >> >> >> > > >>> > 3. CDC feature (leveraging uncovered global index >> framework >> >> and >> >> >> > JSON >> >> >> > > >>> > support). >> >> >> > > >>> > >> >> >> > > >>> > >> >> >> > > >>> > On Wed, Jan 24, 2024 at 8:22 PM Kadir Ozdemir >> >> >> > > >>> > <kozde...@salesforce.com.invalid> wrote: >> >> >> > > >>> > >> >> >> > > >>> > > I suggest including another major change for 5.3, >> Phoenix >> >> CDC, >> >> >> > > >>> > > https://issues.apache.org/jira/browse/PHOENIX-7001. >> The PR >> >> >> for >> >> >> > it >> >> >> > > >>> will >> >> >> > > >>> > be >> >> >> > > >>> > > posted soon. >> >> >> > > >>> > > >> >> >> > > >>> > > On Thu, Jan 25, 2024 at 5:35 AM Viraj Jasani < >> >> >> vjas...@apache.org >> >> >> > > >> >> >> > > >>> wrote: >> >> >> > > >>> > > >> >> >> > > >>> > > > Thanks Istvan! I agree with your points. It’s really >> >> been a >> >> >> > while >> >> >> > > >>> we >> >> >> > > >>> > are >> >> >> > > >>> > > > talking about releasing 5.2.0 and yet due to bandwidth >> >> >> issues, >> >> >> > > >>> unable >> >> >> > > >>> > to >> >> >> > > >>> > > do >> >> >> > > >>> > > > so. >> >> >> > > >>> > > > >> >> >> > > >>> > > > I agree to getting these fixes out, cut 5.2 and start >> the >> >> >> > release >> >> >> > > >>> work >> >> >> > > >>> > > and >> >> >> > > >>> > > > in the meantime I also need to prepare 5.1 backport. >> >> >> > > >>> > > > >> >> >> > > >>> > > > For 5.3, let's plan HBase 3.0 support and JSON as >> major >> >> >> > changes. >> >> >> > > >>> > > > >> >> >> > > >>> > > > >> >> >> > > >>> > > > On Mon, Jan 22, 2024 at 9:17 PM Istvan Toth < >> >> >> st...@apache.org> >> >> >> > > >>> wrote: >> >> >> > > >>> > > > >> >> >> > > >>> > > > > Hi! >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > In my opinion cutting 5.2 now only makes sense IF >> we DO >> >> >> NOT >> >> >> > > plan >> >> >> > > >>> to >> >> >> > > >>> > > > release >> >> >> > > >>> > > > > the outstanding big features (like JSON) in 5.2. , >> >> >> otherwise >> >> >> > > it's >> >> >> > > >>> > just >> >> >> > > >>> > > > > extra work to maintain more branches. >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > Having said that, releasing a 5.2 and 5.1.4 with the >> >> data >> >> >> > > >>> integrity >> >> >> > > >>> > > fixes >> >> >> > > >>> > > > > real soon, and then releasing 5.3 in a few months >> with >> >> >> JSON, >> >> >> > > and >> >> >> > > >>> any >> >> >> > > >>> > > > other >> >> >> > > >>> > > > > outstanding big features >> >> >> > > >>> > > > > that are close to being finished (and HBase 3.0 >> >> support, >> >> >> if >> >> >> > > it's >> >> >> > > >>> > ready >> >> >> > > >>> > > by >> >> >> > > >>> > > > > then) would not be a bad idea. >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > On the CLDR side the only outstanding big feature >> which >> >> >> could >> >> >> > > >>> impact >> >> >> > > >>> > > > > Viraj's integrity work is HBase 3.0 support, and >> even >> >> >> that is >> >> >> > > >>> only >> >> >> > > >>> > > > because >> >> >> > > >>> > > > > it may require some larger refactors of existing >> code, >> >> not >> >> >> > > >>> because it >> >> >> > > >>> > > > would >> >> >> > > >>> > > > > change the actual behaviour or algorithms. >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > Phoenix used to have several minor releases per >> year, >> >> the >> >> >> > > current >> >> >> > > >>> > state >> >> >> > > >>> > > > of >> >> >> > > >>> > > > > extreme longevity of 5.1 and several big new >> features >> >> >> being >> >> >> > > >>> added to >> >> >> > > >>> > it >> >> >> > > >>> > > > > (like uncovered indexes) is not ideal. >> >> >> > > >>> > > > > Releasing 5.2 and 5.3 relatively close together >> could >> >> be a >> >> >> > > >>> return to >> >> >> > > >>> > a >> >> >> > > >>> > > > > quicker cadence for minor releases, which could also >> >> help >> >> >> > with >> >> >> > > >>> the >> >> >> > > >>> > > public >> >> >> > > >>> > > > > image and adoption of Phoenix. >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > We were talking about releasing 5.2 at least a year >> >> ago, >> >> >> and >> >> >> > I >> >> >> > > >>> have >> >> >> > > >>> > > > started >> >> >> > > >>> > > > > working on that then, but then emergencies have come >> >> up at >> >> >> > > >>> $dayjob, >> >> >> > > >>> > > and I >> >> >> > > >>> > > > > could not see that through. >> >> >> > > >>> > > > > (So I am in part responsible for the lack of minor >> >> >> releases) >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > regards >> >> >> > > >>> > > > > Istvan >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > On Mon, Jan 22, 2024 at 11:35 PM Viraj Jasani < >> >> >> > > >>> vjas...@apache.org> >> >> >> > > >>> > > > wrote: >> >> >> > > >>> > > > > >> >> >> > > >>> > > > > > Sorry for the late reply. >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > > No problem with that. I can cut 5.2 branch by the >> >> end of >> >> >> > this >> >> >> > > >>> week >> >> >> > > >>> > or >> >> >> > > >>> > > > at >> >> >> > > >>> > > > > > the start of next week. >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > > If there is any very big change or feature ready >> for >> >> >> merge >> >> >> > to >> >> >> > > >>> > master >> >> >> > > >>> > > > > branch >> >> >> > > >>> > > > > > with PR approvals already in place, please do let >> me >> >> >> know >> >> >> > so >> >> >> > > >>> that I >> >> >> > > >>> > > can >> >> >> > > >>> > > > > > help collaborate on how best we can get it merged >> >> >> without >> >> >> > > >>> impacting >> >> >> > > >>> > > 5.2 >> >> >> > > >>> > > > > > release if required. My main motivation was for >> any >> >> big >> >> >> > > change >> >> >> > > >>> to >> >> >> > > >>> > go >> >> >> > > >>> > > > > > through newly introduced tests so that we know >> that >> >> >> > anything >> >> >> > > >>> > > additional >> >> >> > > >>> > > > > is >> >> >> > > >>> > > > > > not broken, and also to prioritize for upcoming >> 5.2.0 >> >> >> and >> >> >> > > 5.1.4 >> >> >> > > >>> > > > releases. >> >> >> > > >>> > > > > > Moreover, there are several PRs getting merged on >> the >> >> >> > master >> >> >> > > >>> > branch, >> >> >> > > >>> > > we >> >> >> > > >>> > > > > can >> >> >> > > >>> > > > > > continue that as long as they are not very big >> >> changes, >> >> >> > which >> >> >> > > >>> might >> >> >> > > >>> > > > > require >> >> >> > > >>> > > > > > significant time to understand any correlation >> with >> >> data >> >> >> > > >>> integrity >> >> >> > > >>> > > > > issues. >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > > The PR is also ready for review with some >> additional >> >> >> cases >> >> >> > > >>> fixed >> >> >> > > >>> > last >> >> >> > > >>> > > > > week: >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > >> >> >> > > >>> > > >> >> >> > > >>> > >> >> >> > > >>> >> >> >> > > >> >> >> > >> >> >> >> >> >> https://urldefense.com/v3/__https://github.com/apache/phoenix/pull/1736__;!!DCbAVzZNrAf4!D4OVjUp2EWW2BqhGnBxsapDX_AHsibRphIpoFBWfgRsd3dsAikrFLo6PGxdTzGbSXJJ2fJ0j9mcz3asXMXo$ >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > > Depending on the review bandwidth, I am hopeful we >> >> >> should >> >> >> > be >> >> >> > > >>> good >> >> >> > > >>> > to >> >> >> > > >>> > > > land >> >> >> > > >>> > > > > > them sooner. >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > > On Wed, Jan 17, 2024 at 11:31 AM Rushabh Shah >> >> >> > > >>> > > > > > <rushabh.s...@salesforce.com.invalid> wrote: >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > > > Thank you Viraj for initiating this thread. >> >> >> > > >>> > > > > > > >> >> >> > > >>> > > > > > > > Given the critical nature of these issues, I >> >> would >> >> >> like >> >> >> > > to >> >> >> > > >>> > > propose >> >> >> > > >>> > > > > that >> >> >> > > >>> > > > > > > we >> >> >> > > >>> > > > > > > treat this as a high priority for the upcoming >> >> 5.2.0 >> >> >> > > >>> release, and >> >> >> > > >>> > > not >> >> >> > > >>> > > > > > > include any other feature or big change to >> master >> >> >> branch >> >> >> > > >>> until we >> >> >> > > >>> > > > merge >> >> >> > > >>> > > > > > > this. >> >> >> > > >>> > > > > > > >> >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? This >> >> will >> >> >> > > enable >> >> >> > > >>> > other >> >> >> > > >>> > > > > > > developers to merge features into master branch >> >> >> (5.3.0) >> >> >> > and >> >> >> > > >>> you >> >> >> > > >>> > can >> >> >> > > >>> > > > > take >> >> >> > > >>> > > > > > > some more time to make sure we cover all the >> corner >> >> >> cases >> >> >> > > >>> for the >> >> >> > > >>> > > > data >> >> >> > > >>> > > > > > > integrity issues that you uncovered. >> >> >> > > >>> > > > > > > >> >> >> > > >>> > > > > > > >> >> >> > > >>> > > > > > > On Fri, Jan 5, 2024 at 6:38 PM Viraj Jasani < >> >> >> > > >>> vjas...@apache.org> >> >> >> > > >>> > > > > wrote: >> >> >> > > >>> > > > > > > >> >> >> > > >>> > > > > > > > Sounds good, thanks Rajeshbabu. I will try to >> get >> >> >> the >> >> >> > > >>> first PR >> >> >> > > >>> > > out >> >> >> > > >>> > > > > next >> >> >> > > >>> > > > > > > > week and while reviews happen in parallel, >> will >> >> try >> >> >> to >> >> >> > > get >> >> >> > > >>> 5.1 >> >> >> > > >>> > PR >> >> >> > > >>> > > > > soon. >> >> >> > > >>> > > > > > > > >> >> >> > > >>> > > > > > > > >> >> >> > > >>> > > > > > > > On Wed, Jan 3, 2024 at 8:49 PM >> >> >> rajeshb...@apache.org < >> >> >> > > >>> > > > > > > > chrajeshbab...@gmail.com> wrote: >> >> >> > > >>> > > > > > > > >> >> >> > > >>> > > > > > > > > Hi Viraj, >> >> >> > > >>> > > > > > > > > >> >> >> > > >>> > > > > > > > > Would be better to include the changes in >> 5.1.4 >> >> >> as >> >> >> > in >> >> >> > > >>> any >> >> >> > > >>> > way >> >> >> > > >>> > > it >> >> >> > > >>> > > > > > will >> >> >> > > >>> > > > > > > > take >> >> >> > > >>> > > > > > > > > at least 3-4 days to complete the omid >> release. >> >> >> > > >>> > > > > > > > > >> >> >> > > >>> > > > > > > > > Thanks, >> >> >> > > >>> > > > > > > > > Rajeshbabu. >> >> >> > > >>> > > > > > > > > >> >> >> > > >>> > > > > > > > > >> >> >> > > >>> > > > > > > > > On Thu, Jan 4, 2024 at 5:06 AM Viraj Jasani >> < >> >> >> > > >>> > > vjas...@apache.org> >> >> >> > > >>> > > > > > > wrote: >> >> >> > > >>> > > > > > > > > >> >> >> > > >>> > > > > > > > > > Thank you Kadir and Geoffrey for your >> >> replies!! >> >> >> > > >>> > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, which is >> also >> >> >> listed >> >> >> > as >> >> >> > > >>> a Fix >> >> >> > > >>> > > > > Version >> >> >> > > >>> > > > > > > for >> >> >> > > >>> > > > > > > > > > > PHOENIX-7106? >> >> >> > > >>> > > > > > > > > > >> >> >> > > >>> > > > > > > > > > Yes, it also needs to be ported to 5.1. >> Once >> >> the >> >> >> > > >>> master PR >> >> >> > > >>> > is >> >> >> > > >>> > > > up >> >> >> > > >>> > > > > > for >> >> >> > > >>> > > > > > > > > final >> >> >> > > >>> > > > > > > > > > review, I would start working on the >> backport >> >> >> PR. >> >> >> > > >>> > > > > > > > > > We just need some more additional testing >> to >> >> >> ensure >> >> >> > > old >> >> >> > > >>> > > client >> >> >> > > >>> > > > > > (e.g. >> >> >> > > >>> > > > > > > > > 5.1.3) >> >> >> > > >>> > > > > > > > > > is compatible with the new server with the >> >> >> changes. >> >> >> > > >>> > > > > > > > > > >> >> >> > > >>> > > > > > > > > > Hence, yes it is now a blocker for >> upcoming >> >> >> 5.1.4 >> >> >> > as >> >> >> > > >>> well >> >> >> > > >>> > > since >> >> >> > > >>> > > > > > 5.1.4 >> >> >> > > >>> > > > > > > > RC >> >> >> > > >>> > > > > > > > > > preparation is still pending (while Omid >> >> >> release is >> >> >> > > in >> >> >> > > >>> > > > progress). >> >> >> > > >>> > > > > > > > > > Otherwise if 5.1.4 was ready for release, >> I >> >> >> would >> >> >> > > have >> >> >> > > >>> > > proposed >> >> >> > > >>> > > > > > > > immediate >> >> >> > > >>> > > > > > > > > > 5.1.5 release to include the changes >> proposed >> >> >> with >> >> >> > > >>> > > > PHOENIX-7106. >> >> >> > > >>> > > > > > > > > > >> >> >> > > >>> > > > > > > > > > >> >> >> > > >>> > > > > > > > > > On Wed, Jan 3, 2024 at 3:08 PM Geoffrey >> >> Jacoby < >> >> >> > > >>> > > > > gjac...@apache.org >> >> >> > > >>> > > > > > > >> >> >> > > >>> > > > > > > > > wrote: >> >> >> > > >>> > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > I agree that data integrity issues are a >> >> >> higher >> >> >> > > >>> priority >> >> >> > > >>> > > than >> >> >> > > >>> > > > > > > feature >> >> >> > > >>> > > > > > > > > > > development, so I also support the >> >> decision. >> >> >> The >> >> >> > > fact >> >> >> > > >>> > that >> >> >> > > >>> > > > > > several >> >> >> > > >>> > > > > > > of >> >> >> > > >>> > > > > > > > > the >> >> >> > > >>> > > > > > > > > > > major remaining 5.2 features are >> currently >> >> >> being >> >> >> > > >>> > developed >> >> >> > > >>> > > in >> >> >> > > >>> > > > > > > > > > long-running >> >> >> > > >>> > > > > > > > > > > feature branches also helps, as work can >> >> >> continue >> >> >> > > >>> there >> >> >> > > >>> > at >> >> >> > > >>> > > > the >> >> >> > > >>> > > > > > cost >> >> >> > > >>> > > > > > > > of >> >> >> > > >>> > > > > > > > > a >> >> >> > > >>> > > > > > > > > > > rebase later. >> >> >> > > >>> > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, which is >> also >> >> >> listed >> >> >> > as >> >> >> > > >>> a Fix >> >> >> > > >>> > > > > Version >> >> >> > > >>> > > > > > > for >> >> >> > > >>> > > > > > > > > > > PHOENIX-7106? From the bug description >> it >> >> also >> >> >> > > sounds >> >> >> > > >>> > like >> >> >> > > >>> > > > > 5.1.3 >> >> >> > > >>> > > > > > > and >> >> >> > > >>> > > > > > > > > the >> >> >> > > >>> > > > > > > > > > > forthcoming .4 are affected, since we >> have >> >> >> > > >>> server-side >> >> >> > > >>> > > paging >> >> >> > > >>> > > > > in >> >> >> > > >>> > > > > > > 5.1. >> >> >> > > >>> > > > > > > > > > (Feel >> >> >> > > >>> > > > > > > > > > > free to move that to a separate thread >> if >> >> you >> >> >> > feel >> >> >> > > it >> >> >> > > >>> > > should >> >> >> > > >>> > > > > be a >> >> >> > > >>> > > > > > > > > > separate >> >> >> > > >>> > > > > > > > > > > discussion.) Should this be a blocker >> for >> >> >> > releasing >> >> >> > > >>> > 5.1.4? >> >> >> > > >>> > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > Geoffrey >> >> >> > > >>> > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > On Wed, Jan 3, 2024 at 5:06 PM Kadir >> >> Ozdemir < >> >> >> > > >>> > > > > > > > > > > ka...@gsuite.cloud.apache.org> >> >> >> > > >>> > > > > > > > > > > wrote: >> >> >> > > >>> > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > > Being a database, Phoenix has to make >> >> sure >> >> >> that >> >> >> > > the >> >> >> > > >>> > data >> >> >> > > >>> > > > > stays >> >> >> > > >>> > > > > > on >> >> >> > > >>> > > > > > > > > disk >> >> >> > > >>> > > > > > > > > > > > intact and its queries return correct >> >> data. >> >> >> In >> >> >> > > this >> >> >> > > >>> > case, >> >> >> > > >>> > > > > > Phoenix >> >> >> > > >>> > > > > > > > > fails >> >> >> > > >>> > > > > > > > > > > to >> >> >> > > >>> > > > > > > > > > > > return correct data for some queries >> if >> >> >> their >> >> >> > > scans >> >> >> > > >>> > > > > experience >> >> >> > > >>> > > > > > > > region >> >> >> > > >>> > > > > > > > > > > > movement. Now that we know these data >> >> >> integrity >> >> >> > > >>> issues >> >> >> > > >>> > > and >> >> >> > > >>> > > > > how >> >> >> > > >>> > > > > > to >> >> >> > > >>> > > > > > > > > > > reproduce >> >> >> > > >>> > > > > > > > > > > > them, fixing them should be our first >> >> >> priority. >> >> >> > > >>> So, I >> >> >> > > >>> > > fully >> >> >> > > >>> > > > > > > support >> >> >> > > >>> > > > > > > > > > this >> >> >> > > >>> > > > > > > > > > > > proposal. >> >> >> > > >>> > > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > > On Wed, Jan 3, 2024 at 10:58 PM Viraj >> >> >> Jasani < >> >> >> > > >>> > > > > > vjas...@apache.org >> >> >> > > >>> > > > > > > > >> >> >> > > >>> > > > > > > > > > wrote: >> >> >> > > >>> > > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > > > Hello, >> >> >> > > >>> > > > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > > > I would like to bring PHOENIX-7106 >> >> >> > > >>> > > > > > > > > > > > > < >> >> >> > > >>> > > > > > > > >> >> >> > > >>> > > > > > > >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > >> >> >> > > >>> > > > >> >> >> > > >>> > > >> >> >> > > >>> > >> >> >> > > >>> >> >> >> > > >> >> >> > >> >> >> >> >> >> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/PHOENIX-7106__;!!DCbAVzZNrAf4!FZG5sv55IC1NqItQLY7GKWgUG2Do0gSta01gOiSdd36Dx3XHGtQx4M3c9visVXIt9DctPQzS-orob9vhzrCfVA$ >> >> >> > > >>> > > > > > > > > to everyone's >> >> >> > > >>> > > > > > > > > > > > > attention here and brief about the >> data >> >> >> > > integrity >> >> >> > > >>> > > issues >> >> >> > > >>> > > > > that >> >> >> > > >>> > > > > > > we >> >> >> > > >>> > > > > > > > > have >> >> >> > > >>> > > > > > > > > > > in >> >> >> > > >>> > > > > > > > > > > > > various coprocessors. Majority of >> the >> >> >> issues >> >> >> > > are >> >> >> > > >>> > > related >> >> >> > > >>> > > > to >> >> >> > > >>> > > > > > the >> >> >> > > >>> > > > > > > > > fact >> >> >> > > >>> > > > > > > > > > > that >> >> >> > > >>> > > > > > > > > > > > > we do not return valid rowkey for >> >> certain >> >> >> > > >>> queries. If >> >> >> > > >>> > > any >> >> >> > > >>> > > > > > > region >> >> >> > > >>> > > > > > > > > > moves >> >> >> > > >>> > > > > > > > > > > in >> >> >> > > >>> > > > > > > > > > > > > the middle of the scan, the HBase >> >> client >> >> >> > relies >> >> >> > > >>> on >> >> >> > > >>> > the >> >> >> > > >>> > > > last >> >> >> > > >>> > > > > > > > > returned >> >> >> > > >>> > > > > > > > > > > > rowkey >> >> >> > > >>> > > > > > > > > > > > > and accordingly changes the scan >> >> >> boundaries >> >> >> > > >>> while the >> >> >> > > >>> > > > > scanner >> >> >> > > >>> > > > > > > is >> >> >> > > >>> > > > > > > > > > > getting >> >> >> > > >>> > > > > > > > > > > > > reset to continue the scan >> operation. >> >> If >> >> >> the >> >> >> > > >>> region >> >> >> > > >>> > > does >> >> >> > > >>> > > > > not >> >> >> > > >>> > > > > > > > move, >> >> >> > > >>> > > > > > > > > > scan >> >> >> > > >>> > > > > > > > > > > > is >> >> >> > > >>> > > > > > > > > > > > > not expected to return invalid data, >> >> >> however >> >> >> > if >> >> >> > > >>> the >> >> >> > > >>> > > > region >> >> >> > > >>> > > > > > > moves >> >> >> > > >>> > > > > > > > in >> >> >> > > >>> > > > > > > > > > the >> >> >> > > >>> > > > > > > > > > > > > middle of ongoing scan operation, >> scan >> >> >> would >> >> >> > > >>> return >> >> >> > > >>> > > > > > > > > invalid/incorrect >> >> >> > > >>> > > > > > > > > > > > data >> >> >> > > >>> > > > > > > > > > > > > causing data integrity issues. >> >> >> > > >>> > > > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > > > Given the critical nature of these >> >> >> issues, I >> >> >> > > >>> would >> >> >> > > >>> > like >> >> >> > > >>> > > > to >> >> >> > > >>> > > > > > > > propose >> >> >> > > >>> > > > > > > > > > that >> >> >> > > >>> > > > > > > > > > > > we >> >> >> > > >>> > > > > > > > > > > > > treat this as a high priority for >> the >> >> >> > upcoming >> >> >> > > >>> 5.2.0 >> >> >> > > >>> > > > > release, >> >> >> > > >>> > > > > > > and >> >> >> > > >>> > > > > > > > > not >> >> >> > > >>> > > > > > > > > > > > > include any other feature or big >> >> change to >> >> >> > > master >> >> >> > > >>> > > branch >> >> >> > > >>> > > > > > until >> >> >> > > >>> > > > > > > we >> >> >> > > >>> > > > > > > > > > merge >> >> >> > > >>> > > > > > > > > > > > > this. The PR is still not ready as >> >> >> additional >> >> >> > > >>> changes >> >> >> > > >>> > > are >> >> >> > > >>> > > > > > still >> >> >> > > >>> > > > > > > > in >> >> >> > > >>> > > > > > > > > my >> >> >> > > >>> > > > > > > > > > > > > local, requiring rebase with the >> >> current >> >> >> > > master. >> >> >> > > >>> > > > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > > > I would get back to this discuss >> >> thread as >> >> >> > soon >> >> >> > > >>> as >> >> >> > > >>> > the >> >> >> > > >>> > > PR >> >> >> > > >>> > > > > and >> >> >> > > >>> > > > > > > the >> >> >> > > >>> > > > > > > > > doc >> >> >> > > >>> > > > > > > > > > > are >> >> >> > > >>> > > > > > > > > > > > > updated with the latest findings so >> >> far. >> >> >> The >> >> >> > > >>> changes >> >> >> > > >>> > > > > include >> >> >> > > >>> > > > > > > many >> >> >> > > >>> > > > > > > > > of >> >> >> > > >>> > > > > > > > > > > our >> >> >> > > >>> > > > > > > > > > > > > coproc scanner implementations and >> >> hence >> >> >> it >> >> >> > > would >> >> >> > > >>> > > require >> >> >> > > >>> > > > > > > > > significant >> >> >> > > >>> > > > > > > > > > > > > review as well. >> >> >> > > >>> > > > > > > > > > > > > It would be great if we can hold on >> to >> >> >> > merging >> >> >> > > >>> any >> >> >> > > >>> > > > feature >> >> >> > > >>> > > > > or >> >> >> > > >>> > > > > > > big >> >> >> > > >>> > > > > > > > > > > change >> >> >> > > >>> > > > > > > > > > > > to >> >> >> > > >>> > > > > > > > > > > > > master branch until this gets in so >> as >> >> to >> >> >> not >> >> >> > > >>> > > complicate >> >> >> > > >>> > > > > > > > > > > > merging/rebasing. >> >> >> > > >>> > > > > > > > > > > > > Once this is merged to the master >> >> branch, >> >> >> I >> >> >> > > would >> >> >> > > >>> > like >> >> >> > > >>> > > to >> >> >> > > >>> > > > > cut >> >> >> > > >>> > > > > > > 5.2 >> >> >> > > >>> > > > > > > > > > > branch >> >> >> > > >>> > > > > > > > > > > > > from master and we can move forward >> >> with >> >> >> > 5.2.0 >> >> >> > > >>> > release. >> >> >> > > >>> > > > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > > > Please let me know if this looks >> good >> >> or >> >> >> if >> >> >> > you >> >> >> > > >>> have >> >> >> > > >>> > > any >> >> >> > > >>> > > > > > other >> >> >> > > >>> > > > > > > > high >> >> >> > > >>> > > > > > > > > > > > > priority work for 5.2.0. >> >> >> > > >>> > > > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > > >> >> >> > > >>> > > > > > > > > > >> >> >> > > >>> > > > > > > > > >> >> >> > > >>> > > > > > > > >> >> >> > > >>> > > > > > > >> >> >> > > >>> > > > > > >> >> >> > > >>> > > > > >> >> >> > > >>> > > > >> >> >> > > >>> > > >> >> >> > > >>> > >> >> >> > > >>> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> -- >> >> >> > > >> *István Tóth* | Sr. Staff Software Engineer >> >> >> > > >> *Email*: st...@cloudera.com >> >> >> > > >> cloudera.com <https://www.cloudera.com> >> >> >> > > >> [image: Cloudera] <https://www.cloudera.com/> >> >> >> > > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> >> >> >> [image: >> >> >> > > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> >> >> [image: >> >> >> > > >> Cloudera on LinkedIn] < >> >> https://www.linkedin.com/company/cloudera> >> >> >> > > >> ------------------------------ >> >> >> > > >> ------------------------------ >> >> >> > > >> >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> > >> >> >> > >> >
