Sure no worries, we can wait a few more days. In the meantime, I have merged the backport PR on 5.1 branch for the data integrity fixes. Once omid dependency change is in, I believe we are good to start with 5.1.4.
Thank you Rajeshbabu for volunteering to take it up. On Sat, Feb 10, 2024 at 1:28 AM Istvan Toth <st...@apache.org> wrote: > I appreciate the drive to get 5.2.0 out of the door, but I would prefer to > have a few more days to fix the registry issues, > and run some tests on them before cutting the branch, Viraj. > The non-ZK registry support is one of the bigger new features, and I'd > prefer not to have known breaking bugs in the release. > Can we target Friday or the next Monday for the cut ? > > Istvan > > On Sat, Feb 10, 2024 at 10:07 AM rajeshb...@apache.org < > chrajeshbab...@gmail.com> wrote: > > > Yes Viraj, I can release 5.1.4 > > > > Thanks, > > Rajeshbabu. > > > > On Sat, Feb 10, 2024, 10:28 AM Viraj Jasani <vjas...@apache.org> wrote: > > > >> I think we can also target 5.2.1 very soon, perhaps just next month, > with > >> more CVE fixes and any other fixes if ready. > >> > >> > >> On Fri, Feb 9, 2024 at 6:39 PM Viraj Jasani <vjas...@apache.org> wrote: > >> > >> > For 5.2.0, it would be great to focus on the known data integrity > >> issues. > >> > We can fix non-zk registry, cover a few more CVEs by upgrading third > >> party > >> > dependencies and stabilize tests. As for the tests, they don’t seem > >> broken, > >> > but are flaky. I have got multiple builds without any test failures on > >> PR > >> > for PHOENIX-7106. > >> > > >> > If this looks good to you, I can start release preparation next week. > >> What > >> > do you think, Istvan? > >> > > >> > In the meantime, I have 5.1 backport PR open, awaiting good build > >> results > >> > before committing it. > >> > Rajeshbabu, would you like to be RM for 5.1.4 once the PR is merged? > >> > > >> > > >> > > >> > On Fri, Feb 9, 2024 at 11:13 AM Istvan Toth <st...@apache.org> wrote: > >> > > >> >> Yes, they basically make the non-ZK registries unusable. > >> >> (at least the connectionless problems should be fixed.) > >> >> > >> >> I hope to have the final fix for those sometime next week. > >> >> > >> >> Also have we looked at potential CVE issues on master recently ? > >> >> > >> >> I think we should also look at the most flakey tests I linked above, > >> >> and fix them or at least make sure that they are test issues and not > >> real > >> >> bugs. > >> >> > >> >> Istvan > >> >> > >> >> > >> >> > >> >> On Fri, Feb 9, 2024 at 6:55 PM Viraj Jasani <vjas...@apache.org> > >> wrote: > >> >> > >> >> > Thanks Istvan. > >> >> > I would like to cut 5.2 branch from master. Do you see non-ZK > >> registry > >> >> for > >> >> > MapReduce jobs as blocker for 5.2.0? > >> >> > > >> >> > > >> >> > On Fri, Feb 9, 2024 at 12:24 AM Istvan Toth <st...@apache.org> > >> wrote: > >> >> > > >> >> > > ParallelPhoenixConnectionFailureTest.testExecuteQueryChainFailure > >> also > >> >> > > fails too often, especially when the test host is slow and/or the > >> >> load is > >> >> > > high. > >> >> > > On my fast laptop, I can semi-reliably break it by running > >> >> > > mvn clean verify -am -pl phoenix-core -DnumForkedUT=20 > >> >> > > > >> >> > > On Fri, Feb 9, 2024 at 9:19 AM Istvan Toth <st...@apache.org> > >> wrote: > >> >> > > > >> >> > > > We're making progress. > >> >> > > > I can see that Viraj has just landed PHOENIX-7601, and > Rajeshbabu > >> >> has > >> >> > > > released Omid 1.1.1. > >> >> > > > Thank you! > >> >> > > > > >> >> > > > At the moment, the following outstanding issues are on my > radar: > >> >> > > > > >> >> > > > https://issues.apache.org/jira/browse/PHOENIX-7191 > >> >> > > > https://issues.apache.org/jira/browse/PHOENIX-7193 > >> >> > > > > >> >> > > > These are bugs in my non-ZK registry implementation, which were > >> >> found > >> >> > > > during HBase 3 work. > >> >> > > > I have some PRs up, but they may not be complete. I will push > for > >> >> > reviews > >> >> > > > once I have the HBase 3 tests passing, and possibly updated > them > >> >> based > >> >> > on > >> >> > > > that. > >> >> > > > > >> >> > > > We also have a number of very flakey tests, see: > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/test_results_analyzer/ > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > On Tue, Jan 30, 2024 at 7:09 AM Istvan Toth < > st...@cloudera.com> > >> >> > wrote: > >> >> > > > > >> >> > > >> As Viraj wrote, those are just plans. > >> >> > > >> If HBase 3 won't be released by the time the other features > are > >> >> ready, > >> >> > > >> then it won't make it into 5.3. > >> >> > > >> If other major features are ready by that time, then they will > >> be > >> >> > > >> included. (though we are not aware of any now) > >> >> > > >> > >> >> > > >> As for the new major version, in the past Phoenix didn't have > a > >> >> > > >> compatibility module system, > >> >> > > >> so a new branch was required, which didn't support older > >> HBases. > >> >> > Also, > >> >> > > >> the API changes between HBase 1.x and 2.x were much larger, > >> >> > > >> The HBase 2 and 3 API are pretty similar, apart from the > >> removal of > >> >> > > >> deprecated 1.x APIs. (and the protobuf/protocol thing, which > >> >> requires > >> >> > a > >> >> > > >> rather ugly hack). > >> >> > > >> > >> >> > > >> I will start the discussion on how we can add HBase 3 support > as > >> >> soon > >> >> > as > >> >> > > >> I have a working POC patch. > >> >> > > >> > >> >> > > >> We could call 5.3 6.0 instead, after all Phoenix isn't using a > >> >> strict > >> >> > > >> semantic versioning, but then 6.0 would also support HBase 2. > >> >> > > >> If we do not come to a consensus on the version name, we can > >> always > >> >> > have > >> >> > > >> a vote on it. > >> >> > > >> > >> >> > > >> I think that the main motivation is that the community wants > to > >> >> > maintain > >> >> > > >> as few branches as possible. > >> >> > > >> > >> >> > > >> Istvan > >> >> > > >> > >> >> > > >> On Mon, Jan 29, 2024 at 10:02 PM Stephen Jiang < > >> >> > syuanjiang...@gmail.com > >> >> > > > > >> >> > > >> wrote: > >> >> > > >> > >> >> > > >>> I am not sure how close HBase 3.0 is. Even if it is only > less > >> >> than > >> >> > one > >> >> > > >>> year away, the adoption would be low at the beginning. I > don't > >> >> think > >> >> > > 5.3 > >> >> > > >>> should wait for that. And traditionally, Phoenix would > have a > >> >> major > >> >> > > >>> release to support the HBase major release (4.x for HBase 1.x > >> and > >> >> 5.x > >> >> > > for > >> >> > > >>> HBase 2.x), in this case, we are talking about Phoenix 6.0 > for > >> >> HBase > >> >> > > 3.0. > >> >> > > >>> > >> >> > > >>> Maybe we should adopt the HBase release model: master branch > >> for > >> >> next > >> >> > > >>> major > >> >> > > >>> release (6.0) and branch-5.x branch for next 5.x minor > release > >> and > >> >> > > >>> branch-5.2 for 5.2 minor release. > >> >> > > >>> > >> >> > > >>> Thanks > >> >> > > >>> Stephen > >> >> > > >>> > >> >> > > >>> > >> >> > > >>> > >> >> > > >>> On Thu, Jan 25, 2024 at 11:50 PM Viraj Jasani < > >> vjas...@apache.org > >> >> > > >> >> > > >>> wrote: > >> >> > > >>> > >> >> > > >>> > Sounds good. > >> >> > > >>> > > >> >> > > >>> > Planned major changes for 5.3.0: > >> >> > > >>> > > >> >> > > >>> > 1. JSON support. > >> >> > > >>> > 2. HBase 3.0 support. > >> >> > > >>> > 3. CDC feature (leveraging uncovered global index framework > >> and > >> >> > JSON > >> >> > > >>> > support). > >> >> > > >>> > > >> >> > > >>> > > >> >> > > >>> > On Wed, Jan 24, 2024 at 8:22 PM Kadir Ozdemir > >> >> > > >>> > <kozde...@salesforce.com.invalid> wrote: > >> >> > > >>> > > >> >> > > >>> > > I suggest including another major change for 5.3, Phoenix > >> CDC, > >> >> > > >>> > > https://issues.apache.org/jira/browse/PHOENIX-7001. The > PR > >> >> for > >> >> > it > >> >> > > >>> will > >> >> > > >>> > be > >> >> > > >>> > > posted soon. > >> >> > > >>> > > > >> >> > > >>> > > On Thu, Jan 25, 2024 at 5:35 AM Viraj Jasani < > >> >> vjas...@apache.org > >> >> > > > >> >> > > >>> wrote: > >> >> > > >>> > > > >> >> > > >>> > > > Thanks Istvan! I agree with your points. It’s really > >> been a > >> >> > while > >> >> > > >>> we > >> >> > > >>> > are > >> >> > > >>> > > > talking about releasing 5.2.0 and yet due to bandwidth > >> >> issues, > >> >> > > >>> unable > >> >> > > >>> > to > >> >> > > >>> > > do > >> >> > > >>> > > > so. > >> >> > > >>> > > > > >> >> > > >>> > > > I agree to getting these fixes out, cut 5.2 and start > the > >> >> > release > >> >> > > >>> work > >> >> > > >>> > > and > >> >> > > >>> > > > in the meantime I also need to prepare 5.1 backport. > >> >> > > >>> > > > > >> >> > > >>> > > > For 5.3, let's plan HBase 3.0 support and JSON as major > >> >> > changes. > >> >> > > >>> > > > > >> >> > > >>> > > > > >> >> > > >>> > > > On Mon, Jan 22, 2024 at 9:17 PM Istvan Toth < > >> >> st...@apache.org> > >> >> > > >>> wrote: > >> >> > > >>> > > > > >> >> > > >>> > > > > Hi! > >> >> > > >>> > > > > > >> >> > > >>> > > > > In my opinion cutting 5.2 now only makes sense IF we > DO > >> >> NOT > >> >> > > plan > >> >> > > >>> to > >> >> > > >>> > > > release > >> >> > > >>> > > > > the outstanding big features (like JSON) in 5.2. , > >> >> otherwise > >> >> > > it's > >> >> > > >>> > just > >> >> > > >>> > > > > extra work to maintain more branches. > >> >> > > >>> > > > > > >> >> > > >>> > > > > Having said that, releasing a 5.2 and 5.1.4 with the > >> data > >> >> > > >>> integrity > >> >> > > >>> > > fixes > >> >> > > >>> > > > > real soon, and then releasing 5.3 in a few months > with > >> >> JSON, > >> >> > > and > >> >> > > >>> any > >> >> > > >>> > > > other > >> >> > > >>> > > > > outstanding big features > >> >> > > >>> > > > > that are close to being finished (and HBase 3.0 > >> support, > >> >> if > >> >> > > it's > >> >> > > >>> > ready > >> >> > > >>> > > by > >> >> > > >>> > > > > then) would not be a bad idea. > >> >> > > >>> > > > > > >> >> > > >>> > > > > On the CLDR side the only outstanding big feature > which > >> >> could > >> >> > > >>> impact > >> >> > > >>> > > > > Viraj's integrity work is HBase 3.0 support, and even > >> >> that is > >> >> > > >>> only > >> >> > > >>> > > > because > >> >> > > >>> > > > > it may require some larger refactors of existing > code, > >> not > >> >> > > >>> because it > >> >> > > >>> > > > would > >> >> > > >>> > > > > change the actual behaviour or algorithms. > >> >> > > >>> > > > > > >> >> > > >>> > > > > Phoenix used to have several minor releases per year, > >> the > >> >> > > current > >> >> > > >>> > state > >> >> > > >>> > > > of > >> >> > > >>> > > > > extreme longevity of 5.1 and several big new features > >> >> being > >> >> > > >>> added to > >> >> > > >>> > it > >> >> > > >>> > > > > (like uncovered indexes) is not ideal. > >> >> > > >>> > > > > Releasing 5.2 and 5.3 relatively close together could > >> be a > >> >> > > >>> return to > >> >> > > >>> > a > >> >> > > >>> > > > > quicker cadence for minor releases, which could also > >> help > >> >> > with > >> >> > > >>> the > >> >> > > >>> > > public > >> >> > > >>> > > > > image and adoption of Phoenix. > >> >> > > >>> > > > > > >> >> > > >>> > > > > We were talking about releasing 5.2 at least a year > >> ago, > >> >> and > >> >> > I > >> >> > > >>> have > >> >> > > >>> > > > started > >> >> > > >>> > > > > working on that then, but then emergencies have come > >> up at > >> >> > > >>> $dayjob, > >> >> > > >>> > > and I > >> >> > > >>> > > > > could not see that through. > >> >> > > >>> > > > > (So I am in part responsible for the lack of minor > >> >> releases) > >> >> > > >>> > > > > > >> >> > > >>> > > > > regards > >> >> > > >>> > > > > Istvan > >> >> > > >>> > > > > > >> >> > > >>> > > > > > >> >> > > >>> > > > > On Mon, Jan 22, 2024 at 11:35 PM Viraj Jasani < > >> >> > > >>> vjas...@apache.org> > >> >> > > >>> > > > wrote: > >> >> > > >>> > > > > > >> >> > > >>> > > > > > Sorry for the late reply. > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > No problem with that. I can cut 5.2 branch by the > >> end of > >> >> > this > >> >> > > >>> week > >> >> > > >>> > or > >> >> > > >>> > > > at > >> >> > > >>> > > > > > the start of next week. > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > If there is any very big change or feature ready > for > >> >> merge > >> >> > to > >> >> > > >>> > master > >> >> > > >>> > > > > branch > >> >> > > >>> > > > > > with PR approvals already in place, please do let > me > >> >> know > >> >> > so > >> >> > > >>> that I > >> >> > > >>> > > can > >> >> > > >>> > > > > > help collaborate on how best we can get it merged > >> >> without > >> >> > > >>> impacting > >> >> > > >>> > > 5.2 > >> >> > > >>> > > > > > release if required. My main motivation was for any > >> big > >> >> > > change > >> >> > > >>> to > >> >> > > >>> > go > >> >> > > >>> > > > > > through newly introduced tests so that we know that > >> >> > anything > >> >> > > >>> > > additional > >> >> > > >>> > > > > is > >> >> > > >>> > > > > > not broken, and also to prioritize for upcoming > 5.2.0 > >> >> and > >> >> > > 5.1.4 > >> >> > > >>> > > > releases. > >> >> > > >>> > > > > > Moreover, there are several PRs getting merged on > the > >> >> > master > >> >> > > >>> > branch, > >> >> > > >>> > > we > >> >> > > >>> > > > > can > >> >> > > >>> > > > > > continue that as long as they are not very big > >> changes, > >> >> > which > >> >> > > >>> might > >> >> > > >>> > > > > require > >> >> > > >>> > > > > > significant time to understand any correlation with > >> data > >> >> > > >>> integrity > >> >> > > >>> > > > > issues. > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > The PR is also ready for review with some > additional > >> >> cases > >> >> > > >>> fixed > >> >> > > >>> > last > >> >> > > >>> > > > > week: > >> >> > > >>> > > > > > > >> >> > > >>> > > > > >> >> > > >>> > > > >> >> > > >>> > > >> >> > > >>> > >> >> > > > >> >> > > >> >> > >> > https://urldefense.com/v3/__https://github.com/apache/phoenix/pull/1736__;!!DCbAVzZNrAf4!D4OVjUp2EWW2BqhGnBxsapDX_AHsibRphIpoFBWfgRsd3dsAikrFLo6PGxdTzGbSXJJ2fJ0j9mcz3asXMXo$ > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > Depending on the review bandwidth, I am hopeful we > >> >> should > >> >> > be > >> >> > > >>> good > >> >> > > >>> > to > >> >> > > >>> > > > land > >> >> > > >>> > > > > > them sooner. > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > On Wed, Jan 17, 2024 at 11:31 AM Rushabh Shah > >> >> > > >>> > > > > > <rushabh.s...@salesforce.com.invalid> wrote: > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > Thank you Viraj for initiating this thread. > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > > Given the critical nature of these issues, I > >> would > >> >> like > >> >> > > to > >> >> > > >>> > > propose > >> >> > > >>> > > > > that > >> >> > > >>> > > > > > > we > >> >> > > >>> > > > > > > treat this as a high priority for the upcoming > >> 5.2.0 > >> >> > > >>> release, and > >> >> > > >>> > > not > >> >> > > >>> > > > > > > include any other feature or big change to master > >> >> branch > >> >> > > >>> until we > >> >> > > >>> > > > merge > >> >> > > >>> > > > > > > this. > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? This > >> will > >> >> > > enable > >> >> > > >>> > other > >> >> > > >>> > > > > > > developers to merge features into master branch > >> >> (5.3.0) > >> >> > and > >> >> > > >>> you > >> >> > > >>> > can > >> >> > > >>> > > > > take > >> >> > > >>> > > > > > > some more time to make sure we cover all the > corner > >> >> cases > >> >> > > >>> for the > >> >> > > >>> > > > data > >> >> > > >>> > > > > > > integrity issues that you uncovered. > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > On Fri, Jan 5, 2024 at 6:38 PM Viraj Jasani < > >> >> > > >>> vjas...@apache.org> > >> >> > > >>> > > > > wrote: > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > > Sounds good, thanks Rajeshbabu. I will try to > get > >> >> the > >> >> > > >>> first PR > >> >> > > >>> > > out > >> >> > > >>> > > > > next > >> >> > > >>> > > > > > > > week and while reviews happen in parallel, will > >> try > >> >> to > >> >> > > get > >> >> > > >>> 5.1 > >> >> > > >>> > PR > >> >> > > >>> > > > > soon. > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > On Wed, Jan 3, 2024 at 8:49 PM > >> >> rajeshb...@apache.org < > >> >> > > >>> > > > > > > > chrajeshbab...@gmail.com> wrote: > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > Hi Viraj, > >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > Would be better to include the changes in > 5.1.4 > >> >> as > >> >> > in > >> >> > > >>> any > >> >> > > >>> > way > >> >> > > >>> > > it > >> >> > > >>> > > > > > will > >> >> > > >>> > > > > > > > take > >> >> > > >>> > > > > > > > > at least 3-4 days to complete the omid > release. > >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > Thanks, > >> >> > > >>> > > > > > > > > Rajeshbabu. > >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > On Thu, Jan 4, 2024 at 5:06 AM Viraj Jasani < > >> >> > > >>> > > vjas...@apache.org> > >> >> > > >>> > > > > > > wrote: > >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > > Thank you Kadir and Geoffrey for your > >> replies!! > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, which is also > >> >> listed > >> >> > as > >> >> > > >>> a Fix > >> >> > > >>> > > > > Version > >> >> > > >>> > > > > > > for > >> >> > > >>> > > > > > > > > > > PHOENIX-7106? > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > Yes, it also needs to be ported to 5.1. > Once > >> the > >> >> > > >>> master PR > >> >> > > >>> > is > >> >> > > >>> > > > up > >> >> > > >>> > > > > > for > >> >> > > >>> > > > > > > > > final > >> >> > > >>> > > > > > > > > > review, I would start working on the > backport > >> >> PR. > >> >> > > >>> > > > > > > > > > We just need some more additional testing > to > >> >> ensure > >> >> > > old > >> >> > > >>> > > client > >> >> > > >>> > > > > > (e.g. > >> >> > > >>> > > > > > > > > 5.1.3) > >> >> > > >>> > > > > > > > > > is compatible with the new server with the > >> >> changes. > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > Hence, yes it is now a blocker for upcoming > >> >> 5.1.4 > >> >> > as > >> >> > > >>> well > >> >> > > >>> > > since > >> >> > > >>> > > > > > 5.1.4 > >> >> > > >>> > > > > > > > RC > >> >> > > >>> > > > > > > > > > preparation is still pending (while Omid > >> >> release is > >> >> > > in > >> >> > > >>> > > > progress). > >> >> > > >>> > > > > > > > > > Otherwise if 5.1.4 was ready for release, I > >> >> would > >> >> > > have > >> >> > > >>> > > proposed > >> >> > > >>> > > > > > > > immediate > >> >> > > >>> > > > > > > > > > 5.1.5 release to include the changes > proposed > >> >> with > >> >> > > >>> > > > PHOENIX-7106. > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > On Wed, Jan 3, 2024 at 3:08 PM Geoffrey > >> Jacoby < > >> >> > > >>> > > > > gjac...@apache.org > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > > > wrote: > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > > I agree that data integrity issues are a > >> >> higher > >> >> > > >>> priority > >> >> > > >>> > > than > >> >> > > >>> > > > > > > feature > >> >> > > >>> > > > > > > > > > > development, so I also support the > >> decision. > >> >> The > >> >> > > fact > >> >> > > >>> > that > >> >> > > >>> > > > > > several > >> >> > > >>> > > > > > > of > >> >> > > >>> > > > > > > > > the > >> >> > > >>> > > > > > > > > > > major remaining 5.2 features are > currently > >> >> being > >> >> > > >>> > developed > >> >> > > >>> > > in > >> >> > > >>> > > > > > > > > > long-running > >> >> > > >>> > > > > > > > > > > feature branches also helps, as work can > >> >> continue > >> >> > > >>> there > >> >> > > >>> > at > >> >> > > >>> > > > the > >> >> > > >>> > > > > > cost > >> >> > > >>> > > > > > > > of > >> >> > > >>> > > > > > > > > a > >> >> > > >>> > > > > > > > > > > rebase later. > >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, which is also > >> >> listed > >> >> > as > >> >> > > >>> a Fix > >> >> > > >>> > > > > Version > >> >> > > >>> > > > > > > for > >> >> > > >>> > > > > > > > > > > PHOENIX-7106? From the bug description it > >> also > >> >> > > sounds > >> >> > > >>> > like > >> >> > > >>> > > > > 5.1.3 > >> >> > > >>> > > > > > > and > >> >> > > >>> > > > > > > > > the > >> >> > > >>> > > > > > > > > > > forthcoming .4 are affected, since we > have > >> >> > > >>> server-side > >> >> > > >>> > > paging > >> >> > > >>> > > > > in > >> >> > > >>> > > > > > > 5.1. > >> >> > > >>> > > > > > > > > > (Feel > >> >> > > >>> > > > > > > > > > > free to move that to a separate thread if > >> you > >> >> > feel > >> >> > > it > >> >> > > >>> > > should > >> >> > > >>> > > > > be a > >> >> > > >>> > > > > > > > > > separate > >> >> > > >>> > > > > > > > > > > discussion.) Should this be a blocker for > >> >> > releasing > >> >> > > >>> > 5.1.4? > >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > Geoffrey > >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > On Wed, Jan 3, 2024 at 5:06 PM Kadir > >> Ozdemir < > >> >> > > >>> > > > > > > > > > > ka...@gsuite.cloud.apache.org> > >> >> > > >>> > > > > > > > > > > wrote: > >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > Being a database, Phoenix has to make > >> sure > >> >> that > >> >> > > the > >> >> > > >>> > data > >> >> > > >>> > > > > stays > >> >> > > >>> > > > > > on > >> >> > > >>> > > > > > > > > disk > >> >> > > >>> > > > > > > > > > > > intact and its queries return correct > >> data. > >> >> In > >> >> > > this > >> >> > > >>> > case, > >> >> > > >>> > > > > > Phoenix > >> >> > > >>> > > > > > > > > fails > >> >> > > >>> > > > > > > > > > > to > >> >> > > >>> > > > > > > > > > > > return correct data for some queries if > >> >> their > >> >> > > scans > >> >> > > >>> > > > > experience > >> >> > > >>> > > > > > > > region > >> >> > > >>> > > > > > > > > > > > movement. Now that we know these data > >> >> integrity > >> >> > > >>> issues > >> >> > > >>> > > and > >> >> > > >>> > > > > how > >> >> > > >>> > > > > > to > >> >> > > >>> > > > > > > > > > > reproduce > >> >> > > >>> > > > > > > > > > > > them, fixing them should be our first > >> >> priority. > >> >> > > >>> So, I > >> >> > > >>> > > fully > >> >> > > >>> > > > > > > support > >> >> > > >>> > > > > > > > > > this > >> >> > > >>> > > > > > > > > > > > proposal. > >> >> > > >>> > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > On Wed, Jan 3, 2024 at 10:58 PM Viraj > >> >> Jasani < > >> >> > > >>> > > > > > vjas...@apache.org > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > > wrote: > >> >> > > >>> > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > Hello, > >> >> > > >>> > > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > I would like to bring PHOENIX-7106 > >> >> > > >>> > > > > > > > > > > > > < > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > >> >> > > >>> > > > > >> >> > > >>> > > > >> >> > > >>> > > >> >> > > >>> > >> >> > > > >> >> > > >> >> > >> > https://urldefense.com/v3/__https://issues.apache.org/jira/browse/PHOENIX-7106__;!!DCbAVzZNrAf4!FZG5sv55IC1NqItQLY7GKWgUG2Do0gSta01gOiSdd36Dx3XHGtQx4M3c9visVXIt9DctPQzS-orob9vhzrCfVA$ > >> >> > > >>> > > > > > > > > to everyone's > >> >> > > >>> > > > > > > > > > > > > attention here and brief about the > data > >> >> > > integrity > >> >> > > >>> > > issues > >> >> > > >>> > > > > that > >> >> > > >>> > > > > > > we > >> >> > > >>> > > > > > > > > have > >> >> > > >>> > > > > > > > > > > in > >> >> > > >>> > > > > > > > > > > > > various coprocessors. Majority of the > >> >> issues > >> >> > > are > >> >> > > >>> > > related > >> >> > > >>> > > > to > >> >> > > >>> > > > > > the > >> >> > > >>> > > > > > > > > fact > >> >> > > >>> > > > > > > > > > > that > >> >> > > >>> > > > > > > > > > > > > we do not return valid rowkey for > >> certain > >> >> > > >>> queries. If > >> >> > > >>> > > any > >> >> > > >>> > > > > > > region > >> >> > > >>> > > > > > > > > > moves > >> >> > > >>> > > > > > > > > > > in > >> >> > > >>> > > > > > > > > > > > > the middle of the scan, the HBase > >> client > >> >> > relies > >> >> > > >>> on > >> >> > > >>> > the > >> >> > > >>> > > > last > >> >> > > >>> > > > > > > > > returned > >> >> > > >>> > > > > > > > > > > > rowkey > >> >> > > >>> > > > > > > > > > > > > and accordingly changes the scan > >> >> boundaries > >> >> > > >>> while the > >> >> > > >>> > > > > scanner > >> >> > > >>> > > > > > > is > >> >> > > >>> > > > > > > > > > > getting > >> >> > > >>> > > > > > > > > > > > > reset to continue the scan operation. > >> If > >> >> the > >> >> > > >>> region > >> >> > > >>> > > does > >> >> > > >>> > > > > not > >> >> > > >>> > > > > > > > move, > >> >> > > >>> > > > > > > > > > scan > >> >> > > >>> > > > > > > > > > > > is > >> >> > > >>> > > > > > > > > > > > > not expected to return invalid data, > >> >> however > >> >> > if > >> >> > > >>> the > >> >> > > >>> > > > region > >> >> > > >>> > > > > > > moves > >> >> > > >>> > > > > > > > in > >> >> > > >>> > > > > > > > > > the > >> >> > > >>> > > > > > > > > > > > > middle of ongoing scan operation, > scan > >> >> would > >> >> > > >>> return > >> >> > > >>> > > > > > > > > invalid/incorrect > >> >> > > >>> > > > > > > > > > > > data > >> >> > > >>> > > > > > > > > > > > > causing data integrity issues. > >> >> > > >>> > > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > Given the critical nature of these > >> >> issues, I > >> >> > > >>> would > >> >> > > >>> > like > >> >> > > >>> > > > to > >> >> > > >>> > > > > > > > propose > >> >> > > >>> > > > > > > > > > that > >> >> > > >>> > > > > > > > > > > > we > >> >> > > >>> > > > > > > > > > > > > treat this as a high priority for the > >> >> > upcoming > >> >> > > >>> 5.2.0 > >> >> > > >>> > > > > release, > >> >> > > >>> > > > > > > and > >> >> > > >>> > > > > > > > > not > >> >> > > >>> > > > > > > > > > > > > include any other feature or big > >> change to > >> >> > > master > >> >> > > >>> > > branch > >> >> > > >>> > > > > > until > >> >> > > >>> > > > > > > we > >> >> > > >>> > > > > > > > > > merge > >> >> > > >>> > > > > > > > > > > > > this. The PR is still not ready as > >> >> additional > >> >> > > >>> changes > >> >> > > >>> > > are > >> >> > > >>> > > > > > still > >> >> > > >>> > > > > > > > in > >> >> > > >>> > > > > > > > > my > >> >> > > >>> > > > > > > > > > > > > local, requiring rebase with the > >> current > >> >> > > master. > >> >> > > >>> > > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > I would get back to this discuss > >> thread as > >> >> > soon > >> >> > > >>> as > >> >> > > >>> > the > >> >> > > >>> > > PR > >> >> > > >>> > > > > and > >> >> > > >>> > > > > > > the > >> >> > > >>> > > > > > > > > doc > >> >> > > >>> > > > > > > > > > > are > >> >> > > >>> > > > > > > > > > > > > updated with the latest findings so > >> far. > >> >> The > >> >> > > >>> changes > >> >> > > >>> > > > > include > >> >> > > >>> > > > > > > many > >> >> > > >>> > > > > > > > > of > >> >> > > >>> > > > > > > > > > > our > >> >> > > >>> > > > > > > > > > > > > coproc scanner implementations and > >> hence > >> >> it > >> >> > > would > >> >> > > >>> > > require > >> >> > > >>> > > > > > > > > significant > >> >> > > >>> > > > > > > > > > > > > review as well. > >> >> > > >>> > > > > > > > > > > > > It would be great if we can hold on > to > >> >> > merging > >> >> > > >>> any > >> >> > > >>> > > > feature > >> >> > > >>> > > > > or > >> >> > > >>> > > > > > > big > >> >> > > >>> > > > > > > > > > > change > >> >> > > >>> > > > > > > > > > > > to > >> >> > > >>> > > > > > > > > > > > > master branch until this gets in so > as > >> to > >> >> not > >> >> > > >>> > > complicate > >> >> > > >>> > > > > > > > > > > > merging/rebasing. > >> >> > > >>> > > > > > > > > > > > > Once this is merged to the master > >> branch, > >> >> I > >> >> > > would > >> >> > > >>> > like > >> >> > > >>> > > to > >> >> > > >>> > > > > cut > >> >> > > >>> > > > > > > 5.2 > >> >> > > >>> > > > > > > > > > > branch > >> >> > > >>> > > > > > > > > > > > > from master and we can move forward > >> with > >> >> > 5.2.0 > >> >> > > >>> > release. > >> >> > > >>> > > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > Please let me know if this looks good > >> or > >> >> if > >> >> > you > >> >> > > >>> have > >> >> > > >>> > > any > >> >> > > >>> > > > > > other > >> >> > > >>> > > > > > > > high > >> >> > > >>> > > > > > > > > > > > > priority work for 5.2.0. > >> >> > > >>> > > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > > >> >> > > >>> > > > > > > > > > > >> >> > > >>> > > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > >> >> > > >>> > > > > >> >> > > >>> > > > >> >> > > >>> > > >> >> > > >>> > >> >> > > >> > >> >> > > >> > >> >> > > >> -- > >> >> > > >> *István Tóth* | Sr. Staff Software Engineer > >> >> > > >> *Email*: st...@cloudera.com > >> >> > > >> cloudera.com <https://www.cloudera.com> > >> >> > > >> [image: Cloudera] <https://www.cloudera.com/> > >> >> > > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> > >> >> [image: > >> >> > > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> > >> [image: > >> >> > > >> Cloudera on LinkedIn] < > >> https://www.linkedin.com/company/cloudera> > >> >> > > >> ------------------------------ > >> >> > > >> ------------------------------ > >> >> > > >> > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > > >> > > >
