Sure let's go for it. I understand downstreamers are not happy with CVEs coming from our artifacts that were released in 2022.
On Thu, Feb 15, 2024 at 6:05 PM rajeshb...@apache.org < chrajeshbab...@gmail.com> wrote: > Would be better to bump up Hadoop to 3.3.x I feel which has minimal > vulnerabilities compared to Hadoop 3.2.4. > > Thanks, > Rajeshbabu. > > On Fri, Feb 16, 2024, 7:25 AM Viraj Jasani <vjas...@apache.org> wrote: > > > Sure it sounds good to create PR for version upgrades while we are > getting > > close to releasing 5.2.0 and 5.1.4. > > However, if the build has unexpected test failures, we can cut 5.2 first, > > and focus on stabilizing the upgrade changes on master branch PR rather > > than 5.2 branch, allowing faster release. > > > > Some new features like CDC and JSON support will anyway need 5.3 release > > soon. > > > > > > On Thu, Feb 15, 2024 at 7:54 AM Istvan Toth <st...@apache.org> wrote: > > > > > This comment > > > <https://github.com/apache/phoenix/pull/1810#issuecomment-1945998086> > > got > > > me thinking. > > > > > > Most of the community (i.e. SFDC and CLDR) does not particularly care > > about > > > Hadoop and HBase dependency versions, as these are meant to be > overridden > > > anyway, > > > and we both build our own binaries. > > > However, for downstream projects that use Phoenix, old CVE-ridden > Hadoop > > > versions in the public maven artifacts can be a problem. > > > > > > While we cannot influence CVEs coming from Hadoop and HBase > (technically > > we > > > can, but it would be a bad idea to tamper with non-direct dependency > > > versions), > > > we can at least make sure that we use reasonably new Hadoop versions > for > > > building Phoenix (and including those in the public artifacts) > > > > > > Hbase branch-2.4 uses Hadoop 3.1.3 by default, but it switches to 3.2.0 > > > when being built on JDK11+. > > > Hbase branch-2.5 uses Hadoop 3.2.4. (but we still use 3.2.3) > > > > > > Based on the above, I think that we should bump the default Hadoop > > version > > > to 3.2.4 for each supported HBase profile. > > > The main reason we stick to the Hbase Hadoop versions is that we've > often > > > been bitten by binary incompatibilities > > > between Hadoop minor versions, but I think that using the latest 3.2 > > point > > > release should be safe. > > > > > > If there are any problems, the tests will find them, most of those > > > incompatibilities manifest in minicluster anyway. > > > > > > What do you think ? > > > Can you think of anything that this would break ? > > > We can also discuss going straight to 3.3.x, but we can certainly run > > some > > > tests at least to see the results. > > > > > > I have opened https://issues.apache.org/jira/browse/PHOENIX-7216. > > > I suggest treating this as an 5.2 blocker (at least until we decide not > > to) > > > > > > Istvan > > > > > > On Thu, Feb 15, 2024 at 8:49 AM Istvan Toth <st...@apache.org> wrote: > > > > > > > I have committed PHOENIX-7191 (thanks for the review, Viraj). > > > > I have also put up a PR <https://github.com/apache/phoenix/pull/1810 > > > > > for > > > > PHOENIX-7193. > > > > > > > > I was not able to test them comprehensively, as I am still struggling > > > with > > > > some issues with HBase 3 where simple GETs > > > > initiated from a coprocessor hang until they time out, but I wanted > to > > > get > > > > the known fixes in to unblock 5.2. > > > > > > > > Istvan > > > > > > > > On Tue, Feb 13, 2024 at 6:24 AM Istvan Toth <st...@apache.org> > wrote: > > > > > > > >> We didn't really have such branches for past releases (that I have > > > >> followed), but we could change the practice. > > > >> > > > >> When I acted as an RM, I didn't feel the need to branch early, but > if > > it > > > >> helps your planned workflow, then sure. > > > >> At least it would remind us to actually complete the release in a > > > >> reasonable amount of time. > > > >> > > > >> Istvan > > > >> > > > >> On Tue, Feb 13, 2024 at 4:30 AM Viraj Jasani <vjas...@apache.org> > > > wrote: > > > >> > > > >>> No worries, that’s fine, both 5.2.0 and 5.1.4 can wait for non-zk > > > >>> registry > > > >>> fixes for a week or so. > > > >>> > > > >>> On the other hand, how about we still cut 5.2 branch now and keep > > > >>> backporting changes landing on master branch to 5.2 if necessary. > > Once > > > >>> non-zk registries are fixed, I can start with 5.2.0 release > > > preparation. > > > >>> > > > >>> > > > >>> On Sun, Feb 11, 2024 at 11:28 PM Istvan Toth <st...@apache.org> > > wrote: > > > >>> > > > >>> > Thank you, Viraj, > > > >>> > > > > >>> > 5.1.4 also has the non-ZK registries feature, and has the same > bugs > > > as > > > >>> 5.2 > > > >>> > in that code. > > > >>> > > > > >>> > > > > >>> > > > > >>> > On Sat, Feb 10, 2024 at 7:00 PM Viraj Jasani <vjas...@apache.org > > > > > >>> wrote: > > > >>> > > > > >>> > > Sure no worries, we can wait a few more days. > > > >>> > > > > > >>> > > In the meantime, I have merged the backport PR on 5.1 branch > for > > > the > > > >>> data > > > >>> > > integrity fixes. Once omid dependency change is in, I believe > we > > > are > > > >>> good > > > >>> > > to start with 5.1.4. > > > >>> > > > > > >>> > > Thank you Rajeshbabu for volunteering to take it up. > > > >>> > > > > > >>> > > > > > >>> > > On Sat, Feb 10, 2024 at 1:28 AM Istvan Toth <st...@apache.org> > > > >>> wrote: > > > >>> > > > > > >>> > >> I appreciate the drive to get 5.2.0 out of the door, but I > would > > > >>> prefer > > > >>> > to > > > >>> > >> have a few more days to fix the registry issues, > > > >>> > >> and run some tests on them before cutting the branch, Viraj. > > > >>> > >> The non-ZK registry support is one of the bigger new features, > > and > > > >>> I'd > > > >>> > >> prefer not to have known breaking bugs in the release. > > > >>> > >> Can we target Friday or the next Monday for the cut ? > > > >>> > >> > > > >>> > >> Istvan > > > >>> > >> > > > >>> > >> On Sat, Feb 10, 2024 at 10:07 AM rajeshb...@apache.org < > > > >>> > >> chrajeshbab...@gmail.com> wrote: > > > >>> > >> > > > >>> > >> > Yes Viraj, I can release 5.1.4 > > > >>> > >> > > > > >>> > >> > Thanks, > > > >>> > >> > Rajeshbabu. > > > >>> > >> > > > > >>> > >> > On Sat, Feb 10, 2024, 10:28 AM Viraj Jasani < > > vjas...@apache.org > > > > > > > >>> > wrote: > > > >>> > >> > > > > >>> > >> >> I think we can also target 5.2.1 very soon, perhaps just > next > > > >>> month, > > > >>> > >> with > > > >>> > >> >> more CVE fixes and any other fixes if ready. > > > >>> > >> >> > > > >>> > >> >> > > > >>> > >> >> On Fri, Feb 9, 2024 at 6:39 PM Viraj Jasani < > > > vjas...@apache.org> > > > >>> > >> wrote: > > > >>> > >> >> > > > >>> > >> >> > For 5.2.0, it would be great to focus on the known data > > > >>> integrity > > > >>> > >> >> issues. > > > >>> > >> >> > We can fix non-zk registry, cover a few more CVEs by > > > upgrading > > > >>> > third > > > >>> > >> >> party > > > >>> > >> >> > dependencies and stabilize tests. As for the tests, they > > > don’t > > > >>> seem > > > >>> > >> >> broken, > > > >>> > >> >> > but are flaky. I have got multiple builds without any > test > > > >>> failures > > > >>> > >> on > > > >>> > >> >> PR > > > >>> > >> >> > for PHOENIX-7106. > > > >>> > >> >> > > > > >>> > >> >> > If this looks good to you, I can start release > preparation > > > next > > > >>> > week. > > > >>> > >> >> What > > > >>> > >> >> > do you think, Istvan? > > > >>> > >> >> > > > > >>> > >> >> > In the meantime, I have 5.1 backport PR open, awaiting > good > > > >>> build > > > >>> > >> >> results > > > >>> > >> >> > before committing it. > > > >>> > >> >> > Rajeshbabu, would you like to be RM for 5.1.4 once the PR > > is > > > >>> > merged? > > > >>> > >> >> > > > > >>> > >> >> > > > > >>> > >> >> > > > > >>> > >> >> > On Fri, Feb 9, 2024 at 11:13 AM Istvan Toth < > > > st...@apache.org> > > > >>> > >> wrote: > > > >>> > >> >> > > > > >>> > >> >> >> Yes, they basically make the non-ZK registries unusable. > > > >>> > >> >> >> (at least the connectionless problems should be fixed.) > > > >>> > >> >> >> > > > >>> > >> >> >> I hope to have the final fix for those sometime next > week. > > > >>> > >> >> >> > > > >>> > >> >> >> Also have we looked at potential CVE issues on master > > > >>> recently ? > > > >>> > >> >> >> > > > >>> > >> >> >> I think we should also look at the most flakey tests I > > > linked > > > >>> > above, > > > >>> > >> >> >> and fix them or at least make sure that they are test > > issues > > > >>> and > > > >>> > not > > > >>> > >> >> real > > > >>> > >> >> >> bugs. > > > >>> > >> >> >> > > > >>> > >> >> >> Istvan > > > >>> > >> >> >> > > > >>> > >> >> >> > > > >>> > >> >> >> > > > >>> > >> >> >> On Fri, Feb 9, 2024 at 6:55 PM Viraj Jasani < > > > >>> vjas...@apache.org> > > > >>> > >> >> wrote: > > > >>> > >> >> >> > > > >>> > >> >> >> > Thanks Istvan. > > > >>> > >> >> >> > I would like to cut 5.2 branch from master. Do you see > > > >>> non-ZK > > > >>> > >> >> registry > > > >>> > >> >> >> for > > > >>> > >> >> >> > MapReduce jobs as blocker for 5.2.0? > > > >>> > >> >> >> > > > > >>> > >> >> >> > > > > >>> > >> >> >> > On Fri, Feb 9, 2024 at 12:24 AM Istvan Toth < > > > >>> st...@apache.org> > > > >>> > >> >> wrote: > > > >>> > >> >> >> > > > > >>> > >> >> >> > > > > > >>> > >> > > ParallelPhoenixConnectionFailureTest.testExecuteQueryChainFailure > > > >>> > >> >> also > > > >>> > >> >> >> > > fails too often, especially when the test host is > slow > > > >>> and/or > > > >>> > >> the > > > >>> > >> >> >> load is > > > >>> > >> >> >> > > high. > > > >>> > >> >> >> > > On my fast laptop, I can semi-reliably break it by > > > running > > > >>> > >> >> >> > > mvn clean verify -am -pl phoenix-core > -DnumForkedUT=20 > > > >>> > >> >> >> > > > > > >>> > >> >> >> > > On Fri, Feb 9, 2024 at 9:19 AM Istvan Toth < > > > >>> st...@apache.org> > > > >>> > >> >> wrote: > > > >>> > >> >> >> > > > > > >>> > >> >> >> > > > We're making progress. > > > >>> > >> >> >> > > > I can see that Viraj has just landed PHOENIX-7601, > > and > > > >>> > >> Rajeshbabu > > > >>> > >> >> >> has > > > >>> > >> >> >> > > > released Omid 1.1.1. > > > >>> > >> >> >> > > > Thank you! > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > At the moment, the following outstanding issues > are > > on > > > >>> my > > > >>> > >> radar: > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > > https://issues.apache.org/jira/browse/PHOENIX-7191 > > > >>> > >> >> >> > > > > https://issues.apache.org/jira/browse/PHOENIX-7193 > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > These are bugs in my non-ZK registry > implementation, > > > >>> which > > > >>> > >> were > > > >>> > >> >> >> found > > > >>> > >> >> >> > > > during HBase 3 work. > > > >>> > >> >> >> > > > I have some PRs up, but they may not be complete. > I > > > will > > > >>> > push > > > >>> > >> for > > > >>> > >> >> >> > reviews > > > >>> > >> >> >> > > > once I have the HBase 3 tests passing, and > possibly > > > >>> updated > > > >>> > >> them > > > >>> > >> >> >> based > > > >>> > >> >> >> > on > > > >>> > >> >> >> > > > that. > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > We also have a number of very flakey tests, see: > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > > > >>> > >> >> >> > > > > >>> > >> >> >> > > > >>> > >> >> > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/test_results_analyzer/ > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > > On Tue, Jan 30, 2024 at 7:09 AM Istvan Toth < > > > >>> > >> st...@cloudera.com> > > > >>> > >> >> >> > wrote: > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > >> As Viraj wrote, those are just plans. > > > >>> > >> >> >> > > >> If HBase 3 won't be released by the time the > other > > > >>> features > > > >>> > >> are > > > >>> > >> >> >> ready, > > > >>> > >> >> >> > > >> then it won't make it into 5.3. > > > >>> > >> >> >> > > >> If other major features are ready by that time, > > then > > > >>> they > > > >>> > >> will > > > >>> > >> >> be > > > >>> > >> >> >> > > >> included. (though we are not aware of any now) > > > >>> > >> >> >> > > >> > > > >>> > >> >> >> > > >> As for the new major version, in the past Phoenix > > > >>> didn't > > > >>> > >> have a > > > >>> > >> >> >> > > >> compatibility module system, > > > >>> > >> >> >> > > >> so a new branch was required, which didn't > support > > > >>> older > > > >>> > >> >> HBases. > > > >>> > >> >> >> > Also, > > > >>> > >> >> >> > > >> the API changes between HBase 1.x and 2.x were > much > > > >>> larger, > > > >>> > >> >> >> > > >> The HBase 2 and 3 API are pretty similar, apart > > from > > > >>> the > > > >>> > >> >> removal of > > > >>> > >> >> >> > > >> deprecated 1.x APIs. (and the protobuf/protocol > > > thing, > > > >>> > which > > > >>> > >> >> >> requires > > > >>> > >> >> >> > a > > > >>> > >> >> >> > > >> rather ugly hack). > > > >>> > >> >> >> > > >> > > > >>> > >> >> >> > > >> I will start the discussion on how we can add > > HBase 3 > > > >>> > >> support as > > > >>> > >> >> >> soon > > > >>> > >> >> >> > as > > > >>> > >> >> >> > > >> I have a working POC patch. > > > >>> > >> >> >> > > >> > > > >>> > >> >> >> > > >> We could call 5.3 6.0 instead, after all Phoenix > > > isn't > > > >>> > using > > > >>> > >> a > > > >>> > >> >> >> strict > > > >>> > >> >> >> > > >> semantic versioning, but then 6.0 would also > > support > > > >>> HBase > > > >>> > 2. > > > >>> > >> >> >> > > >> If we do not come to a consensus on the version > > name, > > > >>> we > > > >>> > can > > > >>> > >> >> always > > > >>> > >> >> >> > have > > > >>> > >> >> >> > > >> a vote on it. > > > >>> > >> >> >> > > >> > > > >>> > >> >> >> > > >> I think that the main motivation is that the > > > community > > > >>> > wants > > > >>> > >> to > > > >>> > >> >> >> > maintain > > > >>> > >> >> >> > > >> as few branches as possible. > > > >>> > >> >> >> > > >> > > > >>> > >> >> >> > > >> Istvan > > > >>> > >> >> >> > > >> > > > >>> > >> >> >> > > >> On Mon, Jan 29, 2024 at 10:02 PM Stephen Jiang < > > > >>> > >> >> >> > syuanjiang...@gmail.com > > > >>> > >> >> >> > > > > > > >>> > >> >> >> > > >> wrote: > > > >>> > >> >> >> > > >> > > > >>> > >> >> >> > > >>> I am not sure how close HBase 3.0 is. Even if > it > > is > > > >>> only > > > >>> > >> less > > > >>> > >> >> >> than > > > >>> > >> >> >> > one > > > >>> > >> >> >> > > >>> year away, the adoption would be low at the > > > >>> beginning. I > > > >>> > >> don't > > > >>> > >> >> >> think > > > >>> > >> >> >> > > 5.3 > > > >>> > >> >> >> > > >>> should wait for that. And traditionally, > Phoenix > > > >>> would > > > >>> > >> have a > > > >>> > >> >> >> major > > > >>> > >> >> >> > > >>> release to support the HBase major release (4.x > > for > > > >>> HBase > > > >>> > >> 1.x > > > >>> > >> >> and > > > >>> > >> >> >> 5.x > > > >>> > >> >> >> > > for > > > >>> > >> >> >> > > >>> HBase 2.x), in this case, we are talking about > > > >>> Phoenix 6.0 > > > >>> > >> for > > > >>> > >> >> >> HBase > > > >>> > >> >> >> > > 3.0. > > > >>> > >> >> >> > > >>> > > > >>> > >> >> >> > > >>> Maybe we should adopt the HBase release model: > > > master > > > >>> > branch > > > >>> > >> >> for > > > >>> > >> >> >> next > > > >>> > >> >> >> > > >>> major > > > >>> > >> >> >> > > >>> release (6.0) and branch-5.x branch for next 5.x > > > minor > > > >>> > >> release > > > >>> > >> >> and > > > >>> > >> >> >> > > >>> branch-5.2 for 5.2 minor release. > > > >>> > >> >> >> > > >>> > > > >>> > >> >> >> > > >>> Thanks > > > >>> > >> >> >> > > >>> Stephen > > > >>> > >> >> >> > > >>> > > > >>> > >> >> >> > > >>> > > > >>> > >> >> >> > > >>> > > > >>> > >> >> >> > > >>> On Thu, Jan 25, 2024 at 11:50 PM Viraj Jasani < > > > >>> > >> >> vjas...@apache.org > > > >>> > >> >> >> > > > > >>> > >> >> >> > > >>> wrote: > > > >>> > >> >> >> > > >>> > > > >>> > >> >> >> > > >>> > Sounds good. > > > >>> > >> >> >> > > >>> > > > > >>> > >> >> >> > > >>> > Planned major changes for 5.3.0: > > > >>> > >> >> >> > > >>> > > > > >>> > >> >> >> > > >>> > 1. JSON support. > > > >>> > >> >> >> > > >>> > 2. HBase 3.0 support. > > > >>> > >> >> >> > > >>> > 3. CDC feature (leveraging uncovered global > > index > > > >>> > >> framework > > > >>> > >> >> and > > > >>> > >> >> >> > JSON > > > >>> > >> >> >> > > >>> > support). > > > >>> > >> >> >> > > >>> > > > > >>> > >> >> >> > > >>> > > > > >>> > >> >> >> > > >>> > On Wed, Jan 24, 2024 at 8:22 PM Kadir Ozdemir > > > >>> > >> >> >> > > >>> > <kozde...@salesforce.com.invalid> wrote: > > > >>> > >> >> >> > > >>> > > > > >>> > >> >> >> > > >>> > > I suggest including another major change for > > > 5.3, > > > >>> > >> Phoenix > > > >>> > >> >> CDC, > > > >>> > >> >> >> > > >>> > > > > > >>> https://issues.apache.org/jira/browse/PHOENIX-7001. > > > >>> > >> The PR > > > >>> > >> >> >> for > > > >>> > >> >> >> > it > > > >>> > >> >> >> > > >>> will > > > >>> > >> >> >> > > >>> > be > > > >>> > >> >> >> > > >>> > > posted soon. > > > >>> > >> >> >> > > >>> > > > > > >>> > >> >> >> > > >>> > > On Thu, Jan 25, 2024 at 5:35 AM Viraj > Jasani < > > > >>> > >> >> >> vjas...@apache.org > > > >>> > >> >> >> > > > > > >>> > >> >> >> > > >>> wrote: > > > >>> > >> >> >> > > >>> > > > > > >>> > >> >> >> > > >>> > > > Thanks Istvan! I agree with your points. > > It’s > > > >>> really > > > >>> > >> >> been a > > > >>> > >> >> >> > while > > > >>> > >> >> >> > > >>> we > > > >>> > >> >> >> > > >>> > are > > > >>> > >> >> >> > > >>> > > > talking about releasing 5.2.0 and yet due > to > > > >>> > bandwidth > > > >>> > >> >> >> issues, > > > >>> > >> >> >> > > >>> unable > > > >>> > >> >> >> > > >>> > to > > > >>> > >> >> >> > > >>> > > do > > > >>> > >> >> >> > > >>> > > > so. > > > >>> > >> >> >> > > >>> > > > > > > >>> > >> >> >> > > >>> > > > I agree to getting these fixes out, cut > 5.2 > > > and > > > >>> > start > > > >>> > >> the > > > >>> > >> >> >> > release > > > >>> > >> >> >> > > >>> work > > > >>> > >> >> >> > > >>> > > and > > > >>> > >> >> >> > > >>> > > > in the meantime I also need to prepare 5.1 > > > >>> backport. > > > >>> > >> >> >> > > >>> > > > > > > >>> > >> >> >> > > >>> > > > For 5.3, let's plan HBase 3.0 support and > > JSON > > > >>> as > > > >>> > >> major > > > >>> > >> >> >> > changes. > > > >>> > >> >> >> > > >>> > > > > > > >>> > >> >> >> > > >>> > > > > > > >>> > >> >> >> > > >>> > > > On Mon, Jan 22, 2024 at 9:17 PM Istvan > Toth > > < > > > >>> > >> >> >> st...@apache.org> > > > >>> > >> >> >> > > >>> wrote: > > > >>> > >> >> >> > > >>> > > > > > > >>> > >> >> >> > > >>> > > > > Hi! > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > In my opinion cutting 5.2 now only makes > > > >>> sense IF > > > >>> > >> we DO > > > >>> > >> >> >> NOT > > > >>> > >> >> >> > > plan > > > >>> > >> >> >> > > >>> to > > > >>> > >> >> >> > > >>> > > > release > > > >>> > >> >> >> > > >>> > > > > the outstanding big features (like JSON) > > in > > > >>> 5.2. , > > > >>> > >> >> >> otherwise > > > >>> > >> >> >> > > it's > > > >>> > >> >> >> > > >>> > just > > > >>> > >> >> >> > > >>> > > > > extra work to maintain more branches. > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > Having said that, releasing a 5.2 and > > 5.1.4 > > > >>> with > > > >>> > the > > > >>> > >> >> data > > > >>> > >> >> >> > > >>> integrity > > > >>> > >> >> >> > > >>> > > fixes > > > >>> > >> >> >> > > >>> > > > > real soon, and then releasing 5.3 in a > few > > > >>> months > > > >>> > >> with > > > >>> > >> >> >> JSON, > > > >>> > >> >> >> > > and > > > >>> > >> >> >> > > >>> any > > > >>> > >> >> >> > > >>> > > > other > > > >>> > >> >> >> > > >>> > > > > outstanding big features > > > >>> > >> >> >> > > >>> > > > > that are close to being finished (and > > HBase > > > >>> 3.0 > > > >>> > >> >> support, > > > >>> > >> >> >> if > > > >>> > >> >> >> > > it's > > > >>> > >> >> >> > > >>> > ready > > > >>> > >> >> >> > > >>> > > by > > > >>> > >> >> >> > > >>> > > > > then) would not be a bad idea. > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > On the CLDR side the only outstanding > big > > > >>> feature > > > >>> > >> which > > > >>> > >> >> >> could > > > >>> > >> >> >> > > >>> impact > > > >>> > >> >> >> > > >>> > > > > Viraj's integrity work is HBase 3.0 > > support, > > > >>> and > > > >>> > >> even > > > >>> > >> >> >> that is > > > >>> > >> >> >> > > >>> only > > > >>> > >> >> >> > > >>> > > > because > > > >>> > >> >> >> > > >>> > > > > it may require some larger refactors of > > > >>> existing > > > >>> > >> code, > > > >>> > >> >> not > > > >>> > >> >> >> > > >>> because it > > > >>> > >> >> >> > > >>> > > > would > > > >>> > >> >> >> > > >>> > > > > change the actual behaviour or > algorithms. > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > Phoenix used to have several minor > > releases > > > >>> per > > > >>> > >> year, > > > >>> > >> >> the > > > >>> > >> >> >> > > current > > > >>> > >> >> >> > > >>> > state > > > >>> > >> >> >> > > >>> > > > of > > > >>> > >> >> >> > > >>> > > > > extreme longevity of 5.1 and several big > > new > > > >>> > >> features > > > >>> > >> >> >> being > > > >>> > >> >> >> > > >>> added to > > > >>> > >> >> >> > > >>> > it > > > >>> > >> >> >> > > >>> > > > > (like uncovered indexes) is not ideal. > > > >>> > >> >> >> > > >>> > > > > Releasing 5.2 and 5.3 relatively close > > > >>> together > > > >>> > >> could > > > >>> > >> >> be a > > > >>> > >> >> >> > > >>> return to > > > >>> > >> >> >> > > >>> > a > > > >>> > >> >> >> > > >>> > > > > quicker cadence for minor releases, > which > > > >>> could > > > >>> > also > > > >>> > >> >> help > > > >>> > >> >> >> > with > > > >>> > >> >> >> > > >>> the > > > >>> > >> >> >> > > >>> > > public > > > >>> > >> >> >> > > >>> > > > > image and adoption of Phoenix. > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > We were talking about releasing 5.2 at > > > least a > > > >>> > year > > > >>> > >> >> ago, > > > >>> > >> >> >> and > > > >>> > >> >> >> > I > > > >>> > >> >> >> > > >>> have > > > >>> > >> >> >> > > >>> > > > started > > > >>> > >> >> >> > > >>> > > > > working on that then, but then > emergencies > > > >>> have > > > >>> > come > > > >>> > >> >> up at > > > >>> > >> >> >> > > >>> $dayjob, > > > >>> > >> >> >> > > >>> > > and I > > > >>> > >> >> >> > > >>> > > > > could not see that through. > > > >>> > >> >> >> > > >>> > > > > (So I am in part responsible for the > lack > > of > > > >>> minor > > > >>> > >> >> >> releases) > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > regards > > > >>> > >> >> >> > > >>> > > > > Istvan > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > On Mon, Jan 22, 2024 at 11:35 PM Viraj > > > Jasani > > > >>> < > > > >>> > >> >> >> > > >>> vjas...@apache.org> > > > >>> > >> >> >> > > >>> > > > wrote: > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > > Sorry for the late reply. > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes > > > >>> sense? > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > No problem with that. I can cut 5.2 > > branch > > > >>> by > > > >>> > the > > > >>> > >> >> end of > > > >>> > >> >> >> > this > > > >>> > >> >> >> > > >>> week > > > >>> > >> >> >> > > >>> > or > > > >>> > >> >> >> > > >>> > > > at > > > >>> > >> >> >> > > >>> > > > > > the start of next week. > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > If there is any very big change or > > feature > > > >>> ready > > > >>> > >> for > > > >>> > >> >> >> merge > > > >>> > >> >> >> > to > > > >>> > >> >> >> > > >>> > master > > > >>> > >> >> >> > > >>> > > > > branch > > > >>> > >> >> >> > > >>> > > > > > with PR approvals already in place, > > please > > > >>> do > > > >>> > let > > > >>> > >> me > > > >>> > >> >> >> know > > > >>> > >> >> >> > so > > > >>> > >> >> >> > > >>> that I > > > >>> > >> >> >> > > >>> > > can > > > >>> > >> >> >> > > >>> > > > > > help collaborate on how best we can > get > > it > > > >>> > merged > > > >>> > >> >> >> without > > > >>> > >> >> >> > > >>> impacting > > > >>> > >> >> >> > > >>> > > 5.2 > > > >>> > >> >> >> > > >>> > > > > > release if required. My main > motivation > > > was > > > >>> for > > > >>> > >> any > > > >>> > >> >> big > > > >>> > >> >> >> > > change > > > >>> > >> >> >> > > >>> to > > > >>> > >> >> >> > > >>> > go > > > >>> > >> >> >> > > >>> > > > > > through newly introduced tests so that > > we > > > >>> know > > > >>> > >> that > > > >>> > >> >> >> > anything > > > >>> > >> >> >> > > >>> > > additional > > > >>> > >> >> >> > > >>> > > > > is > > > >>> > >> >> >> > > >>> > > > > > not broken, and also to prioritize for > > > >>> upcoming > > > >>> > >> 5.2.0 > > > >>> > >> >> >> and > > > >>> > >> >> >> > > 5.1.4 > > > >>> > >> >> >> > > >>> > > > releases. > > > >>> > >> >> >> > > >>> > > > > > Moreover, there are several PRs > getting > > > >>> merged > > > >>> > on > > > >>> > >> the > > > >>> > >> >> >> > master > > > >>> > >> >> >> > > >>> > branch, > > > >>> > >> >> >> > > >>> > > we > > > >>> > >> >> >> > > >>> > > > > can > > > >>> > >> >> >> > > >>> > > > > > continue that as long as they are not > > very > > > >>> big > > > >>> > >> >> changes, > > > >>> > >> >> >> > which > > > >>> > >> >> >> > > >>> might > > > >>> > >> >> >> > > >>> > > > > require > > > >>> > >> >> >> > > >>> > > > > > significant time to understand any > > > >>> correlation > > > >>> > >> with > > > >>> > >> >> data > > > >>> > >> >> >> > > >>> integrity > > > >>> > >> >> >> > > >>> > > > > issues. > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > The PR is also ready for review with > > some > > > >>> > >> additional > > > >>> > >> >> >> cases > > > >>> > >> >> >> > > >>> fixed > > > >>> > >> >> >> > > >>> > last > > > >>> > >> >> >> > > >>> > > > > week: > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > > >>> > >> >> >> > > >>> > > > > > >>> > >> >> >> > > >>> > > > > >>> > >> >> >> > > >>> > > > >>> > >> >> >> > > > > > >>> > >> >> >> > > > > >>> > >> >> >> > > > >>> > >> >> > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://urldefense.com/v3/__https://github.com/apache/phoenix/pull/1736__;!!DCbAVzZNrAf4!D4OVjUp2EWW2BqhGnBxsapDX_AHsibRphIpoFBWfgRsd3dsAikrFLo6PGxdTzGbSXJJ2fJ0j9mcz3asXMXo$ > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > Depending on the review bandwidth, I > am > > > >>> hopeful > > > >>> > we > > > >>> > >> >> >> should > > > >>> > >> >> >> > be > > > >>> > >> >> >> > > >>> good > > > >>> > >> >> >> > > >>> > to > > > >>> > >> >> >> > > >>> > > > land > > > >>> > >> >> >> > > >>> > > > > > them sooner. > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > On Wed, Jan 17, 2024 at 11:31 AM > Rushabh > > > >>> Shah > > > >>> > >> >> >> > > >>> > > > > > <rushabh.s...@salesforce.com.invalid> > > > >>> wrote: > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > > Thank you Viraj for initiating this > > > >>> thread. > > > >>> > >> >> >> > > >>> > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > Given the critical nature of these > > > >>> issues, I > > > >>> > >> >> would > > > >>> > >> >> >> like > > > >>> > >> >> >> > > to > > > >>> > >> >> >> > > >>> > > propose > > > >>> > >> >> >> > > >>> > > > > that > > > >>> > >> >> >> > > >>> > > > > > > we > > > >>> > >> >> >> > > >>> > > > > > > treat this as a high priority for > the > > > >>> upcoming > > > >>> > >> >> 5.2.0 > > > >>> > >> >> >> > > >>> release, and > > > >>> > >> >> >> > > >>> > > not > > > >>> > >> >> >> > > >>> > > > > > > include any other feature or big > > change > > > to > > > >>> > >> master > > > >>> > >> >> >> branch > > > >>> > >> >> >> > > >>> until we > > > >>> > >> >> >> > > >>> > > > merge > > > >>> > >> >> >> > > >>> > > > > > > this. > > > >>> > >> >> >> > > >>> > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > Do you think cutting 5.2.0 now makes > > > >>> sense? > > > >>> > This > > > >>> > >> >> will > > > >>> > >> >> >> > > enable > > > >>> > >> >> >> > > >>> > other > > > >>> > >> >> >> > > >>> > > > > > > developers to merge features into > > master > > > >>> > branch > > > >>> > >> >> >> (5.3.0) > > > >>> > >> >> >> > and > > > >>> > >> >> >> > > >>> you > > > >>> > >> >> >> > > >>> > can > > > >>> > >> >> >> > > >>> > > > > take > > > >>> > >> >> >> > > >>> > > > > > > some more time to make sure we cover > > all > > > >>> the > > > >>> > >> corner > > > >>> > >> >> >> cases > > > >>> > >> >> >> > > >>> for the > > > >>> > >> >> >> > > >>> > > > data > > > >>> > >> >> >> > > >>> > > > > > > integrity issues that you uncovered. > > > >>> > >> >> >> > > >>> > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > On Fri, Jan 5, 2024 at 6:38 PM Viraj > > > >>> Jasani < > > > >>> > >> >> >> > > >>> vjas...@apache.org> > > > >>> > >> >> >> > > >>> > > > > wrote: > > > >>> > >> >> >> > > >>> > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > Sounds good, thanks Rajeshbabu. I > > will > > > >>> try > > > >>> > to > > > >>> > >> get > > > >>> > >> >> >> the > > > >>> > >> >> >> > > >>> first PR > > > >>> > >> >> >> > > >>> > > out > > > >>> > >> >> >> > > >>> > > > > next > > > >>> > >> >> >> > > >>> > > > > > > > week and while reviews happen in > > > >>> parallel, > > > >>> > >> will > > > >>> > >> >> try > > > >>> > >> >> >> to > > > >>> > >> >> >> > > get > > > >>> > >> >> >> > > >>> 5.1 > > > >>> > >> >> >> > > >>> > PR > > > >>> > >> >> >> > > >>> > > > > soon. > > > >>> > >> >> >> > > >>> > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > On Wed, Jan 3, 2024 at 8:49 PM > > > >>> > >> >> >> rajeshb...@apache.org < > > > >>> > >> >> >> > > >>> > > > > > > > chrajeshbab...@gmail.com> wrote: > > > >>> > >> >> >> > > >>> > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > Hi Viraj, > > > >>> > >> >> >> > > >>> > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > Would be better to include the > > > >>> changes in > > > >>> > >> 5.1.4 > > > >>> > >> >> >> as > > > >>> > >> >> >> > in > > > >>> > >> >> >> > > >>> any > > > >>> > >> >> >> > > >>> > way > > > >>> > >> >> >> > > >>> > > it > > > >>> > >> >> >> > > >>> > > > > > will > > > >>> > >> >> >> > > >>> > > > > > > > take > > > >>> > >> >> >> > > >>> > > > > > > > > at least 3-4 days to complete > the > > > omid > > > >>> > >> release. > > > >>> > >> >> >> > > >>> > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > Thanks, > > > >>> > >> >> >> > > >>> > > > > > > > > Rajeshbabu. > > > >>> > >> >> >> > > >>> > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > On Thu, Jan 4, 2024 at 5:06 AM > > Viraj > > > >>> > Jasani > > > >>> > >> < > > > >>> > >> >> >> > > >>> > > vjas...@apache.org> > > > >>> > >> >> >> > > >>> > > > > > > wrote: > > > >>> > >> >> >> > > >>> > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > Thank you Kadir and Geoffrey > for > > > >>> your > > > >>> > >> >> replies!! > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, > > > which > > > >>> is > > > >>> > >> also > > > >>> > >> >> >> listed > > > >>> > >> >> >> > as > > > >>> > >> >> >> > > >>> a Fix > > > >>> > >> >> >> > > >>> > > > > Version > > > >>> > >> >> >> > > >>> > > > > > > for > > > >>> > >> >> >> > > >>> > > > > > > > > > > PHOENIX-7106? > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > Yes, it also needs to be > ported > > to > > > >>> 5.1. > > > >>> > >> Once > > > >>> > >> >> the > > > >>> > >> >> >> > > >>> master PR > > > >>> > >> >> >> > > >>> > is > > > >>> > >> >> >> > > >>> > > > up > > > >>> > >> >> >> > > >>> > > > > > for > > > >>> > >> >> >> > > >>> > > > > > > > > final > > > >>> > >> >> >> > > >>> > > > > > > > > > review, I would start working > on > > > the > > > >>> > >> backport > > > >>> > >> >> >> PR. > > > >>> > >> >> >> > > >>> > > > > > > > > > We just need some more > > additional > > > >>> > testing > > > >>> > >> to > > > >>> > >> >> >> ensure > > > >>> > >> >> >> > > old > > > >>> > >> >> >> > > >>> > > client > > > >>> > >> >> >> > > >>> > > > > > (e.g. > > > >>> > >> >> >> > > >>> > > > > > > > > 5.1.3) > > > >>> > >> >> >> > > >>> > > > > > > > > > is compatible with the new > > server > > > >>> with > > > >>> > the > > > >>> > >> >> >> changes. > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > Hence, yes it is now a blocker > > for > > > >>> > >> upcoming > > > >>> > >> >> >> 5.1.4 > > > >>> > >> >> >> > as > > > >>> > >> >> >> > > >>> well > > > >>> > >> >> >> > > >>> > > since > > > >>> > >> >> >> > > >>> > > > > > 5.1.4 > > > >>> > >> >> >> > > >>> > > > > > > > RC > > > >>> > >> >> >> > > >>> > > > > > > > > > preparation is still pending > > > (while > > > >>> Omid > > > >>> > >> >> >> release is > > > >>> > >> >> >> > > in > > > >>> > >> >> >> > > >>> > > > progress). > > > >>> > >> >> >> > > >>> > > > > > > > > > Otherwise if 5.1.4 was ready > for > > > >>> > release, > > > >>> > >> I > > > >>> > >> >> >> would > > > >>> > >> >> >> > > have > > > >>> > >> >> >> > > >>> > > proposed > > > >>> > >> >> >> > > >>> > > > > > > > immediate > > > >>> > >> >> >> > > >>> > > > > > > > > > 5.1.5 release to include the > > > changes > > > >>> > >> proposed > > > >>> > >> >> >> with > > > >>> > >> >> >> > > >>> > > > PHOENIX-7106. > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > On Wed, Jan 3, 2024 at 3:08 PM > > > >>> Geoffrey > > > >>> > >> >> Jacoby < > > > >>> > >> >> >> > > >>> > > > > gjac...@apache.org > > > >>> > >> >> >> > > >>> > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > wrote: > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > I agree that data integrity > > > issues > > > >>> > are a > > > >>> > >> >> >> higher > > > >>> > >> >> >> > > >>> priority > > > >>> > >> >> >> > > >>> > > than > > > >>> > >> >> >> > > >>> > > > > > > feature > > > >>> > >> >> >> > > >>> > > > > > > > > > > development, so I also > support > > > the > > > >>> > >> >> decision. > > > >>> > >> >> >> The > > > >>> > >> >> >> > > fact > > > >>> > >> >> >> > > >>> > that > > > >>> > >> >> >> > > >>> > > > > > several > > > >>> > >> >> >> > > >>> > > > > > > of > > > >>> > >> >> >> > > >>> > > > > > > > > the > > > >>> > >> >> >> > > >>> > > > > > > > > > > major remaining 5.2 features > > are > > > >>> > >> currently > > > >>> > >> >> >> being > > > >>> > >> >> >> > > >>> > developed > > > >>> > >> >> >> > > >>> > > in > > > >>> > >> >> >> > > >>> > > > > > > > > > long-running > > > >>> > >> >> >> > > >>> > > > > > > > > > > feature branches also helps, > > as > > > >>> work > > > >>> > can > > > >>> > >> >> >> continue > > > >>> > >> >> >> > > >>> there > > > >>> > >> >> >> > > >>> > at > > > >>> > >> >> >> > > >>> > > > the > > > >>> > >> >> >> > > >>> > > > > > cost > > > >>> > >> >> >> > > >>> > > > > > > > of > > > >>> > >> >> >> > > >>> > > > > > > > > a > > > >>> > >> >> >> > > >>> > > > > > > > > > > rebase later. > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > How does this affect 5.1.4, > > > which > > > >>> is > > > >>> > >> also > > > >>> > >> >> >> listed > > > >>> > >> >> >> > as > > > >>> > >> >> >> > > >>> a Fix > > > >>> > >> >> >> > > >>> > > > > Version > > > >>> > >> >> >> > > >>> > > > > > > for > > > >>> > >> >> >> > > >>> > > > > > > > > > > PHOENIX-7106? From the bug > > > >>> description > > > >>> > >> it > > > >>> > >> >> also > > > >>> > >> >> >> > > sounds > > > >>> > >> >> >> > > >>> > like > > > >>> > >> >> >> > > >>> > > > > 5.1.3 > > > >>> > >> >> >> > > >>> > > > > > > and > > > >>> > >> >> >> > > >>> > > > > > > > > the > > > >>> > >> >> >> > > >>> > > > > > > > > > > forthcoming .4 are affected, > > > >>> since we > > > >>> > >> have > > > >>> > >> >> >> > > >>> server-side > > > >>> > >> >> >> > > >>> > > paging > > > >>> > >> >> >> > > >>> > > > > in > > > >>> > >> >> >> > > >>> > > > > > > 5.1. > > > >>> > >> >> >> > > >>> > > > > > > > > > (Feel > > > >>> > >> >> >> > > >>> > > > > > > > > > > free to move that to a > > separate > > > >>> thread > > > >>> > >> if > > > >>> > >> >> you > > > >>> > >> >> >> > feel > > > >>> > >> >> >> > > it > > > >>> > >> >> >> > > >>> > > should > > > >>> > >> >> >> > > >>> > > > > be a > > > >>> > >> >> >> > > >>> > > > > > > > > > separate > > > >>> > >> >> >> > > >>> > > > > > > > > > > discussion.) Should this be > a > > > >>> blocker > > > >>> > >> for > > > >>> > >> >> >> > releasing > > > >>> > >> >> >> > > >>> > 5.1.4? > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > Geoffrey > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > On Wed, Jan 3, 2024 at > 5:06 PM > > > >>> Kadir > > > >>> > >> >> Ozdemir < > > > >>> > >> >> >> > > >>> > > > > > > > > > > > ka...@gsuite.cloud.apache.org > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > wrote: > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > Being a database, Phoenix > > has > > > to > > > >>> > make > > > >>> > >> >> sure > > > >>> > >> >> >> that > > > >>> > >> >> >> > > the > > > >>> > >> >> >> > > >>> > data > > > >>> > >> >> >> > > >>> > > > > stays > > > >>> > >> >> >> > > >>> > > > > > on > > > >>> > >> >> >> > > >>> > > > > > > > > disk > > > >>> > >> >> >> > > >>> > > > > > > > > > > > intact and its queries > > return > > > >>> > correct > > > >>> > >> >> data. > > > >>> > >> >> >> In > > > >>> > >> >> >> > > this > > > >>> > >> >> >> > > >>> > case, > > > >>> > >> >> >> > > >>> > > > > > Phoenix > > > >>> > >> >> >> > > >>> > > > > > > > > fails > > > >>> > >> >> >> > > >>> > > > > > > > > > > to > > > >>> > >> >> >> > > >>> > > > > > > > > > > > return correct data for > some > > > >>> queries > > > >>> > >> if > > > >>> > >> >> >> their > > > >>> > >> >> >> > > scans > > > >>> > >> >> >> > > >>> > > > > experience > > > >>> > >> >> >> > > >>> > > > > > > > region > > > >>> > >> >> >> > > >>> > > > > > > > > > > > movement. Now that we know > > > these > > > >>> > data > > > >>> > >> >> >> integrity > > > >>> > >> >> >> > > >>> issues > > > >>> > >> >> >> > > >>> > > and > > > >>> > >> >> >> > > >>> > > > > how > > > >>> > >> >> >> > > >>> > > > > > to > > > >>> > >> >> >> > > >>> > > > > > > > > > > reproduce > > > >>> > >> >> >> > > >>> > > > > > > > > > > > them, fixing them should > be > > > our > > > >>> > first > > > >>> > >> >> >> priority. > > > >>> > >> >> >> > > >>> So, I > > > >>> > >> >> >> > > >>> > > fully > > > >>> > >> >> >> > > >>> > > > > > > support > > > >>> > >> >> >> > > >>> > > > > > > > > > this > > > >>> > >> >> >> > > >>> > > > > > > > > > > > proposal. > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > On Wed, Jan 3, 2024 at > > > 10:58 PM > > > >>> > Viraj > > > >>> > >> >> >> Jasani < > > > >>> > >> >> >> > > >>> > > > > > vjas...@apache.org > > > >>> > >> >> >> > > >>> > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > wrote: > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > Hello, > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > I would like to bring > > > >>> PHOENIX-7106 > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > < > > > >>> > >> >> >> > > >>> > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > >>> > >> >> >> > > >>> > > > > > > >>> > >> >> >> > > >>> > > > > > >>> > >> >> >> > > >>> > > > > >>> > >> >> >> > > >>> > > > >>> > >> >> >> > > > > > >>> > >> >> >> > > > > >>> > >> >> >> > > > >>> > >> >> > > > >>> > >> > > > >>> > > > > >>> > > > > > > https://urldefense.com/v3/__https://issues.apache.org/jira/browse/PHOENIX-7106__;!!DCbAVzZNrAf4!FZG5sv55IC1NqItQLY7GKWgUG2Do0gSta01gOiSdd36Dx3XHGtQx4M3c9visVXIt9DctPQzS-orob9vhzrCfVA$ > > > >>> > >> >> >> > > >>> > > > > > > > > to everyone's > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > attention here and brief > > > >>> about the > > > >>> > >> data > > > >>> > >> >> >> > > integrity > > > >>> > >> >> >> > > >>> > > issues > > > >>> > >> >> >> > > >>> > > > > that > > > >>> > >> >> >> > > >>> > > > > > > we > > > >>> > >> >> >> > > >>> > > > > > > > > have > > > >>> > >> >> >> > > >>> > > > > > > > > > > in > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > various coprocessors. > > > >>> Majority of > > > >>> > >> the > > > >>> > >> >> >> issues > > > >>> > >> >> >> > > are > > > >>> > >> >> >> > > >>> > > related > > > >>> > >> >> >> > > >>> > > > to > > > >>> > >> >> >> > > >>> > > > > > the > > > >>> > >> >> >> > > >>> > > > > > > > > fact > > > >>> > >> >> >> > > >>> > > > > > > > > > > that > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > we do not return valid > > > rowkey > > > >>> for > > > >>> > >> >> certain > > > >>> > >> >> >> > > >>> queries. If > > > >>> > >> >> >> > > >>> > > any > > > >>> > >> >> >> > > >>> > > > > > > region > > > >>> > >> >> >> > > >>> > > > > > > > > > moves > > > >>> > >> >> >> > > >>> > > > > > > > > > > in > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > the middle of the scan, > > the > > > >>> HBase > > > >>> > >> >> client > > > >>> > >> >> >> > relies > > > >>> > >> >> >> > > >>> on > > > >>> > >> >> >> > > >>> > the > > > >>> > >> >> >> > > >>> > > > last > > > >>> > >> >> >> > > >>> > > > > > > > > returned > > > >>> > >> >> >> > > >>> > > > > > > > > > > > rowkey > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > and accordingly changes > > the > > > >>> scan > > > >>> > >> >> >> boundaries > > > >>> > >> >> >> > > >>> while the > > > >>> > >> >> >> > > >>> > > > > scanner > > > >>> > >> >> >> > > >>> > > > > > > is > > > >>> > >> >> >> > > >>> > > > > > > > > > > getting > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > reset to continue the > scan > > > >>> > >> operation. > > > >>> > >> >> If > > > >>> > >> >> >> the > > > >>> > >> >> >> > > >>> region > > > >>> > >> >> >> > > >>> > > does > > > >>> > >> >> >> > > >>> > > > > not > > > >>> > >> >> >> > > >>> > > > > > > > move, > > > >>> > >> >> >> > > >>> > > > > > > > > > scan > > > >>> > >> >> >> > > >>> > > > > > > > > > > > is > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > not expected to return > > > invalid > > > >>> > data, > > > >>> > >> >> >> however > > > >>> > >> >> >> > if > > > >>> > >> >> >> > > >>> the > > > >>> > >> >> >> > > >>> > > > region > > > >>> > >> >> >> > > >>> > > > > > > moves > > > >>> > >> >> >> > > >>> > > > > > > > in > > > >>> > >> >> >> > > >>> > > > > > > > > > the > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > middle of ongoing scan > > > >>> operation, > > > >>> > >> scan > > > >>> > >> >> >> would > > > >>> > >> >> >> > > >>> return > > > >>> > >> >> >> > > >>> > > > > > > > > invalid/incorrect > > > >>> > >> >> >> > > >>> > > > > > > > > > > > data > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > causing data integrity > > > issues. > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > Given the critical > nature > > of > > > >>> these > > > >>> > >> >> >> issues, I > > > >>> > >> >> >> > > >>> would > > > >>> > >> >> >> > > >>> > like > > > >>> > >> >> >> > > >>> > > > to > > > >>> > >> >> >> > > >>> > > > > > > > propose > > > >>> > >> >> >> > > >>> > > > > > > > > > that > > > >>> > >> >> >> > > >>> > > > > > > > > > > > we > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > treat this as a high > > > priority > > > >>> for > > > >>> > >> the > > > >>> > >> >> >> > upcoming > > > >>> > >> >> >> > > >>> 5.2.0 > > > >>> > >> >> >> > > >>> > > > > release, > > > >>> > >> >> >> > > >>> > > > > > > and > > > >>> > >> >> >> > > >>> > > > > > > > > not > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > include any other > feature > > or > > > >>> big > > > >>> > >> >> change to > > > >>> > >> >> >> > > master > > > >>> > >> >> >> > > >>> > > branch > > > >>> > >> >> >> > > >>> > > > > > until > > > >>> > >> >> >> > > >>> > > > > > > we > > > >>> > >> >> >> > > >>> > > > > > > > > > merge > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > this. The PR is still > not > > > >>> ready as > > > >>> > >> >> >> additional > > > >>> > >> >> >> > > >>> changes > > > >>> > >> >> >> > > >>> > > are > > > >>> > >> >> >> > > >>> > > > > > still > > > >>> > >> >> >> > > >>> > > > > > > > in > > > >>> > >> >> >> > > >>> > > > > > > > > my > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > local, requiring rebase > > with > > > >>> the > > > >>> > >> >> current > > > >>> > >> >> >> > > master. > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > I would get back to this > > > >>> discuss > > > >>> > >> >> thread as > > > >>> > >> >> >> > soon > > > >>> > >> >> >> > > >>> as > > > >>> > >> >> >> > > >>> > the > > > >>> > >> >> >> > > >>> > > PR > > > >>> > >> >> >> > > >>> > > > > and > > > >>> > >> >> >> > > >>> > > > > > > the > > > >>> > >> >> >> > > >>> > > > > > > > > doc > > > >>> > >> >> >> > > >>> > > > > > > > > > > are > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > updated with the latest > > > >>> findings > > > >>> > so > > > >>> > >> >> far. > > > >>> > >> >> >> The > > > >>> > >> >> >> > > >>> changes > > > >>> > >> >> >> > > >>> > > > > include > > > >>> > >> >> >> > > >>> > > > > > > many > > > >>> > >> >> >> > > >>> > > > > > > > > of > > > >>> > >> >> >> > > >>> > > > > > > > > > > our > > > >>> > >> >> >> > > >>> > > > > > > > > > > > > coproc scanner > > > >>> implementations and > > > >>> > >> >> hence > > > >>> > >> >> >> it > > > >