I think we can also target 5.2.1 very soon, perhaps just next month, with
more CVE fixes and any other fixes if ready.


On Fri, Feb 9, 2024 at 6:39 PM Viraj Jasani <vjas...@apache.org> wrote:

> For 5.2.0, it would be great to focus on the known data integrity issues.
> We can fix non-zk registry, cover a few more CVEs by upgrading third party
> dependencies and stabilize tests. As for the tests, they don’t seem broken,
> but are flaky. I have got multiple builds without any test failures on PR
> for PHOENIX-7106.
>
> If this looks good to you, I can start release preparation next week. What
> do you think, Istvan?
>
> In the meantime, I have 5.1 backport PR open, awaiting good build results
> before committing it.
> Rajeshbabu, would you like to be RM for 5.1.4 once the PR is merged?
>
>
>
> On Fri, Feb 9, 2024 at 11:13 AM Istvan Toth <st...@apache.org> wrote:
>
>> Yes, they basically make the non-ZK registries unusable.
>> (at least the connectionless problems should be fixed.)
>>
>> I hope to have the final fix for those sometime next week.
>>
>> Also have we looked at potential CVE issues on master recently ?
>>
>> I think we should also look at the most flakey tests I linked above,
>> and fix them or at least make sure that they are test issues and not real
>> bugs.
>>
>> Istvan
>>
>>
>>
>> On Fri, Feb 9, 2024 at 6:55 PM Viraj Jasani <vjas...@apache.org> wrote:
>>
>> > Thanks Istvan.
>> > I would like to cut 5.2 branch from master. Do you see non-ZK registry
>> for
>> > MapReduce jobs as blocker for 5.2.0?
>> >
>> >
>> > On Fri, Feb 9, 2024 at 12:24 AM Istvan Toth <st...@apache.org> wrote:
>> >
>> > > ParallelPhoenixConnectionFailureTest.testExecuteQueryChainFailure also
>> > > fails too often, especially when the test host is slow and/or the
>> load is
>> > > high.
>> > > On my fast laptop, I can semi-reliably break it by running
>> > > mvn clean verify -am -pl phoenix-core -DnumForkedUT=20
>> > >
>> > > On Fri, Feb 9, 2024 at 9:19 AM Istvan Toth <st...@apache.org> wrote:
>> > >
>> > > > We're making progress.
>> > > > I can see that Viraj has just landed PHOENIX-7601, and Rajeshbabu
>> has
>> > > > released Omid 1.1.1.
>> > > > Thank you!
>> > > >
>> > > > At the moment, the following outstanding issues are on my radar:
>> > > >
>> > > > https://issues.apache.org/jira/browse/PHOENIX-7191
>> > > > https://issues.apache.org/jira/browse/PHOENIX-7193
>> > > >
>> > > > These are bugs in my non-ZK registry implementation, which were
>> found
>> > > > during HBase 3 work.
>> > > > I have some PRs up, but they may not be complete. I will push for
>> > reviews
>> > > > once I have the HBase 3 tests passing, and possibly updated them
>> based
>> > on
>> > > > that.
>> > > >
>> > > > We also have a number of very flakey tests, see:
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/test_results_analyzer/
>> > > >
>> > > >
>> > > >
>> > > > On Tue, Jan 30, 2024 at 7:09 AM Istvan Toth <st...@cloudera.com>
>> > wrote:
>> > > >
>> > > >> As Viraj wrote, those are just plans.
>> > > >> If HBase 3 won't be released by the time the other features are
>> ready,
>> > > >> then it won't make it into 5.3.
>> > > >> If other major features are ready by that time, then they will be
>> > > >> included. (though we are not aware of any now)
>> > > >>
>> > > >> As for the new major version, in the past Phoenix didn't have a
>> > > >> compatibility module system,
>> > > >> so a new branch was required,  which didn't support older HBases.
>> > Also,
>> > > >> the API changes between HBase 1.x and 2.x were much larger,
>> > > >> The HBase 2 and 3 API are pretty similar, apart from the removal of
>> > > >> deprecated 1.x APIs. (and the protobuf/protocol thing, which
>> requires
>> > a
>> > > >> rather ugly hack).
>> > > >>
>> > > >> I will start the discussion on how we can add HBase 3 support as
>> soon
>> > as
>> > > >> I have a working POC patch.
>> > > >>
>> > > >> We could call 5.3 6.0 instead, after all Phoenix isn't using a
>> strict
>> > > >> semantic versioning, but then 6.0 would also support HBase 2.
>> > > >> If we do not come to a consensus on the version name, we can always
>> > have
>> > > >> a vote on it.
>> > > >>
>> > > >> I think that the main motivation is that the community wants to
>> > maintain
>> > > >> as few branches as possible.
>> > > >>
>> > > >> Istvan
>> > > >>
>> > > >> On Mon, Jan 29, 2024 at 10:02 PM Stephen Jiang <
>> > syuanjiang...@gmail.com
>> > > >
>> > > >> wrote:
>> > > >>
>> > > >>> I am not sure how close HBase 3.0 is.  Even if it is only less
>> than
>> > one
>> > > >>> year away, the adoption would be low at the beginning.  I don't
>> think
>> > > 5.3
>> > > >>> should wait for that.  And traditionally,  Phoenix would have a
>> major
>> > > >>> release to support the HBase major release (4.x for HBase 1.x and
>> 5.x
>> > > for
>> > > >>> HBase 2.x), in this case, we are talking about Phoenix 6.0 for
>> HBase
>> > > 3.0.
>> > > >>>
>> > > >>> Maybe we should adopt the HBase release model: master branch for
>> next
>> > > >>> major
>> > > >>> release (6.0) and branch-5.x branch for next 5.x minor release and
>> > > >>> branch-5.2 for 5.2 minor release.
>> > > >>>
>> > > >>> Thanks
>> > > >>> Stephen
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> On Thu, Jan 25, 2024 at 11:50 PM Viraj Jasani <vjas...@apache.org
>> >
>> > > >>> wrote:
>> > > >>>
>> > > >>> > Sounds good.
>> > > >>> >
>> > > >>> > Planned major changes for 5.3.0:
>> > > >>> >
>> > > >>> > 1. JSON support.
>> > > >>> > 2. HBase 3.0 support.
>> > > >>> > 3. CDC feature (leveraging uncovered global index framework and
>> > JSON
>> > > >>> > support).
>> > > >>> >
>> > > >>> >
>> > > >>> > On Wed, Jan 24, 2024 at 8:22 PM Kadir Ozdemir
>> > > >>> > <kozde...@salesforce.com.invalid> wrote:
>> > > >>> >
>> > > >>> > > I suggest including another major change for 5.3, Phoenix CDC,
>> > > >>> > > https://issues.apache.org/jira/browse/PHOENIX-7001. The PR
>> for
>> > it
>> > > >>> will
>> > > >>> > be
>> > > >>> > > posted soon.
>> > > >>> > >
>> > > >>> > > On Thu, Jan 25, 2024 at 5:35 AM Viraj Jasani <
>> vjas...@apache.org
>> > >
>> > > >>> wrote:
>> > > >>> > >
>> > > >>> > > > Thanks Istvan! I agree with your points. It’s really been a
>> > while
>> > > >>> we
>> > > >>> > are
>> > > >>> > > > talking about releasing 5.2.0 and yet due to bandwidth
>> issues,
>> > > >>> unable
>> > > >>> > to
>> > > >>> > > do
>> > > >>> > > > so.
>> > > >>> > > >
>> > > >>> > > > I agree to getting these fixes out, cut 5.2 and start the
>> > release
>> > > >>> work
>> > > >>> > > and
>> > > >>> > > > in the meantime I also need to prepare 5.1 backport.
>> > > >>> > > >
>> > > >>> > > > For 5.3, let's plan HBase 3.0 support and JSON as major
>> > changes.
>> > > >>> > > >
>> > > >>> > > >
>> > > >>> > > > On Mon, Jan 22, 2024 at 9:17 PM Istvan Toth <
>> st...@apache.org>
>> > > >>> wrote:
>> > > >>> > > >
>> > > >>> > > > > Hi!
>> > > >>> > > > >
>> > > >>> > > > > In my opinion cutting 5.2 now only makes sense IF we DO
>> NOT
>> > > plan
>> > > >>> to
>> > > >>> > > > release
>> > > >>> > > > > the outstanding big features (like JSON) in 5.2. ,
>> otherwise
>> > > it's
>> > > >>> > just
>> > > >>> > > > > extra work to maintain more  branches.
>> > > >>> > > > >
>> > > >>> > > > > Having said that, releasing a 5.2 and 5.1.4 with the data
>> > > >>> integrity
>> > > >>> > > fixes
>> > > >>> > > > > real soon, and then releasing 5.3 in a few months with
>> JSON,
>> > > and
>> > > >>> any
>> > > >>> > > > other
>> > > >>> > > > > outstanding big features
>> > > >>> > > > > that are close to being finished (and HBase 3.0 support,
>> if
>> > > it's
>> > > >>> > ready
>> > > >>> > > by
>> > > >>> > > > > then) would not be a bad idea.
>> > > >>> > > > >
>> > > >>> > > > > On the CLDR side the only outstanding big feature which
>> could
>> > > >>> impact
>> > > >>> > > > > Viraj's integrity work is HBase 3.0 support, and even
>> that is
>> > > >>> only
>> > > >>> > > > because
>> > > >>> > > > > it may require some larger refactors of existing code, not
>> > > >>> because it
>> > > >>> > > > would
>> > > >>> > > > > change the actual behaviour or algorithms.
>> > > >>> > > > >
>> > > >>> > > > > Phoenix used to have several minor releases per year, the
>> > > current
>> > > >>> > state
>> > > >>> > > > of
>> > > >>> > > > > extreme longevity of 5.1 and several big new features
>> being
>> > > >>> added to
>> > > >>> > it
>> > > >>> > > > > (like uncovered indexes) is not ideal.
>> > > >>> > > > > Releasing 5.2 and 5.3 relatively close together could be a
>> > > >>> return to
>> > > >>> > a
>> > > >>> > > > > quicker cadence for minor releases, which could also help
>> > with
>> > > >>> the
>> > > >>> > > public
>> > > >>> > > > > image and adoption of Phoenix.
>> > > >>> > > > >
>> > > >>> > > > > We were talking about releasing 5.2 at least a year ago,
>> and
>> > I
>> > > >>> have
>> > > >>> > > > started
>> > > >>> > > > > working on that then, but then emergencies have come up at
>> > > >>> $dayjob,
>> > > >>> > > and I
>> > > >>> > > > > could not see that through.
>> > > >>> > > > > (So I am in part responsible for the lack of minor
>> releases)
>> > > >>> > > > >
>> > > >>> > > > > regards
>> > > >>> > > > > Istvan
>> > > >>> > > > >
>> > > >>> > > > >
>> > > >>> > > > > On Mon, Jan 22, 2024 at 11:35 PM Viraj Jasani <
>> > > >>> vjas...@apache.org>
>> > > >>> > > > wrote:
>> > > >>> > > > >
>> > > >>> > > > > > Sorry for the late reply.
>> > > >>> > > > > >
>> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense?
>> > > >>> > > > > >
>> > > >>> > > > > > No problem with that. I can cut 5.2 branch by the end of
>> > this
>> > > >>> week
>> > > >>> > or
>> > > >>> > > > at
>> > > >>> > > > > > the start of next week.
>> > > >>> > > > > >
>> > > >>> > > > > > If there is any very big change or feature ready for
>> merge
>> > to
>> > > >>> > master
>> > > >>> > > > > branch
>> > > >>> > > > > > with PR approvals already in place, please do let me
>> know
>> > so
>> > > >>> that I
>> > > >>> > > can
>> > > >>> > > > > > help collaborate on how best we can get it merged
>> without
>> > > >>> impacting
>> > > >>> > > 5.2
>> > > >>> > > > > > release if required. My main motivation was for any big
>> > > change
>> > > >>> to
>> > > >>> > go
>> > > >>> > > > > > through newly introduced tests so that we know that
>> > anything
>> > > >>> > > additional
>> > > >>> > > > > is
>> > > >>> > > > > > not broken, and also to prioritize for upcoming 5.2.0
>> and
>> > > 5.1.4
>> > > >>> > > > releases.
>> > > >>> > > > > > Moreover, there are several PRs getting merged on the
>> > master
>> > > >>> > branch,
>> > > >>> > > we
>> > > >>> > > > > can
>> > > >>> > > > > > continue that as long as they are not very big changes,
>> > which
>> > > >>> might
>> > > >>> > > > > require
>> > > >>> > > > > > significant time to understand any correlation with data
>> > > >>> integrity
>> > > >>> > > > > issues.
>> > > >>> > > > > >
>> > > >>> > > > > > The PR is also ready for review with some additional
>> cases
>> > > >>> fixed
>> > > >>> > last
>> > > >>> > > > > week:
>> > > >>> > > > > >
>> > > >>> > > >
>> > > >>> > >
>> > > >>> >
>> > > >>>
>> > >
>> >
>> https://urldefense.com/v3/__https://github.com/apache/phoenix/pull/1736__;!!DCbAVzZNrAf4!D4OVjUp2EWW2BqhGnBxsapDX_AHsibRphIpoFBWfgRsd3dsAikrFLo6PGxdTzGbSXJJ2fJ0j9mcz3asXMXo$
>> > > >>> > > > > >
>> > > >>> > > > > > Depending on the review bandwidth, I am hopeful we
>> should
>> > be
>> > > >>> good
>> > > >>> > to
>> > > >>> > > > land
>> > > >>> > > > > > them sooner.
>> > > >>> > > > > >
>> > > >>> > > > > >
>> > > >>> > > > > > On Wed, Jan 17, 2024 at 11:31 AM Rushabh Shah
>> > > >>> > > > > > <rushabh.s...@salesforce.com.invalid> wrote:
>> > > >>> > > > > >
>> > > >>> > > > > > > Thank you Viraj for initiating this thread.
>> > > >>> > > > > > >
>> > > >>> > > > > > > > Given the critical nature of these issues, I would
>> like
>> > > to
>> > > >>> > > propose
>> > > >>> > > > > that
>> > > >>> > > > > > > we
>> > > >>> > > > > > > treat this as a high priority for the upcoming 5.2.0
>> > > >>> release, and
>> > > >>> > > not
>> > > >>> > > > > > > include any other feature or big change to master
>> branch
>> > > >>> until we
>> > > >>> > > > merge
>> > > >>> > > > > > > this.
>> > > >>> > > > > > >
>> > > >>> > > > > > > Do you think cutting 5.2.0 now makes sense? This will
>> > > enable
>> > > >>> > other
>> > > >>> > > > > > > developers to merge features into master branch
>> (5.3.0)
>> > and
>> > > >>> you
>> > > >>> > can
>> > > >>> > > > > take
>> > > >>> > > > > > > some more time to make sure we cover all the corner
>> cases
>> > > >>> for the
>> > > >>> > > > data
>> > > >>> > > > > > > integrity issues that you uncovered.
>> > > >>> > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > > > On Fri, Jan 5, 2024 at 6:38 PM Viraj Jasani <
>> > > >>> vjas...@apache.org>
>> > > >>> > > > > wrote:
>> > > >>> > > > > > >
>> > > >>> > > > > > > > Sounds good, thanks Rajeshbabu. I will try to get
>> the
>> > > >>> first PR
>> > > >>> > > out
>> > > >>> > > > > next
>> > > >>> > > > > > > > week and while reviews happen in parallel, will try
>> to
>> > > get
>> > > >>> 5.1
>> > > >>> > PR
>> > > >>> > > > > soon.
>> > > >>> > > > > > > >
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > On Wed, Jan 3, 2024 at 8:49 PM
>> rajeshb...@apache.org <
>> > > >>> > > > > > > > chrajeshbab...@gmail.com> wrote:
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > > Hi Viraj,
>> > > >>> > > > > > > > >
>> > > >>> > > > > > > > > Would be better to include the changes in 5.1.4
>> as
>> > in
>> > > >>> any
>> > > >>> > way
>> > > >>> > > it
>> > > >>> > > > > > will
>> > > >>> > > > > > > > take
>> > > >>> > > > > > > > > at least 3-4 days to complete the omid release.
>> > > >>> > > > > > > > >
>> > > >>> > > > > > > > > Thanks,
>> > > >>> > > > > > > > > Rajeshbabu.
>> > > >>> > > > > > > > >
>> > > >>> > > > > > > > >
>> > > >>> > > > > > > > > On Thu, Jan 4, 2024 at 5:06 AM Viraj Jasani <
>> > > >>> > > vjas...@apache.org>
>> > > >>> > > > > > > wrote:
>> > > >>> > > > > > > > >
>> > > >>> > > > > > > > > > Thank you Kadir and Geoffrey for your replies!!
>> > > >>> > > > > > > > > >
>> > > >>> > > > > > > > > > > How does this affect 5.1.4, which is also
>> listed
>> > as
>> > > >>> a Fix
>> > > >>> > > > > Version
>> > > >>> > > > > > > for
>> > > >>> > > > > > > > > > > PHOENIX-7106?
>> > > >>> > > > > > > > > >
>> > > >>> > > > > > > > > > Yes, it also needs to be ported to 5.1. Once the
>> > > >>> master PR
>> > > >>> > is
>> > > >>> > > > up
>> > > >>> > > > > > for
>> > > >>> > > > > > > > > final
>> > > >>> > > > > > > > > > review, I would start working on the backport
>> PR.
>> > > >>> > > > > > > > > > We just need some more additional testing to
>> ensure
>> > > old
>> > > >>> > > client
>> > > >>> > > > > > (e.g.
>> > > >>> > > > > > > > > 5.1.3)
>> > > >>> > > > > > > > > > is compatible with the new server with the
>> changes.
>> > > >>> > > > > > > > > >
>> > > >>> > > > > > > > > > Hence, yes it is now a blocker for upcoming
>> 5.1.4
>> > as
>> > > >>> well
>> > > >>> > > since
>> > > >>> > > > > > 5.1.4
>> > > >>> > > > > > > > RC
>> > > >>> > > > > > > > > > preparation is still pending (while Omid
>> release is
>> > > in
>> > > >>> > > > progress).
>> > > >>> > > > > > > > > > Otherwise if 5.1.4 was ready for release, I
>> would
>> > > have
>> > > >>> > > proposed
>> > > >>> > > > > > > > immediate
>> > > >>> > > > > > > > > > 5.1.5 release to include the changes proposed
>> with
>> > > >>> > > > PHOENIX-7106.
>> > > >>> > > > > > > > > >
>> > > >>> > > > > > > > > >
>> > > >>> > > > > > > > > > On Wed, Jan 3, 2024 at 3:08 PM Geoffrey Jacoby <
>> > > >>> > > > > gjac...@apache.org
>> > > >>> > > > > > >
>> > > >>> > > > > > > > > wrote:
>> > > >>> > > > > > > > > >
>> > > >>> > > > > > > > > > > I agree that data integrity issues are a
>> higher
>> > > >>> priority
>> > > >>> > > than
>> > > >>> > > > > > > feature
>> > > >>> > > > > > > > > > > development, so I also support the decision.
>> The
>> > > fact
>> > > >>> > that
>> > > >>> > > > > > several
>> > > >>> > > > > > > of
>> > > >>> > > > > > > > > the
>> > > >>> > > > > > > > > > > major remaining 5.2 features are currently
>> being
>> > > >>> > developed
>> > > >>> > > in
>> > > >>> > > > > > > > > > long-running
>> > > >>> > > > > > > > > > > feature branches also helps, as work can
>> continue
>> > > >>> there
>> > > >>> > at
>> > > >>> > > > the
>> > > >>> > > > > > cost
>> > > >>> > > > > > > > of
>> > > >>> > > > > > > > > a
>> > > >>> > > > > > > > > > > rebase later.
>> > > >>> > > > > > > > > > >
>> > > >>> > > > > > > > > > > How does this affect 5.1.4, which is also
>> listed
>> > as
>> > > >>> a Fix
>> > > >>> > > > > Version
>> > > >>> > > > > > > for
>> > > >>> > > > > > > > > > > PHOENIX-7106? From the bug description it also
>> > > sounds
>> > > >>> > like
>> > > >>> > > > > 5.1.3
>> > > >>> > > > > > > and
>> > > >>> > > > > > > > > the
>> > > >>> > > > > > > > > > > forthcoming .4 are affected, since we have
>> > > >>> server-side
>> > > >>> > > paging
>> > > >>> > > > > in
>> > > >>> > > > > > > 5.1.
>> > > >>> > > > > > > > > > (Feel
>> > > >>> > > > > > > > > > > free to move that to a separate thread if you
>> > feel
>> > > it
>> > > >>> > > should
>> > > >>> > > > > be a
>> > > >>> > > > > > > > > > separate
>> > > >>> > > > > > > > > > > discussion.) Should this be a blocker for
>> > releasing
>> > > >>> > 5.1.4?
>> > > >>> > > > > > > > > > >
>> > > >>> > > > > > > > > > > Geoffrey
>> > > >>> > > > > > > > > > >
>> > > >>> > > > > > > > > > >
>> > > >>> > > > > > > > > > > On Wed, Jan 3, 2024 at 5:06 PM Kadir Ozdemir <
>> > > >>> > > > > > > > > > > ka...@gsuite.cloud.apache.org>
>> > > >>> > > > > > > > > > > wrote:
>> > > >>> > > > > > > > > > >
>> > > >>> > > > > > > > > > > > Being a database, Phoenix has to make sure
>> that
>> > > the
>> > > >>> > data
>> > > >>> > > > > stays
>> > > >>> > > > > > on
>> > > >>> > > > > > > > > disk
>> > > >>> > > > > > > > > > > > intact and its queries return correct data.
>> In
>> > > this
>> > > >>> > case,
>> > > >>> > > > > > Phoenix
>> > > >>> > > > > > > > > fails
>> > > >>> > > > > > > > > > > to
>> > > >>> > > > > > > > > > > > return correct data for some queries if
>> their
>> > > scans
>> > > >>> > > > > experience
>> > > >>> > > > > > > > region
>> > > >>> > > > > > > > > > > > movement. Now that we know these data
>> integrity
>> > > >>> issues
>> > > >>> > > and
>> > > >>> > > > > how
>> > > >>> > > > > > to
>> > > >>> > > > > > > > > > > reproduce
>> > > >>> > > > > > > > > > > > them, fixing them should be our first
>> priority.
>> > > >>> So, I
>> > > >>> > > fully
>> > > >>> > > > > > > support
>> > > >>> > > > > > > > > > this
>> > > >>> > > > > > > > > > > > proposal.
>> > > >>> > > > > > > > > > > >
>> > > >>> > > > > > > > > > > > On Wed, Jan 3, 2024 at 10:58 PM Viraj
>> Jasani <
>> > > >>> > > > > > vjas...@apache.org
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > > > wrote:
>> > > >>> > > > > > > > > > > >
>> > > >>> > > > > > > > > > > > > Hello,
>> > > >>> > > > > > > > > > > > >
>> > > >>> > > > > > > > > > > > > I would like to bring PHOENIX-7106
>> > > >>> > > > > > > > > > > > > <
>> > > >>> > > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > >
>> > > >>> > > > >
>> > > >>> > > >
>> > > >>> > >
>> > > >>> >
>> > > >>>
>> > >
>> >
>> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/PHOENIX-7106__;!!DCbAVzZNrAf4!FZG5sv55IC1NqItQLY7GKWgUG2Do0gSta01gOiSdd36Dx3XHGtQx4M3c9visVXIt9DctPQzS-orob9vhzrCfVA$
>> > > >>> > > > > > > > > to everyone's
>> > > >>> > > > > > > > > > > > > attention here and brief about the data
>> > > integrity
>> > > >>> > > issues
>> > > >>> > > > > that
>> > > >>> > > > > > > we
>> > > >>> > > > > > > > > have
>> > > >>> > > > > > > > > > > in
>> > > >>> > > > > > > > > > > > > various coprocessors. Majority of the
>> issues
>> > > are
>> > > >>> > > related
>> > > >>> > > > to
>> > > >>> > > > > > the
>> > > >>> > > > > > > > > fact
>> > > >>> > > > > > > > > > > that
>> > > >>> > > > > > > > > > > > > we do not return valid rowkey for certain
>> > > >>> queries. If
>> > > >>> > > any
>> > > >>> > > > > > > region
>> > > >>> > > > > > > > > > moves
>> > > >>> > > > > > > > > > > in
>> > > >>> > > > > > > > > > > > > the middle of the scan, the HBase client
>> > relies
>> > > >>> on
>> > > >>> > the
>> > > >>> > > > last
>> > > >>> > > > > > > > > returned
>> > > >>> > > > > > > > > > > > rowkey
>> > > >>> > > > > > > > > > > > > and accordingly changes the scan
>> boundaries
>> > > >>> while the
>> > > >>> > > > > scanner
>> > > >>> > > > > > > is
>> > > >>> > > > > > > > > > > getting
>> > > >>> > > > > > > > > > > > > reset to continue the scan operation. If
>> the
>> > > >>> region
>> > > >>> > > does
>> > > >>> > > > > not
>> > > >>> > > > > > > > move,
>> > > >>> > > > > > > > > > scan
>> > > >>> > > > > > > > > > > > is
>> > > >>> > > > > > > > > > > > > not expected to return invalid data,
>> however
>> > if
>> > > >>> the
>> > > >>> > > > region
>> > > >>> > > > > > > moves
>> > > >>> > > > > > > > in
>> > > >>> > > > > > > > > > the
>> > > >>> > > > > > > > > > > > > middle of ongoing scan operation, scan
>> would
>> > > >>> return
>> > > >>> > > > > > > > > invalid/incorrect
>> > > >>> > > > > > > > > > > > data
>> > > >>> > > > > > > > > > > > > causing data integrity issues.
>> > > >>> > > > > > > > > > > > >
>> > > >>> > > > > > > > > > > > > Given the critical nature of these
>> issues, I
>> > > >>> would
>> > > >>> > like
>> > > >>> > > > to
>> > > >>> > > > > > > > propose
>> > > >>> > > > > > > > > > that
>> > > >>> > > > > > > > > > > > we
>> > > >>> > > > > > > > > > > > > treat this as a high priority for the
>> > upcoming
>> > > >>> 5.2.0
>> > > >>> > > > > release,
>> > > >>> > > > > > > and
>> > > >>> > > > > > > > > not
>> > > >>> > > > > > > > > > > > > include any other feature or big change to
>> > > master
>> > > >>> > > branch
>> > > >>> > > > > > until
>> > > >>> > > > > > > we
>> > > >>> > > > > > > > > > merge
>> > > >>> > > > > > > > > > > > > this. The PR is still not ready as
>> additional
>> > > >>> changes
>> > > >>> > > are
>> > > >>> > > > > > still
>> > > >>> > > > > > > > in
>> > > >>> > > > > > > > > my
>> > > >>> > > > > > > > > > > > > local, requiring rebase with the current
>> > > master.
>> > > >>> > > > > > > > > > > > >
>> > > >>> > > > > > > > > > > > > I would get back to this discuss thread as
>> > soon
>> > > >>> as
>> > > >>> > the
>> > > >>> > > PR
>> > > >>> > > > > and
>> > > >>> > > > > > > the
>> > > >>> > > > > > > > > doc
>> > > >>> > > > > > > > > > > are
>> > > >>> > > > > > > > > > > > > updated with the latest findings so far.
>> The
>> > > >>> changes
>> > > >>> > > > > include
>> > > >>> > > > > > > many
>> > > >>> > > > > > > > > of
>> > > >>> > > > > > > > > > > our
>> > > >>> > > > > > > > > > > > > coproc scanner implementations and hence
>> it
>> > > would
>> > > >>> > > require
>> > > >>> > > > > > > > > significant
>> > > >>> > > > > > > > > > > > > review as well.
>> > > >>> > > > > > > > > > > > > It would be great if we can hold on to
>> > merging
>> > > >>> any
>> > > >>> > > > feature
>> > > >>> > > > > or
>> > > >>> > > > > > > big
>> > > >>> > > > > > > > > > > change
>> > > >>> > > > > > > > > > > > to
>> > > >>> > > > > > > > > > > > > master branch until this gets in so as to
>> not
>> > > >>> > > complicate
>> > > >>> > > > > > > > > > > > merging/rebasing.
>> > > >>> > > > > > > > > > > > > Once this is merged to the master branch,
>> I
>> > > would
>> > > >>> > like
>> > > >>> > > to
>> > > >>> > > > > cut
>> > > >>> > > > > > > 5.2
>> > > >>> > > > > > > > > > > branch
>> > > >>> > > > > > > > > > > > > from master and we can move forward with
>> > 5.2.0
>> > > >>> > release.
>> > > >>> > > > > > > > > > > > >
>> > > >>> > > > > > > > > > > > > Please let me know if this looks good or
>> if
>> > you
>> > > >>> have
>> > > >>> > > any
>> > > >>> > > > > > other
>> > > >>> > > > > > > > high
>> > > >>> > > > > > > > > > > > > priority work for 5.2.0.
>> > > >>> > > > > > > > > > > > >
>> > > >>> > > > > > > > > > > >
>> > > >>> > > > > > > > > > >
>> > > >>> > > > > > > > > >
>> > > >>> > > > > > > > >
>> > > >>> > > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > >
>> > > >>> > > > >
>> > > >>> > > >
>> > > >>> > >
>> > > >>> >
>> > > >>>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> *István Tóth* | Sr. Staff Software Engineer
>> > > >> *Email*: st...@cloudera.com
>> > > >> cloudera.com <https://www.cloudera.com>
>> > > >> [image: Cloudera] <https://www.cloudera.com/>
>> > > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera>
>> [image:
>> > > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> > > >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> > > >> ------------------------------
>> > > >> ------------------------------
>> > > >>
>> > > >
>> > >
>> >
>>
>

Reply via email to