[
https://issues.apache.org/jira/browse/HBASE-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629861#comment-14629861
]
Andrew Purtell edited comment on HBASE-12596 at 7/16/15 3:49 PM:
-----------------------------------------------------------------
That's changed since the new rules went into effect for 1.0 and up. Here and
elsewhere we are able to accept new features in 1.x minor revs (branch-1), 0.98
because of its continuation of old style numbering/policy, and master. There
will be a rolling upgrade capable upgrade path from 0.98 to a branch-1 release
with a contiguous feature set.
This is the consequence of adopting semver for 1.0 and up but not earlier. As
0.98 RM I can accept changes that add features on our minor releases (0.98.x)
but not on our patch releases (0.98.x.y) as long as they do not break wire
compatibility. On branch-1 this policy is similar but now the minors are 1.x
and the patch versions are 1.x.y.
We have contributors and users wanting enhancements in 0.98 that are allowed
there. I'm not going to retire 0.98 as RM until we have consensus it should be
EOL.
bq. So if it is in 0.98, we probably want this in 1.0, 1.1, and especially 1.2
as well.
You can ask [~enis], [~ndimiduk], and [~busbey]. I've done this before and have
found Enis does not accept changes for 1.0 that do not fit into the bug fix
bucket. This makes sense as 1.0 operates under the new semver regime. I'd
presume that the case here but I'm not speaking for him. (Not to single Enis
out here, I also presume Nick and Sean will make similar choices, but there's a
history of decisions for branch-1.0 that we can talk about.)
The old policy for 0.98 (wire compat is the only dealbreaker) came about as a
result of discussions leading up to the 0.98.0 release. Consensus and user
needs evolve though. Java binary compat became a pain point for downstreamers
and Phoenix so now I also check that as part of the RC process. Consensus can
evolve again, to a new policy that disallows anything but bug fixes to 0.98. If
so we should as a courtesy have a discussion with 0.98 users on the
implications. Already the code divergence between 0.98 and branch-1 is
significant enough to make all but trivial changes an effort to backport. If we
change the 0.98 accept criteria to only allow what can be committed to all 1.x
branches, this shuts down all enhancements to 0.98 because of change policies
applied to the 1.x branches by their RMs, and that divergence will get a lot
bigger. At that point I would advocate that 0.98 is EOLed and retire as its RM
since it would no longer need dedicated attention. At some point those two
actions should happen anyway.
Edit: Consolidated two posts and at-mentions.
was (Author: apurtell):
That's changed since the new rules went into effect for 1.0 and up. Here and
elsewhere we are able to accept new features in 1.x minor revs (branch-1), 0.98
because of its continuation of old style numbering/policy, and master. There
will be a rolling upgrade capable upgrade path from 0.98 to a branch-1 release
with a contiguous feature set.
This is the consequence of adopting semver for 1.0 and up but not earlier. As
0.98 RM I can accept changes that add features on our minor releases (0.98.x)
but not on our patch releases (0.98.x.y) as long as they do not break wire
compatibility. On branch-1 this policy is similar but now the minors are 1.x
and the patch versions are 1.x.y.
We have contributors and users wanting enhancements in 0.98 that are allowed
there. I'm not going to retire 0.98 as RM until we have consensus it should be
EOL.
bq. So if it is in 0.98, we probably want this in 1.0, 1.1, and especially 1.2
as well.
You can ask [~enis], [~ndimiduk], and [~busbey]. I've done this before and have
found Enis does not accept changes for 1.0 that do not fit into the bug fix
bucket. This makes sense as 1.0 operates under the new semver regime. I'd
presume that the case here but I'm not speaking for him. (Not to single Enis
out here, I also presume Nick and Sean will make similar choices.)
The old policy for 0.98 (wire compat is the only dealbreaker) came about as a
result of discussions leading up to the 0.98.0 release. Consensus and user
needs evolve though. Java binary compat became a pain point for downstreamers
and Phoenix so now I also check that as part of the RC process. Consensus can
evolve again, to a new policy that disallows anything but bug fixes to 0.98. If
so we should as a courtesy have a discussion with 0.98 users on the
implications. Already the code divergence between 0.98 and branch-1 is
significant enough to make all but trivial changes an effort to backport. If we
change the 0.98 accept criteria to only allow what can be committed to all 1.x
branches, this shuts down all enhancements to 0.98 because of change policies
applied to the 1.x branches by their RMs, and that divergence will get a lot
bigger. At that point I would advocate that 0.98 is EOLed and retire as its RM
since it would no longer need dedicated attention. At some point those two
actions should happen anyway.
Edit: Consolidated two posts and at-mentions.
> bulkload needs to follow locality
> ---------------------------------
>
> Key: HBASE-12596
> URL: https://issues.apache.org/jira/browse/HBASE-12596
> Project: HBase
> Issue Type: Improvement
> Components: HFile, regionserver
> Affects Versions: 0.98.8
> Environment: hadoop-2.3.0, hbase-0.98.8, jdk1.7
> Reporter: Victor Xu
> Assignee: Victor Xu
> Fix For: 2.0.0, 0.98.14, 1.3.0
>
> Attachments: HBASE-12596-0.98-v1.patch, HBASE-12596-0.98-v2.patch,
> HBASE-12596-0.98-v3.patch, HBASE-12596-0.98-v4.patch,
> HBASE-12596-0.98-v5.patch, HBASE-12596-0.98-v6.patch,
> HBASE-12596-branch-1-v1.patch, HBASE-12596-branch-1-v2.patch,
> HBASE-12596-master-v1.patch, HBASE-12596-master-v2.patch,
> HBASE-12596-master-v3.patch, HBASE-12596-master-v4.patch,
> HBASE-12596-master-v5.patch, HBASE-12596-master-v6.patch, HBASE-12596.patch
>
>
> Normally, we have 2 steps to perform a bulkload: 1. use a job to write HFiles
> to be loaded; 2. Move these HFiles to the right hdfs directory. However, the
> locality could be loss during the first step. Why not just write the HFiles
> directly into the right place? We can do this easily because
> StoreFile.WriterBuilder has the "withFavoredNodes" method, and we just need
> to call it in HFileOutputFormat's getNewWriter().
> This feature is enabled by default, and we could use
> 'hbase.bulkload.locality.sensitive.enabled=false' to disable it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
