On 11/15/17 10:34 AM, Andrew Wang wrote:
Hi Junping,
On Wed, Nov 15, 2017 at 1:37 AM, Junping Du <j...@hortonworks.com> wrote:
3. Beside incompatibilities, there is also possible to have performance
regressions (lower throughput, higher latency, slower job running, bigger
memory footprint or even memory leaking, etc.) for new hadoop releases.
While the performance impact of migration (if any) could be neglectable to
some users, other users could be very sensitive and wish to roll back if it
happens on their production cluster.
Yes, bugs exist. I won't claim that 3.0.0 is bug-free. All new releases
can potentially introduce new bugs.
However, I don't think rollback is the solution. In my experience, users
rarely rollback since it's so disruptive and causes data loss. It's much
more common that they patch and upgrade. With that in mind, I'd rather we
spend our effort on making 3.0.x high-quality vs. making it easier to
rollback.
The root of my concern in announcing a "bridge release" is that it
discourages users from upgrading to 3.0.0 until a bridge release is out. I
strongly believe the level of quality provided by 3.0.0 is at least equal
to new 2.x minor releases, given our extended testing and integration
process, and we don't have bridge releases for 2.x.
This is why I asked for a list of known issues with 2.x -> 3.0 upgrades,
that would necessitate a bridge release. Arun raised a concern about NM
rollback. Are there any other *known* issues?
While going over the JACC report as part of YARN-6142, I filed
HADOOP-14534, MAPREDUCE-6902, and YARN-6717 to document the major issues
that I ran across. I think we found one or two other JIRAs which we
marked as incompatible as part of this investigation.
The protobuf changes should be forward compatible going from 2.8.0 to 3.0.0.
YARN-6798 should fix the NM state store versioning when upgrading from
2.9.0 to 3.0.0. 2.8.0 to 3.0.0 could have an issue if the relevant
features are enabled. (queued containers, work-preserving NM restart
w/AMRMProxy).
-Ray
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
