[jira] [Created] (HADOOP-15121) Encounter NullPointerException when using DecayRpcScheduler

[Tao Jie (JIRA)]

Tao Jie created HADOOP-15121:


 Summary: Encounter NullPointerException when using 
DecayRpcScheduler
 Key: HADOOP-15121
 URL: https://issues.apache.org/jira/browse/HADOOP-15121
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Tao Jie


I set ipc.8020.scheduler.impl to org.apache.hadoop.ipc.DecayRpcScheduler, but 
got excetion in namenode:
{code}
2017-12-15 15:26:34,662 ERROR impl.MetricsSourceAdapter 
(MetricsSourceAdapter.java:getMetrics(202)) - Error getting metrics from source 
DecayRpcSchedulerMetrics2.ipc.8020
java.lang.NullPointerException
at 
org.apache.hadoop.ipc.DecayRpcScheduler$MetricsProxy.getMetrics(DecayRpcScheduler.java:781)
at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:199)
at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:182)
at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:155)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(DefaultMBeanServerInterceptor.java:333)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:319)
at 
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:66)
at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:222)
at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:100)
at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:268)
at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:233)
at 
org.apache.hadoop.ipc.DecayRpcScheduler$MetricsProxy.registerMetrics2Source(DecayRpcScheduler.java:709)
at 
org.apache.hadoop.ipc.DecayRpcScheduler$MetricsProxy.(DecayRpcScheduler.java:685)
at 
org.apache.hadoop.ipc.DecayRpcScheduler$MetricsProxy.getInstance(DecayRpcScheduler.java:693)
at 
org.apache.hadoop.ipc.DecayRpcScheduler.(DecayRpcScheduler.java:236)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.apache.hadoop.ipc.CallQueueManager.createScheduler(CallQueueManager.java:102)
at 
org.apache.hadoop.ipc.CallQueueManager.(CallQueueManager.java:76)
at org.apache.hadoop.ipc.Server.(Server.java:2612)
at org.apache.hadoop.ipc.RPC$Server.(RPC.java:958)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server.(ProtobufRpcEngine.java:374)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:349)
at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:800)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.(NameNodeRpcServer.java:415)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createRpcServer(NameNode.java:755)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:697)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:905)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:884)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1610)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1678)
{code}
It seems that {{metricsProxy}} in DecayRpcScheduler should initiate its 
{{delegate}} field in its Initialization method




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Missing some trunk commit history

[Eric Yang]

Hi Chris,

I am looking for a way to reduce time spent on testing latest commits.
When trunk broke, it is usually someone didn't do a full build to test the
patch.  If a feature merge results in 100+ commit additions.  It is
difficult to tell where breakage occurred.  A single merge point is easier
to isolate the problem for other people who did not do the merge.  If I did
a git pull, and the last tracking hash code is 5 pages beyond git log.  It
can cause panic like I did this morning, and quite exhausting to find the
actual commit that broke trunk and attempt to test the 100+ commits
individually with full build to isolate the problem.  People who did the
feature merge is likely already did the full build test to ensure they
didn't break trunk, but there is no easy indicator where the rebase start
and ends.  Therefore, other people will have to spend extra time to test
each commit individually.  It reduces the productivity for me to prove that
my pre-commit patch unit test failure was caused by other's check in.  I
lost the entire day to isolate trunk build breakage for node manager was
caused by YARN-7381, and I was only able to find this base on github method
to sort commits by date instead of git log approach of showing commit
histories.  If I was testing this one by one based on git log, then I am
probably not done testing yet.  If we can propose to use merge without
rebase for trunk, it might be more efficient for analyze bugs for
pre-commit builds.

regards,
Eric

On Thu, Dec 14, 2017 at 6:52 PM, Chris Douglas  wrote:

> Eric-
>
> What problem are you trying to solve? Most of us understand how git works,
> you can omit that. -C
>
> On Thu, Dec 14, 2017 at 6:31 PM Eric Yang  wrote:
>
> > We are currently requesting committer to commit code base on:
> > https://wiki.apache.org/hadoop/HowToCommit
> >
> > To set branch.autosetuprebase always:
> >
> > Base on the current preference, the history is linear, and it is
> described
> > in this graph as Rebase and Merge:
> >
> >
> > https://wac-cdn.atlassian.com/dam/jcr:df39b1f1-2686-4ee5-
> 90bf-9836783342ce/10.svg?cdnVersion=iq
> >
> > It could cause a false alarm on blaming the wrong person for trunk
> > breakage because it takes more time to iterate through all commits from
> > feature branch, while the recent commits (blue dots), are much further
> back
> > in history base on the rebase.  If it was only one merge commit, it would
> > be faster to skip through the entire branch and find recent breakages.
> >
> > When there are several feature branches merged in short period of time,
> > the extra work done to check history revision of branches took much more
> > time.  This is a pain point for people that care about trunk stability
> but
> > can’t afford all day to run full build base on each commit to isolate the
> > breakage.
> >
> > I understand your usage for looking at multiple branches to find a commit
> > to make sure maintenance branches have the proper commits or backport.
> > Rebase + merge works best for maintenance branches.  However, I am not
> > convinced that rebase + merge strategy is the efficient way to manage
> trunk
> > stability.  Is there be a better way to manage this?  Probably, we can
> > recommend trunk to use merge without rebase, but maintenance branches
> apply
> > rebase + merge strategy.  Thoughts?
> >
> > regards,
> > Eric
> >
> > On 12/14/17, 5:16 PM, "Chris Douglas"  wrote:
> >
> > I'm sorry, I literally don't understand what you've written. What do
> > clicks
> > on github have to do with merges?
> >
> > Are you talking about git bisect, where one would first identify the
> > branch
> > where the error was introduced, then run a second regression over the
> > feature branch? With similar semantics for blame?
> >
> > Again, I'd rather have the history of the branch, with rebases prior
> to
> > merge to ensure that feature branches don't create particularly
> > complicated
> > graphs.
> >
> > Perhaps I haven't understood the problem you're solving. The thread
> > started
> > with confusion over dates. Is that the problem? Or that rebases
> create
> > intermediate states that never existed on the branch (due to
> > conflicts),
> > and that complicates analysis? -C
> >
> > On Thu, Dec 14, 2017 at 2:31 PM Eric Yang 
> > wrote:
> >
> > > When details are rebased, the number of entries to test through the
> > linear
> > > history is much more than a merge point to isolate where the error
> > might
> > > have occurred.  It is similar to traverse a tree structure, for
> each
> > > branch, there are n branches to walk through.  If we can know where
> > the
> > > problem is before traverse to individual branches.  It can
> expertise
> > the
> > > process to find the root cause.  IMHO, I think the number of clicks
> > between
> > > pagination vs drop down on github 

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

[Weiwei Yang]

Hi Andrew

Congratulations on the 3.0 GA, another mile stone of Hadoop history. Thanks for 
driving this release, excellent work!
Took a look at the release notes, also seems nice. But one question, who is 
maintaining https://wiki.apache.org/hadoop/PoweredBy? Some of info is quite out 
of date, would like to know how to update.

Thanks

--
Weiwei

On 15 Dec 2017, 2:45 AM +0800, Andrew Wang , wrote:
Hi all,

I'm pleased to announce that Apache Hadoop 3.0.0 is generally available
(GA).

3.0.0 GA consists of 302 bug fixes, improvements, and other enhancements
since 3.0.0-beta1. This release marks a point of quality and stability for
the 3.0.0 release line, and users of earlier 3.0.0-alpha and -beta releases
are encouraged to upgrade.

Looking back, 3.0.0 GA is the culmination of over a year of work on the
3.0.0 line, starting with 3.0.0-alpha1 which was released in September
2016. Altogether, 3.0.0 incorporates 6,242 changes since 2.7.0.

Users are encouraged to read the overview of major changes
 in 3.0.0. The GA release
notes

Re: Missing some trunk commit history

[Chris Douglas]

Eric-

What problem are you trying to solve? Most of us understand how git works,
you can omit that. -C

On Thu, Dec 14, 2017 at 6:31 PM Eric Yang  wrote:

> We are currently requesting committer to commit code base on:
> https://wiki.apache.org/hadoop/HowToCommit
>
> To set branch.autosetuprebase always:
>
> Base on the current preference, the history is linear, and it is described
> in this graph as Rebase and Merge:
>
>
> https://wac-cdn.atlassian.com/dam/jcr:df39b1f1-2686-4ee5-90bf-9836783342ce/10.svg?cdnVersion=iq
>
> It could cause a false alarm on blaming the wrong person for trunk
> breakage because it takes more time to iterate through all commits from
> feature branch, while the recent commits (blue dots), are much further back
> in history base on the rebase.  If it was only one merge commit, it would
> be faster to skip through the entire branch and find recent breakages.
>
> When there are several feature branches merged in short period of time,
> the extra work done to check history revision of branches took much more
> time.  This is a pain point for people that care about trunk stability but
> can’t afford all day to run full build base on each commit to isolate the
> breakage.
>
> I understand your usage for looking at multiple branches to find a commit
> to make sure maintenance branches have the proper commits or backport.
> Rebase + merge works best for maintenance branches.  However, I am not
> convinced that rebase + merge strategy is the efficient way to manage trunk
> stability.  Is there be a better way to manage this?  Probably, we can
> recommend trunk to use merge without rebase, but maintenance branches apply
> rebase + merge strategy.  Thoughts?
>
> regards,
> Eric
>
> On 12/14/17, 5:16 PM, "Chris Douglas"  wrote:
>
> I'm sorry, I literally don't understand what you've written. What do
> clicks
> on github have to do with merges?
>
> Are you talking about git bisect, where one would first identify the
> branch
> where the error was introduced, then run a second regression over the
> feature branch? With similar semantics for blame?
>
> Again, I'd rather have the history of the branch, with rebases prior to
> merge to ensure that feature branches don't create particularly
> complicated
> graphs.
>
> Perhaps I haven't understood the problem you're solving. The thread
> started
> with confusion over dates. Is that the problem? Or that rebases create
> intermediate states that never existed on the branch (due to
> conflicts),
> and that complicates analysis? -C
>
> On Thu, Dec 14, 2017 at 2:31 PM Eric Yang 
> wrote:
>
> > When details are rebased, the number of entries to test through the
> linear
> > history is much more than a merge point to isolate where the error
> might
> > have occurred.  It is similar to traverse a tree structure, for each
> > branch, there are n branches to walk through.  If we can know where
> the
> > problem is before traverse to individual branches.  It can expertise
> the
> > process to find the root cause.  IMHO, I think the number of clicks
> between
> > pagination vs drop down on github branch selection, the later seems
> more
> > work, but it is usually less clicks for feature branches that lived
> for a
> > couple months.
> >
> > Regards,
> > Eric
> >
> > On 12/14/17, 2:09 PM, "Chris Douglas"  wrote:
> >
> > I'd rather have the history. Otherwise tools like blame point
> only to
> > a parent/umbrella JIRA, not the issue where the change was
> discussed.
> >
> > We can force a merge commit so it's clear the branch was
> developed
> > outside the mainline. -C
> >
> >
> > On Thu, Dec 14, 2017 at 1:18 PM, Eric Yang <
> ey...@hortonworks.com>
> > wrote:
> > > +1 on squash merge to keep history compressed.  The rebase +
> merge
> > contains good deals, but it is easy to get confused for people that
> doesn’t
> > know about the rebase option is turned on by default for Hadoop.
> > >
> > > Regards,
> > > Eric
> > >
> > > On 12/14/17, 12:06 PM, "Arun Suresh" 
> wrote:
> > >
> > > Another option - atleast for feature branches is to maybe
> squash
> > merge -
> > > this way we see it as a single commit ? Although we will
> loose
> > the feature
> > > branch history (I am ok with that though)
> > >
> > > Cheers
> > > -Arun
> > >
> > > On Thu, Dec 14, 2017 at 11:32 AM, Eric Yang <
> > ey...@hortonworks.com> wrote:
> > >
> > > > Thank you for the pointer.  I guess all merge are done
> using
> > rebase +
> > > > merge.  This is the reason that timeline is out of order.
> > > >
> 

Re: Missing some trunk commit history

[Eric Yang]

We are currently requesting committer to commit code base on:
https://wiki.apache.org/hadoop/HowToCommit

To set branch.autosetuprebase always:

Base on the current preference, the history is linear, and it is described in 
this graph as Rebase and Merge:

https://wac-cdn.atlassian.com/dam/jcr:df39b1f1-2686-4ee5-90bf-9836783342ce/10.svg?cdnVersion=iq

It could cause a false alarm on blaming the wrong person for trunk breakage 
because it takes more time to iterate through all commits from feature branch, 
while the recent commits (blue dots), are much further back in history base on 
the rebase.  If it was only one merge commit, it would be faster to skip 
through the entire branch and find recent breakages.

When there are several feature branches merged in short period of time, the 
extra work done to check history revision of branches took much more time.  
This is a pain point for people that care about trunk stability but can’t 
afford all day to run full build base on each commit to isolate the breakage.  

I understand your usage for looking at multiple branches to find a commit to 
make sure maintenance branches have the proper commits or backport.  Rebase + 
merge works best for maintenance branches.  However, I am not convinced that 
rebase + merge strategy is the efficient way to manage trunk stability.  Is 
there be a better way to manage this?  Probably, we can recommend trunk to use 
merge without rebase, but maintenance branches apply rebase + merge strategy.  
Thoughts?

regards,
Eric

On 12/14/17, 5:16 PM, "Chris Douglas"  wrote:

I'm sorry, I literally don't understand what you've written. What do clicks
on github have to do with merges?

Are you talking about git bisect, where one would first identify the branch
where the error was introduced, then run a second regression over the
feature branch? With similar semantics for blame?

Again, I'd rather have the history of the branch, with rebases prior to
merge to ensure that feature branches don't create particularly complicated
graphs.

Perhaps I haven't understood the problem you're solving. The thread started
with confusion over dates. Is that the problem? Or that rebases create
intermediate states that never existed on the branch (due to conflicts),
and that complicates analysis? -C

On Thu, Dec 14, 2017 at 2:31 PM Eric Yang  wrote:

> When details are rebased, the number of entries to test through the linear
> history is much more than a merge point to isolate where the error might
> have occurred.  It is similar to traverse a tree structure, for each
> branch, there are n branches to walk through.  If we can know where the
> problem is before traverse to individual branches.  It can expertise the
> process to find the root cause.  IMHO, I think the number of clicks 
between
> pagination vs drop down on github branch selection, the later seems more
> work, but it is usually less clicks for feature branches that lived for a
> couple months.
>
> Regards,
> Eric
>
> On 12/14/17, 2:09 PM, "Chris Douglas"  wrote:
>
> I'd rather have the history. Otherwise tools like blame point only to
> a parent/umbrella JIRA, not the issue where the change was discussed.
>
> We can force a merge commit so it's clear the branch was developed
> outside the mainline. -C
>
>
> On Thu, Dec 14, 2017 at 1:18 PM, Eric Yang 
> wrote:
> > +1 on squash merge to keep history compressed.  The rebase + merge
> contains good deals, but it is easy to get confused for people that 
doesn’t
> know about the rebase option is turned on by default for Hadoop.
> >
> > Regards,
> > Eric
> >
> > On 12/14/17, 12:06 PM, "Arun Suresh"  wrote:
> >
> > Another option - atleast for feature branches is to maybe squash
> merge -
> > this way we see it as a single commit ? Although we will loose
> the feature
> > branch history (I am ok with that though)
> >
> > Cheers
> > -Arun
> >
> > On Thu, Dec 14, 2017 at 11:32 AM, Eric Yang <
> ey...@hortonworks.com> wrote:
> >
> > > Thank you for the pointer.  I guess all merge are done using
> rebase +
> > > merge.  This is the reason that timeline is out of order.
> > >
> > > Would it be more useful to merge without rebasing for feature
> branch merge
> > > to avoid timeline confusions?  The argument for not rebasing,
> it would be
> > > easier to find the root cause of trunk failure was due to
> merge or some
> > > recent commits.
> > >
> > > Regards,
> >  

[jira] [Created] (HADOOP-15120) AzureNativeFileSystemStore

[Yin Huai (JIRA)]

Yin Huai created HADOOP-15120:
-

 Summary: AzureNativeFileSystemStore
 Key: HADOOP-15120
 URL: https://issues.apache.org/jira/browse/HADOOP-15120
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Yin Huai






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15119) AzureNativeFileSystemStore's rename swallow InterruptedExceptions

[Yin Huai (JIRA)]

Yin Huai created HADOOP-15119:
-

 Summary: AzureNativeFileSystemStore's rename swallow 
InterruptedExceptions
 Key: HADOOP-15119
 URL: https://issues.apache.org/jira/browse/HADOOP-15119
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs/azure
Reporter: Yin Huai


AzureNativeFileSystemStore's rename calls 
[waitForCopyToComplete|https://github.com/apache/hadoop/blob/d69b7358b65128197b4c0fe5ef3c02f3d59864b3/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java#L2742],
 which swallow InterruptedExceptions and prevent the current thread being from 
interrupted. Once we catch the exception, it will be nice to call 
Thread.currentThread().interrupt(). So, if this thread is blocked at a later 
time, an InterruptedException will be properly thrown.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Missing some trunk commit history

[Chris Douglas]

I'm sorry, I literally don't understand what you've written. What do clicks
on github have to do with merges?

Are you talking about git bisect, where one would first identify the branch
where the error was introduced, then run a second regression over the
feature branch? With similar semantics for blame?

Again, I'd rather have the history of the branch, with rebases prior to
merge to ensure that feature branches don't create particularly complicated
graphs.

Perhaps I haven't understood the problem you're solving. The thread started
with confusion over dates. Is that the problem? Or that rebases create
intermediate states that never existed on the branch (due to conflicts),
and that complicates analysis? -C

On Thu, Dec 14, 2017 at 2:31 PM Eric Yang  wrote:

> When details are rebased, the number of entries to test through the linear
> history is much more than a merge point to isolate where the error might
> have occurred.  It is similar to traverse a tree structure, for each
> branch, there are n branches to walk through.  If we can know where the
> problem is before traverse to individual branches.  It can expertise the
> process to find the root cause.  IMHO, I think the number of clicks between
> pagination vs drop down on github branch selection, the later seems more
> work, but it is usually less clicks for feature branches that lived for a
> couple months.
>
> Regards,
> Eric
>
> On 12/14/17, 2:09 PM, "Chris Douglas"  wrote:
>
> I'd rather have the history. Otherwise tools like blame point only to
> a parent/umbrella JIRA, not the issue where the change was discussed.
>
> We can force a merge commit so it's clear the branch was developed
> outside the mainline. -C
>
>
> On Thu, Dec 14, 2017 at 1:18 PM, Eric Yang 
> wrote:
> > +1 on squash merge to keep history compressed.  The rebase + merge
> contains good deals, but it is easy to get confused for people that doesn’t
> know about the rebase option is turned on by default for Hadoop.
> >
> > Regards,
> > Eric
> >
> > On 12/14/17, 12:06 PM, "Arun Suresh"  wrote:
> >
> > Another option - atleast for feature branches is to maybe squash
> merge -
> > this way we see it as a single commit ? Although we will loose
> the feature
> > branch history (I am ok with that though)
> >
> > Cheers
> > -Arun
> >
> > On Thu, Dec 14, 2017 at 11:32 AM, Eric Yang <
> ey...@hortonworks.com> wrote:
> >
> > > Thank you for the pointer.  I guess all merge are done using
> rebase +
> > > merge.  This is the reason that timeline is out of order.
> > >
> > > Would it be more useful to merge without rebasing for feature
> branch merge
> > > to avoid timeline confusions?  The argument for not rebasing,
> it would be
> > > easier to find the root cause of trunk failure was due to
> merge or some
> > > recent commits.
> > >
> > > Regards,
> > > Eric
> > >
> > > From: Sunil G 
> > > Date: Thursday, December 14, 2017 at 11:11 AM
> > > To: Eric Yang 
> > > Cc: Hadoop Common 
> > > Subject: Re: Missing some trunk commit history
> > >
> > > Hi Eric.
> > >
> > > A branch merge has happened during that time, and hence you
> might have
> > > seen some old commits from that branch. If you go down
> further, you could
> > > see those commits.
> > >
> > > Copied from my git log:
> > >
> > > commit 40b0045ebe0752cd3d1d09be00acbabdea983799
> > > Author: Weiwei Yang >
> > > Date:   Wed Dec 6 17:52:41 2017 +0800
> > >
> > > YARN-7610. Extend Distributed Shell to support launching
> job with
> > > opportunistic containers. Contributed by Weiwei Yang.
> > >
> > > commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f
> > > Author: Xiao Chen >
> > > Date:   Tue Dec 5 20:48:02 2017 -0800
> > >
> > > HDFS-12872. EC Checksum broken when BlockAccessToken is
> enabled.
> > >
> > > commit 05c347fe51c01494ed8110f8f116a01c90205f13
> > > Author: Weiwei Yang >
> > > Date:   Wed Dec 6 12:21:52 2017 +0800
> > >
> > > YARN-7611. Node manager web UI should display container
> type in
> > > containers page. Contributed by Weiwei Yang.
> > >
> > > commit 73b86979d661f4ad56fcfc3a05a403dfcb2a860e
> > > Author: Kai Zheng  > > inc.com>>
> > > Date:   Wed Dec 6 12:01:36 2017 +0800
> > >
> 

Re: Missing some trunk commit history

[Eric Yang]

When details are rebased, the number of entries to test through the linear 
history is much more than a merge point to isolate where the error might have 
occurred.  It is similar to traverse a tree structure, for each branch, there 
are n branches to walk through.  If we can know where the problem is before 
traverse to individual branches.  It can expertise the process to find the root 
cause.  IMHO, I think the number of clicks between pagination vs drop down on 
github branch selection, the later seems more work, but it is usually less 
clicks for feature branches that lived for a couple months.

Regards,
Eric

On 12/14/17, 2:09 PM, "Chris Douglas"  wrote:

I'd rather have the history. Otherwise tools like blame point only to
a parent/umbrella JIRA, not the issue where the change was discussed.

We can force a merge commit so it's clear the branch was developed
outside the mainline. -C


On Thu, Dec 14, 2017 at 1:18 PM, Eric Yang  wrote:
> +1 on squash merge to keep history compressed.  The rebase + merge 
contains good deals, but it is easy to get confused for people that doesn’t 
know about the rebase option is turned on by default for Hadoop.
>
> Regards,
> Eric
>
> On 12/14/17, 12:06 PM, "Arun Suresh"  wrote:
>
> Another option - atleast for feature branches is to maybe squash 
merge -
> this way we see it as a single commit ? Although we will loose the 
feature
> branch history (I am ok with that though)
>
> Cheers
> -Arun
>
> On Thu, Dec 14, 2017 at 11:32 AM, Eric Yang  
wrote:
>
> > Thank you for the pointer.  I guess all merge are done using rebase 
+
> > merge.  This is the reason that timeline is out of order.
> >
> > Would it be more useful to merge without rebasing for feature 
branch merge
> > to avoid timeline confusions?  The argument for not rebasing, it 
would be
> > easier to find the root cause of trunk failure was due to merge or 
some
> > recent commits.
> >
> > Regards,
> > Eric
> >
> > From: Sunil G 
> > Date: Thursday, December 14, 2017 at 11:11 AM
> > To: Eric Yang 
> > Cc: Hadoop Common 
> > Subject: Re: Missing some trunk commit history
> >
> > Hi Eric.
> >
> > A branch merge has happened during that time, and hence you might 
have
> > seen some old commits from that branch. If you go down further, you 
could
> > see those commits.
> >
> > Copied from my git log:
> >
> > commit 40b0045ebe0752cd3d1d09be00acbabdea983799
> > Author: Weiwei Yang >
> > Date:   Wed Dec 6 17:52:41 2017 +0800
> >
> > YARN-7610. Extend Distributed Shell to support launching job 
with
> > opportunistic containers. Contributed by Weiwei Yang.
> >
> > commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f
> > Author: Xiao Chen >
> > Date:   Tue Dec 5 20:48:02 2017 -0800
> >
> > HDFS-12872. EC Checksum broken when BlockAccessToken is enabled.
> >
> > commit 05c347fe51c01494ed8110f8f116a01c90205f13
> > Author: Weiwei Yang >
> > Date:   Wed Dec 6 12:21:52 2017 +0800
> >
> > YARN-7611. Node manager web UI should display container type in
> > containers page. Contributed by Weiwei Yang.
> >
> > commit 73b86979d661f4ad56fcfc3a05a403dfcb2a860e
> > Author: Kai Zheng 
 > inc.com>>
> > Date:   Wed Dec 6 12:01:36 2017 +0800
> >
> > HADOOP-15039. Move SemaphoredDelegatingExecutor to 
hadoop-common.
> > Contributed by Genmao Yu
> >
> > commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b
> > Author: Akira Ajisaka 
>
> > Date:   Wed Dec 6 11:40:33 2017 +0900
> >
> > HDFS-12889. Router UI is missing robots.txt file. Contributed by
> > Bharat Viswanadham.
> >
> > commit 0311cf05358cd75388f48f048c44fba52ec90f00
> > Author: Wangda Tan >
> > Date:   Tue Dec 5 13:09:49 2017 -0800
> >
> > YARN-7381. Enable the configuration: 
yarn.nodemanager.log-container-debug-info.enabled
> > by default in yarn-default.xml. (Xuan Gong via wangda)
> >
> > Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
> >
> > commit 

Re: Missing some trunk commit history

[Chris Douglas]

I'd rather have the history. Otherwise tools like blame point only to
a parent/umbrella JIRA, not the issue where the change was discussed.

We can force a merge commit so it's clear the branch was developed
outside the mainline. -C


On Thu, Dec 14, 2017 at 1:18 PM, Eric Yang  wrote:
> +1 on squash merge to keep history compressed.  The rebase + merge contains 
> good deals, but it is easy to get confused for people that doesn’t know about 
> the rebase option is turned on by default for Hadoop.
>
> Regards,
> Eric
>
> On 12/14/17, 12:06 PM, "Arun Suresh"  wrote:
>
> Another option - atleast for feature branches is to maybe squash merge -
> this way we see it as a single commit ? Although we will loose the feature
> branch history (I am ok with that though)
>
> Cheers
> -Arun
>
> On Thu, Dec 14, 2017 at 11:32 AM, Eric Yang  wrote:
>
> > Thank you for the pointer.  I guess all merge are done using rebase +
> > merge.  This is the reason that timeline is out of order.
> >
> > Would it be more useful to merge without rebasing for feature branch 
> merge
> > to avoid timeline confusions?  The argument for not rebasing, it would 
> be
> > easier to find the root cause of trunk failure was due to merge or some
> > recent commits.
> >
> > Regards,
> > Eric
> >
> > From: Sunil G 
> > Date: Thursday, December 14, 2017 at 11:11 AM
> > To: Eric Yang 
> > Cc: Hadoop Common 
> > Subject: Re: Missing some trunk commit history
> >
> > Hi Eric.
> >
> > A branch merge has happened during that time, and hence you might have
> > seen some old commits from that branch. If you go down further, you 
> could
> > see those commits.
> >
> > Copied from my git log:
> >
> > commit 40b0045ebe0752cd3d1d09be00acbabdea983799
> > Author: Weiwei Yang >
> > Date:   Wed Dec 6 17:52:41 2017 +0800
> >
> > YARN-7610. Extend Distributed Shell to support launching job with
> > opportunistic containers. Contributed by Weiwei Yang.
> >
> > commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f
> > Author: Xiao Chen >
> > Date:   Tue Dec 5 20:48:02 2017 -0800
> >
> > HDFS-12872. EC Checksum broken when BlockAccessToken is enabled.
> >
> > commit 05c347fe51c01494ed8110f8f116a01c90205f13
> > Author: Weiwei Yang >
> > Date:   Wed Dec 6 12:21:52 2017 +0800
> >
> > YARN-7611. Node manager web UI should display container type in
> > containers page. Contributed by Weiwei Yang.
> >
> > commit 73b86979d661f4ad56fcfc3a05a403dfcb2a860e
> > Author: Kai Zheng 
>  > inc.com>>
> > Date:   Wed Dec 6 12:01:36 2017 +0800
> >
> > HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common.
> > Contributed by Genmao Yu
> >
> > commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b
> > Author: Akira Ajisaka >
> > Date:   Wed Dec 6 11:40:33 2017 +0900
> >
> > HDFS-12889. Router UI is missing robots.txt file. Contributed by
> > Bharat Viswanadham.
> >
> > commit 0311cf05358cd75388f48f048c44fba52ec90f00
> > Author: Wangda Tan >
> > Date:   Tue Dec 5 13:09:49 2017 -0800
> >
> > YARN-7381. Enable the configuration: 
> yarn.nodemanager.log-container-debug-info.enabled
> > by default in yarn-default.xml. (Xuan Gong via wangda)
> >
> > Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
> >
> > commit 6555af81a26b0b72ec3bee7034e01f5bd84b1564
> > Author: Aaron Fabbri >
> > Date:   Tue Dec 5 11:06:32 2017 -0800
> >
> > HADOOP-14475 Metrics of S3A don't print out when enabled. 
> Contributed
> > by Younger and Sean Mackrory.
> >
> >
> >
> > - Sunil
> >
> >
> > On Fri, Dec 15, 2017 at 12:29 AM Eric Yang 
>  ey...@hortonworks.com>> wrote:
> > Hi all,
> >
> > While troubleshooting a trunk build failure, I notice the commit history
> > for trunk between Nov 30th to Dec 6th are squashed or disappeared for no
> > reason.  This seems to have taken place in the last 24 hours.  I can see
> > the commit logs from github UI.  When doing a new clone from Apache Git 
> and
> > Github, the commit histories between those dates are gone.  I usually
> > maintain two git repositories, one for testing and one for development.
> > Both repositories were sync up with github frequently, and only test
> > repository was 

Re: Missing some trunk commit history

[Eric Yang]

+1 on squash merge to keep history compressed.  The rebase + merge contains 
good deals, but it is easy to get confused for people that doesn’t know about 
the rebase option is turned on by default for Hadoop.

Regards,
Eric

On 12/14/17, 12:06 PM, "Arun Suresh"  wrote:

Another option - atleast for feature branches is to maybe squash merge -
this way we see it as a single commit ? Although we will loose the feature
branch history (I am ok with that though)

Cheers
-Arun

On Thu, Dec 14, 2017 at 11:32 AM, Eric Yang  wrote:

> Thank you for the pointer.  I guess all merge are done using rebase +
> merge.  This is the reason that timeline is out of order.
>
> Would it be more useful to merge without rebasing for feature branch merge
> to avoid timeline confusions?  The argument for not rebasing, it would be
> easier to find the root cause of trunk failure was due to merge or some
> recent commits.
>
> Regards,
> Eric
>
> From: Sunil G 
> Date: Thursday, December 14, 2017 at 11:11 AM
> To: Eric Yang 
> Cc: Hadoop Common 
> Subject: Re: Missing some trunk commit history
>
> Hi Eric.
>
> A branch merge has happened during that time, and hence you might have
> seen some old commits from that branch. If you go down further, you could
> see those commits.
>
> Copied from my git log:
>
> commit 40b0045ebe0752cd3d1d09be00acbabdea983799
> Author: Weiwei Yang >
> Date:   Wed Dec 6 17:52:41 2017 +0800
>
> YARN-7610. Extend Distributed Shell to support launching job with
> opportunistic containers. Contributed by Weiwei Yang.
>
> commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f
> Author: Xiao Chen >
> Date:   Tue Dec 5 20:48:02 2017 -0800
>
> HDFS-12872. EC Checksum broken when BlockAccessToken is enabled.
>
> commit 05c347fe51c01494ed8110f8f116a01c90205f13
> Author: Weiwei Yang >
> Date:   Wed Dec 6 12:21:52 2017 +0800
>
> YARN-7611. Node manager web UI should display container type in
> containers page. Contributed by Weiwei Yang.
>
> commit 73b86979d661f4ad56fcfc3a05a403dfcb2a860e
> Author: Kai Zheng  inc.com>>
> Date:   Wed Dec 6 12:01:36 2017 +0800
>
> HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common.
> Contributed by Genmao Yu
>
> commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b
> Author: Akira Ajisaka >
> Date:   Wed Dec 6 11:40:33 2017 +0900
>
> HDFS-12889. Router UI is missing robots.txt file. Contributed by
> Bharat Viswanadham.
>
> commit 0311cf05358cd75388f48f048c44fba52ec90f00
> Author: Wangda Tan >
> Date:   Tue Dec 5 13:09:49 2017 -0800
>
> YARN-7381. Enable the configuration: 
yarn.nodemanager.log-container-debug-info.enabled
> by default in yarn-default.xml. (Xuan Gong via wangda)
>
> Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
>
> commit 6555af81a26b0b72ec3bee7034e01f5bd84b1564
> Author: Aaron Fabbri >
> Date:   Tue Dec 5 11:06:32 2017 -0800
>
> HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed
> by Younger and Sean Mackrory.
>
>
>
> - Sunil
>
>
> On Fri, Dec 15, 2017 at 12:29 AM Eric Yang > wrote:
> Hi all,
>
> While troubleshooting a trunk build failure, I notice the commit history
> for trunk between Nov 30th to Dec 6th are squashed or disappeared for no
> reason.  This seems to have taken place in the last 24 hours.  I can see
> the commit logs from github UI.  When doing a new clone from Apache Git 
and
> Github, the commit histories between those dates are gone.  I usually
> maintain two git repositories, one for testing and one for development.
> Both repositories were sync up with github frequently, and only test
> repository was updated today and the missing history only reflect in test
> repository.  This is the reason that I have the impression that this might
> have happened in the last 24 hours.  I did some spot check to see if the
> missing commits are in trunk.  The code seems to be in place, and only
> commit history is gone.
>
> Is there any way to fix the commit history?  Hopefully this is not a git
> bug, but some peer review might find out the root cause that could help to
> understand the damage.  Thank you
>
  

Re: [VOTE] Release Apache Hadoop 2.7.5 (RC1)

[Zhe Zhang]

Great work Konstantin.

+1 (binding)
- Downloaded both source and binary tar files and verified checksums
- Deployed pseudo-distributed cluster and verified basic HDFS operations
(mkdir, copyFromLocal)
- Verified both NameNode and RM UIs

On Wed, Dec 13, 2017 at 12:05 PM Jonathan Hung  wrote:

> Thanks Konstantin for working on this.
>
> +1 (non-binding)
> - Downloaded binary and verified md5
> - Deployed RM HA and tested failover
>
>
>
>
> Jonathan Hung
>
> On Wed, Dec 13, 2017 at 11:02 AM, Eric Payne  .invalid
> > wrote:
>
> > Thanks for the hard work on this release, Konstantin.
> > +1 (binding)
> > - Built from source
> > - Verified that refreshing of queues works as expected.
> >
> > - Verified can run multiple users in a single queue
> > - Ran terasort test
> > - Verified that cross-queue preemption works as expected
> > Thanks. Eric Payne
> >
> >   From: Konstantin Shvachko 
> >  To: "common-dev@hadoop.apache.org" ; "
> > hdfs-...@hadoop.apache.org" ; "
> > mapreduce-...@hadoop.apache.org" ; "
> > yarn-...@hadoop.apache.org" 
> >  Sent: Thursday, December 7, 2017 9:22 PM
> >  Subject: [VOTE] Release Apache Hadoop 2.7.5 (RC1)
> >
> > Hi everybody,
> >
> > I updated CHANGES.txt and fixed documentation links.
> > Also committed  MAPREDUCE-6165, which fixes a consistently failing test.
> >
> > This is RC1 for the next dot release of Apache Hadoop 2.7 line. The
> > previous one 2.7.4 was release August 4, 2017.
> > Release 2.7.5 includes critical bug fixes and optimizations. See more
> > details in Release Note:
> > http://home.apache.org/~shv/hadoop-2.7.5-RC1/releasenotes.html
> >
> > The RC0 is available at: http://home.apache.org/~shv/hadoop-2.7.5-RC1/
> >
> > Please give it a try and vote on this thread. The vote will run for 5
> days
> > ending 12/13/2017.
> >
> > My up to date public key is available from:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > Thanks,
> > --Konstantin
> >
> >
> >
> >
>
-- 
Zhe Zhang
Apache Hadoop Committer
http://zhe-thoughts.github.io/about/ | @oldcap

Re: Missing some trunk commit history

[Arun Suresh]

Another option - atleast for feature branches is to maybe squash merge -
this way we see it as a single commit ? Although we will loose the feature
branch history (I am ok with that though)

Cheers
-Arun

On Thu, Dec 14, 2017 at 11:32 AM, Eric Yang  wrote:

> Thank you for the pointer.  I guess all merge are done using rebase +
> merge.  This is the reason that timeline is out of order.
>
> Would it be more useful to merge without rebasing for feature branch merge
> to avoid timeline confusions?  The argument for not rebasing, it would be
> easier to find the root cause of trunk failure was due to merge or some
> recent commits.
>
> Regards,
> Eric
>
> From: Sunil G 
> Date: Thursday, December 14, 2017 at 11:11 AM
> To: Eric Yang 
> Cc: Hadoop Common 
> Subject: Re: Missing some trunk commit history
>
> Hi Eric.
>
> A branch merge has happened during that time, and hence you might have
> seen some old commits from that branch. If you go down further, you could
> see those commits.
>
> Copied from my git log:
>
> commit 40b0045ebe0752cd3d1d09be00acbabdea983799
> Author: Weiwei Yang >
> Date:   Wed Dec 6 17:52:41 2017 +0800
>
> YARN-7610. Extend Distributed Shell to support launching job with
> opportunistic containers. Contributed by Weiwei Yang.
>
> commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f
> Author: Xiao Chen >
> Date:   Tue Dec 5 20:48:02 2017 -0800
>
> HDFS-12872. EC Checksum broken when BlockAccessToken is enabled.
>
> commit 05c347fe51c01494ed8110f8f116a01c90205f13
> Author: Weiwei Yang >
> Date:   Wed Dec 6 12:21:52 2017 +0800
>
> YARN-7611. Node manager web UI should display container type in
> containers page. Contributed by Weiwei Yang.
>
> commit 73b86979d661f4ad56fcfc3a05a403dfcb2a860e
> Author: Kai Zheng  inc.com>>
> Date:   Wed Dec 6 12:01:36 2017 +0800
>
> HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common.
> Contributed by Genmao Yu
>
> commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b
> Author: Akira Ajisaka >
> Date:   Wed Dec 6 11:40:33 2017 +0900
>
> HDFS-12889. Router UI is missing robots.txt file. Contributed by
> Bharat Viswanadham.
>
> commit 0311cf05358cd75388f48f048c44fba52ec90f00
> Author: Wangda Tan >
> Date:   Tue Dec 5 13:09:49 2017 -0800
>
> YARN-7381. Enable the configuration: 
> yarn.nodemanager.log-container-debug-info.enabled
> by default in yarn-default.xml. (Xuan Gong via wangda)
>
> Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
>
> commit 6555af81a26b0b72ec3bee7034e01f5bd84b1564
> Author: Aaron Fabbri >
> Date:   Tue Dec 5 11:06:32 2017 -0800
>
> HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed
> by Younger and Sean Mackrory.
>
>
>
> - Sunil
>
>
> On Fri, Dec 15, 2017 at 12:29 AM Eric Yang > wrote:
> Hi all,
>
> While troubleshooting a trunk build failure, I notice the commit history
> for trunk between Nov 30th to Dec 6th are squashed or disappeared for no
> reason.  This seems to have taken place in the last 24 hours.  I can see
> the commit logs from github UI.  When doing a new clone from Apache Git and
> Github, the commit histories between those dates are gone.  I usually
> maintain two git repositories, one for testing and one for development.
> Both repositories were sync up with github frequently, and only test
> repository was updated today and the missing history only reflect in test
> repository.  This is the reason that I have the impression that this might
> have happened in the last 24 hours.  I did some spot check to see if the
> missing commits are in trunk.  The code seems to be in place, and only
> commit history is gone.
>
> Is there any way to fix the commit history?  Hopefully this is not a git
> bug, but some peer review might find out the root cause that could help to
> understand the damage.  Thank you
>
> Regards,
> Eric
>

Re: Missing some trunk commit history

[Eric Yang]

Thank you for the pointer.  I guess all merge are done using rebase + merge.  
This is the reason that timeline is out of order.

Would it be more useful to merge without rebasing for feature branch merge to 
avoid timeline confusions?  The argument for not rebasing, it would be easier 
to find the root cause of trunk failure was due to merge or some recent commits.

Regards,
Eric

From: Sunil G 
Date: Thursday, December 14, 2017 at 11:11 AM
To: Eric Yang 
Cc: Hadoop Common 
Subject: Re: Missing some trunk commit history

Hi Eric.

A branch merge has happened during that time, and hence you might have seen 
some old commits from that branch. If you go down further, you could see those 
commits.

Copied from my git log:

commit 40b0045ebe0752cd3d1d09be00acbabdea983799
Author: Weiwei Yang >
Date:   Wed Dec 6 17:52:41 2017 +0800

YARN-7610. Extend Distributed Shell to support launching job with 
opportunistic containers. Contributed by Weiwei Yang.

commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f
Author: Xiao Chen >
Date:   Tue Dec 5 20:48:02 2017 -0800

HDFS-12872. EC Checksum broken when BlockAccessToken is enabled.

commit 05c347fe51c01494ed8110f8f116a01c90205f13
Author: Weiwei Yang >
Date:   Wed Dec 6 12:21:52 2017 +0800

YARN-7611. Node manager web UI should display container type in containers 
page. Contributed by Weiwei Yang.

commit 73b86979d661f4ad56fcfc3a05a403dfcb2a860e
Author: Kai Zheng 
>
Date:   Wed Dec 6 12:01:36 2017 +0800

HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. 
Contributed by Genmao Yu

commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b
Author: Akira Ajisaka >
Date:   Wed Dec 6 11:40:33 2017 +0900

HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat 
Viswanadham.

commit 0311cf05358cd75388f48f048c44fba52ec90f00
Author: Wangda Tan >
Date:   Tue Dec 5 13:09:49 2017 -0800

YARN-7381. Enable the configuration: 
yarn.nodemanager.log-container-debug-info.enabled by default in 
yarn-default.xml. (Xuan Gong via wangda)

Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87

commit 6555af81a26b0b72ec3bee7034e01f5bd84b1564
Author: Aaron Fabbri >
Date:   Tue Dec 5 11:06:32 2017 -0800

HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed by 
Younger and Sean Mackrory.



- Sunil


On Fri, Dec 15, 2017 at 12:29 AM Eric Yang 
> wrote:
Hi all,

While troubleshooting a trunk build failure, I notice the commit history for 
trunk between Nov 30th to Dec 6th are squashed or disappeared for no reason.  
This seems to have taken place in the last 24 hours.  I can see the commit logs 
from github UI.  When doing a new clone from Apache Git and Github, the commit 
histories between those dates are gone.  I usually maintain two git 
repositories, one for testing and one for development.  Both repositories were 
sync up with github frequently, and only test repository was updated today and 
the missing history only reflect in test repository.  This is the reason that I 
have the impression that this might have happened in the last 24 hours.  I did 
some spot check to see if the missing commits are in trunk.  The code seems to 
be in place, and only commit history is gone.

Is there any way to fix the commit history?  Hopefully this is not a git bug, 
but some peer review might find out the root cause that could help to 
understand the damage.  Thank you

Regards,
Eric

Re: Missing some trunk commit history

[Sunil G]

Hi Eric.

A branch merge has happened during that time, and hence you might have seen
some old commits from that branch. If you go down further, you could see
those commits.

Copied from my git log:

commit 40b0045ebe0752cd3d1d09be00acbabdea983799
Author: Weiwei Yang 
Date:   Wed Dec 6 17:52:41 2017 +0800

YARN-7610. Extend Distributed Shell to support launching job with
opportunistic containers. Contributed by Weiwei Yang.

commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f
Author: Xiao Chen 
Date:   Tue Dec 5 20:48:02 2017 -0800

HDFS-12872. EC Checksum broken when BlockAccessToken is enabled.

commit 05c347fe51c01494ed8110f8f116a01c90205f13
Author: Weiwei Yang 
Date:   Wed Dec 6 12:21:52 2017 +0800

YARN-7611. Node manager web UI should display container type in
containers page. Contributed by Weiwei Yang.

commit 73b86979d661f4ad56fcfc3a05a403dfcb2a860e
Author: Kai Zheng 
Date:   Wed Dec 6 12:01:36 2017 +0800

HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common.
Contributed by Genmao Yu

commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b
Author: Akira Ajisaka 
Date:   Wed Dec 6 11:40:33 2017 +0900

HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat
Viswanadham.

commit 0311cf05358cd75388f48f048c44fba52ec90f00
Author: Wangda Tan 
Date:   Tue Dec 5 13:09:49 2017 -0800

YARN-7381. Enable the configuration:
yarn.nodemanager.log-container-debug-info.enabled by default in
yarn-default.xml. (Xuan Gong via wangda)

Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87

commit 6555af81a26b0b72ec3bee7034e01f5bd84b1564
Author: Aaron Fabbri 
Date:   Tue Dec 5 11:06:32 2017 -0800

HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed
by Younger and Sean Mackrory.



- Sunil


On Fri, Dec 15, 2017 at 12:29 AM Eric Yang  wrote:

> Hi all,
>
> While troubleshooting a trunk build failure, I notice the commit history
> for trunk between Nov 30th to Dec 6th are squashed or disappeared for no
> reason.  This seems to have taken place in the last 24 hours.  I can see
> the commit logs from github UI.  When doing a new clone from Apache Git and
> Github, the commit histories between those dates are gone.  I usually
> maintain two git repositories, one for testing and one for development.
> Both repositories were sync up with github frequently, and only test
> repository was updated today and the missing history only reflect in test
> repository.  This is the reason that I have the impression that this might
> have happened in the last 24 hours.  I did some spot check to see if the
> missing commits are in trunk.  The code seems to be in place, and only
> commit history is gone.
>
> Is there any way to fix the commit history?  Hopefully this is not a git
> bug, but some peer review might find out the root cause that could help to
> understand the damage.  Thank you
>
> Regards,
> Eric
>
>

Missing some trunk commit history

[Eric Yang]

Hi all,

While troubleshooting a trunk build failure, I notice the commit history for 
trunk between Nov 30th to Dec 6th are squashed or disappeared for no reason.  
This seems to have taken place in the last 24 hours.  I can see the commit logs 
from github UI.  When doing a new clone from Apache Git and Github, the commit 
histories between those dates are gone.  I usually maintain two git 
repositories, one for testing and one for development.  Both repositories were 
sync up with github frequently, and only test repository was updated today and 
the missing history only reflect in test repository.  This is the reason that I 
have the impression that this might have happened in the last 24 hours.  I did 
some spot check to see if the missing commits are in trunk.  The code seems to 
be in place, and only commit history is gone.

Is there any way to fix the commit history?  Hopefully this is not a git bug, 
but some peer review might find out the root cause that could help to 
understand the damage.  Thank you

Regards,
Eric

[ANNOUNCE] Apache Hadoop 3.0.0 GA is released

[Andrew Wang]

Hi all,

I'm pleased to announce that Apache Hadoop 3.0.0 is generally available
(GA).

3.0.0 GA consists of 302 bug fixes, improvements, and other enhancements
since 3.0.0-beta1. This release marks a point of quality and stability for
the 3.0.0 release line, and users of earlier 3.0.0-alpha and -beta releases
are encouraged to upgrade.

Looking back, 3.0.0 GA is the culmination of over a year of work on the
3.0.0 line, starting with 3.0.0-alpha1 which was released in September
2016. Altogether, 3.0.0 incorporates 6,242 changes since 2.7.0.

Users are encouraged to read the overview of major changes
 in 3.0.0. The GA release
notes

 and changelog

detail
the changes since 3.0.0-beta1.

The ASF press release provides additional color and highlights some of the
major features:

https://globenewswire.com/news-release/2017/12/14/1261879/0/en/The-Apache-Software-Foundation-Announces-Apache-Hadoop-v3-0-0-General-Availability.html

Let me end by thanking the many, many contributors who helped with this
release line. We've only had three major releases in Hadoop's 10 year
history, and this is our biggest major release ever. It's an incredible
accomplishment for our community, and I'm proud to have worked with all of
you.

Best,
Andrew

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/622/

[Dec 13, 2017 5:19:58 PM] (sunilg) YARN-7643. Handle recovery of applications 
in case of auto-created leaf
[Dec 13, 2017 9:05:32 PM] (wang) Update CHANGES, RELEASENOTES, jdiff for 3.0.0 
release.
[Dec 13, 2017 10:56:14 PM] (cliang) HADOOP-14914. Change to a safely casting 
long to int. Contributed by
[Dec 14, 2017 12:29:04 AM] (eyang) YARN-7540.  Route YARN service CLI function 
through YARN Service API.
[Dec 14, 2017 3:05:11 AM] (zhengkai.zk) Revert "HADOOP-15039. Move 
SemaphoredDelegatingExecutor to
[Dec 14, 2017 3:14:52 AM] (zhengkai.zk) HADOOP-15039. Move 
SemaphoredDelegatingExecutor to hadoop-common.
[Dec 14, 2017 8:00:53 AM] (yqlin) HDFS-12883. RBF: Document Router and State 
Store metrics. Contributed by
[Dec 14, 2017 10:41:57 AM] (sunilg) YARN-7119. Support multiple resource types 
in rmadmin updateNodeResource
[Dec 14, 2017 3:45:44 PM] (jlowe) HDFS-12881. Output streams closed with 
IOUtils suppressing write errors.
[Dec 14, 2017 4:18:08 PM] (jlowe) HADOOP-15085. Output streams closed with 
IOUtils suppressing write

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org