date:20200623

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC1)

2020-06-23 Thread Akira Ajisaka

Hi Gabor,

Thank you for your work!

Kihwal reported IBR leak in standby NameNode:
https://issues.apache.org/jira/browse/HDFS-15421.
I think this is a blocker and this affects 3.1.4-RC1. Would you check this?

Best regards,
Akira

On Mon, Jun 22, 2020 at 10:26 PM Gabor Bota 
wrote:

> Hi folks,
>
> I have put together a release candidate (RC1) for Hadoop 3.1.4.
>
> The RC is available at: http://people.apache.org/~gabota/hadoop-3.1.4-RC1/
> The RC tag in git is here:
> https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC1
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1267/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> and http://keys.gnupg.net/pks/lookup?op=get=0xB86249D83539B38C
>
> Please try the release and vote. The vote will run for 5 weekdays,
> until June 30. 2020. 23:00 CET.
>
> Thanks,
> Gabor
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>

Update guava to 27.0-jre in hadoop branch-2.10

2020-06-23 Thread Ahmed Hussein

Hi folks,

I was looking into upgrading guava to  27.0-jre on branch-2.10 in order to
address the vulnerabilities reported as CVE-2018-10237
.
Since there are concerns using Java8, the plan is to stick to JDK7.

Obviously, it is expected that the upgrade will break downstream projects.

I opened this for discussion to get feedback and make sure that we have
common ground to address the security of vulnerabilities.

Let me know WDYT.

--
Best Regards,

*Ahmed Hussein, PhD*

Re: Update guava to 27.0-jre in hadoop branch-2.10

2020-06-23 Thread Wei-Chiu Chuang

Ahmed,
I see you filed HADOOP-17083
 for the same
discussion.

I started a thread a while ago in the Hadoop dev mailing list to share my
experience adopting guava27.  Simply porting HADOOP-15960
 to branch-2 will break
miserably because all downstream applications will not compile/run. It took
us half a year to get it harmonized across Cloudera's stack and I don't
want to see you spending time on that.

I feel the better approach is HADOOP-16924
 where we shade and
then update guava. There is more work inside Hadoop to change references to
the shaded guava classpath, but it'll save you more time later.

On Tue, Jun 23, 2020 at 9:09 AM Ahmed Hussein  wrote:

> Hi folks,
>
> I was looking into upgrading guava to  27.0-jre on branch-2.10 in order to
> address the vulnerabilities reported as CVE-2018-10237
> .
> Since there are concerns using Java8, the plan is to stick to JDK7.
>
> Obviously, it is expected that the upgrade will break downstream projects.
>
> I opened this for discussion to get feedback and make sure that we have
> common ground to address the security of vulnerabilities.
>
> Let me know WDYT.
>
> --
> Best Regards,
>
> *Ahmed Hussein, PhD*
>

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC1)

2020-06-23 Thread Kihwal Lee

Gabor,
If you want to release asap, you can simply revert HDFS-14941 in the
release branch for now. It is causing the issue and was committed after
3.1.3.  This causes failure of the automated upgrade process and namenode
memory leak.

Kihwal

On Tue, Jun 23, 2020 at 8:47 AM Akira Ajisaka  wrote:

> Hi Gabor,
>
> Thank you for your work!
>
> Kihwal reported IBR leak in standby NameNode:
> https://issues.apache.org/jira/browse/HDFS-15421.
> I think this is a blocker and this affects 3.1.4-RC1. Would you check this?
>
> Best regards,
> Akira
>
> On Mon, Jun 22, 2020 at 10:26 PM Gabor Bota  .invalid>
> wrote:
>
> > Hi folks,
> >
> > I have put together a release candidate (RC1) for Hadoop 3.1.4.
> >
> > The RC is available at:
> http://people.apache.org/~gabota/hadoop-3.1.4-RC1/
> > The RC tag in git is here:
> > https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC1
> > The maven artifacts are staged at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1267/
> >
> > You can find my public key at:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > and http://keys.gnupg.net/pks/lookup?op=get=0xB86249D83539B38C
> >
> > Please try the release and vote. The vote will run for 5 weekdays,
> > until June 30. 2020. 23:00 CET.
> >
> > Thanks,
> > Gabor
> >
> > -
> > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
> >
> >
>

[jira] [Created] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-23 Thread Ahmed Hussein (Jira)

Ahmed Hussein created HADOOP-17083:
--

 Summary: Update guava to 27.0-jre in hadoop branch-2.10
 Key: HADOOP-17083
 URL: https://issues.apache.org/jira/browse/HADOOP-17083
 Project: Hadoop Common
  Issue Type: Bug
  Components: common, security
Affects Versions: 2.10.0
Reporter: Ahmed Hussein
Assignee: Ahmed Hussein


com.google.guava:guava should be upgraded to 27.0-jre due to new CVE's found 
[CVE-2018-10237|https://nvd.nist.gov/vuln/detail/CVE-2018-10237].

 

The upgrade should not affect the version of java used. branch-2.10 still 
sticks to JDK7



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [DISCUSS] Guidelines for Code cleanup JIRAs

2020-06-23 Thread Shashikant Banerjee

+1

Thanks
Shashi

On Mon, Jan 13, 2020 at 10:27 PM Ahmed Hussein  wrote:

> +1
> Can we also make sure to add a label for the code cleanup Jiras? At least,
> this will make it easy to search and filter jiras.
>
> On Mon, Jan 13, 2020 at 7:24 AM Wei-Chiu Chuang 
> wrote:
>
> > +1
> >
> > On Thu, Jan 9, 2020 at 9:33 AM epa...@apache.org 
> > wrote:
> >
> > > There was some discussion on
> > > https://issues.apache.org/jira/browse/YARN-9052
> > > about concerns surrounding the costs/benefits of code cleanup JIRAs.
> This
> > > email
> > > is to get the discussion going within a wider audience.
> > >
> > > The positive points for code cleanup JIRAs:
> > > - Clean up tech debt
> > > - Make code more readable
> > > - Make code more maintainable
> > > - Make code more performant
> > >
> > > The concerns regarding code cleanup JIRAs are as follows:
> > > - If the changes only go into trunk, then contributors and committers
> > > trying to
> > >  backport to prior releases will have to create and test multiple patch
> > > versions.
> > > - Some have voiced concerns that code cleanup JIRAs may not be tested
> as
> > >   thoroughly as features and bug fixes because functionality is not
> > > supposed to
> > >   change.
> > > - Any patches awaiting review that are touching the same code will have
> > to
> > > be
> > >   redone, re-tested, and re-reviewed.
> > > - JIRAs that are opened for code cleanup and not worked on right away
> > tend
> > > to
> > >   clutter up the JIRA space.
> > >
> > > Here are my opinions:
> > > - Code changes of any kind force a non-trivial amount of overhead for
> > other
> > >   developers. For code cleanup JIRAs, sometimes the usability,
> > > maintainability,
> > >   and performance is worth the overhead (as in the case of YARN-9052).
> > > - Before opening any JIRA, please always consider whether or not the
> > added
> > >   usability will outweigh the added pain you are causing other
> > developers.
> > > - If you believe the benefits outweigh the costs, please backport the
> > > changes
> > >   yourself to all active lines. My preference is to port all the way
> back
> > > to 2.10.
> > > - Please don't run code analysis tools and then open many JIRAs that
> > > document
> > >   those findings. That activity does not put any thought into this
> > > cost-benefit
> > >   analysis.
> > >
> > > Thanks everyone. I'm looking forward to your thoughts. I appreciate all
> > > you do
> > > for the open source community and it is always a pleasure to work with
> > you.
> > > -Eric Payne
> > >
> > > -
> > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> > >
> > >
> >
>

[jira] [Created] (HADOOP-17085) javadoc failing in the yetus report with the latest trunk

2020-06-23 Thread Ishani (Jira)

Ishani created HADOOP-17085:
---

 Summary: javadoc failing in the yetus report with the latest trunk
 Key: HADOOP-17085
 URL: https://issues.apache.org/jira/browse/HADOOP-17085
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: build, yetus
Reporter: Ishani


javadoc is failing in the latest yetus report on  trunk. below is a report from 
an empty PR where it is failing.

 

 *-1 overall*
||Vote||Subsystem||Runtime||Comment||
|+0 |reexec|26m 14s|Docker mode activated.|
| | |_ Prechecks _| |
|+1 |dupname|0m 0s|No case conflicting files found.|
|+1 |[@author|https://github.com/author]|0m 0s|The patch does not contain any 
[@author|https://github.com/author] tags.|
|-1 ❌|test4tests|0m 0s|The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch.|
| | |_ trunk Compile Tests _| |
|+1 |mvninstall|23m 14s|trunk passed|
|+1 |compile|0m 46s|trunk passed with JDK 
Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04|
|+1 |compile|0m 30s|trunk passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09|
|+1 |checkstyle|0m 24s|trunk passed|
|+1 |mvnsite|0m 35s|trunk passed|
|+1 |shadedclient|16m 53s|branch has no errors when building and testing our 
client artifacts.|
|-1 ❌|javadoc|0m 25s|hadoop-azure in trunk failed with JDK 
Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.|
|+1 |javadoc|0m 23s|trunk passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09|
|+0 |spotbugs|0m 53s|Used deprecated FindBugs config; considering switching to 
SpotBugs.|
|+1 |findbugs|0m 51s|trunk passed|
| | |_ Patch Compile Tests _| |
|+1 |mvninstall|0m 27s|the patch passed|
|+1 |compile|0m 27s|the patch passed with JDK 
Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04|
|+1 |javac|0m 27s|the patch passed|
|+1 |compile|0m 22s|the patch passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09|
|+1 |javac|0m 22s|the patch passed|
|+1 |checkstyle|0m 15s|the patch passed|
|+1 |mvnsite|0m 24s|the patch passed|
|+1 |whitespace|0m 0s|The patch has no whitespace issues.|
|+1 |shadedclient|15m 29s|patch has no errors when building and testing our 
client artifacts.|
|-1 ❌|javadoc|0m 22s|hadoop-azure in the patch failed with JDK 
Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.|
|+1 |javadoc|0m 20s|the patch passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09|
|+1 |findbugs|0m 53s|the patch passed|
| | |_ Other Tests _| |
|+1 |unit|1m 19s|hadoop-azure in the patch passed.|
|+1 |asflicense|0m 28s|The patch does not generate ASF License warnings.|
| | |92m 45s| |

||Subsystem||Report/Notes||
|Docker|ClientAPI=1.40 ServerAPI=1.40 base: 
[https://builds.apache.org/job/hadoop-multibranch/job/PR-2091/1/artifact/out/Dockerfile]|
|GITHUB PR|[#2091|https://github.com/apache/hadoop/pull/2091]|
|Optional Tests|dupname asflicense compile javac javadoc mvninstall mvnsite 
unit shadedclient findbugs checkstyle|
|uname|Linux ddd84b65f91e 4.15.0-101-generic 
[#102|https://github.com/apache/hadoop/pull/102]-Ubuntu SMP Mon May 11 10:07:26 
UTC 2020 x86_64 x86_64 x86_64 GNU/Linux|
|Build tool|maven|
|Personality|personality/hadoop.sh|
|git revision|trunk / 
[{{7c02d18}}|https://github.com/apache/hadoop/commit/7c02d1889bbeabc73c95a4c83f0cd204365ff410]|
|Default Java|Private Build-1.8.0_252-8u252-b09-1~18.04-b09|
|Multi-JDK 
versions|/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04
 /usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09|
|javadoc|[https://builds.apache.org/job/hadoop-multibranch/job/PR-2091/1/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt]|
|javadoc|[https://builds.apache.org/job/hadoop-multibranch/job/PR-2091/1/artifact/out/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt]|
|Test 
Results|[https://builds.apache.org/job/hadoop-multibranch/job/PR-2091/1/testReport/]|
|Max. process+thread count|308 (vs. ulimit of 5500)|
|modules|C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure|
|Console 
output|[https://builds.apache.org/job/hadoop-multibranch/job/PR-2091/1/console]|
|versions|git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1|
|Powered by|Apache Yetus 0.12.0 
[https://yetus.apache.org|https://yetus.apache.org/]|

This message was automatically generated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-17084) Update Dockerfile_aarch64 to use Bionic

2020-06-23 Thread RuiChen (Jira)

RuiChen created HADOOP-17084:


 Summary: Update Dockerfile_aarch64 to use Bionic
 Key: HADOOP-17084
 URL: https://issues.apache.org/jira/browse/HADOOP-17084
 Project: Hadoop Common
  Issue Type: Improvement
  Components: build, test
Reporter: RuiChen


Dockerfile for x86 have been updated to apply Ubuntu Bionic, JDK11 and other 
changes, we should make Dockerfile for aarch64 following these changes, keep 
same behavior.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC1)

Update guava to 27.0-jre in hadoop branch-2.10

Re: Update guava to 27.0-jre in hadoop branch-2.10

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC1)

[jira] [Created] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

Re: [DISCUSS] Guidelines for Code cleanup JIRAs

[jira] [Created] (HADOOP-17085) javadoc failing in the yetus report with the latest trunk

[jira] [Created] (HADOOP-17084) Update Dockerfile_aarch64 to use Bionic

8 matches

Site Navigation

Mail list logo

Footer information