[jira] [Created] (HADOOP-14340) Enable KMS and HttpFS to exclude weak SSL ciphers
John Zhuge created HADOOP-14340:
---
Summary: Enable KMS and HttpFS to exclude weak SSL ciphers
Key: HADOOP-14340
URL: https://issues.apache.org/jira/browse/HADOOP-14340
Project: Hadoop Common
Issue Type: Improvement
Components: kms
Affects Versions: 3.0.0-alpha2
Reporter: John Zhuge
Assignee: John Zhuge
Priority: Minor
HADOOP-12668 added {{HttpServer2$Builder#excludeCiphers}} to exclude SSL
ciphers. Enable KMS and HttpFS to use this feature by modifying
{{HttpServer2$Builder#loadSSLConfiguration}} calld by both.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14339) Fix warnings from Spotbugs in hadoop-mapreduce
Weiwei Yang created HADOOP-14339: Summary: Fix warnings from Spotbugs in hadoop-mapreduce Key: HADOOP-14339 URL: https://issues.apache.org/jira/browse/HADOOP-14339 Project: Hadoop Common Issue Type: Sub-task Reporter: Weiwei Yang Assignee: Weiwei Yang Fix warnings from Spotbugs in hadoop-mapreduce since switched from findbugs to spotbugs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14338) Fix warnings from Spotbugs in hadoop-yarn
Weiwei Yang created HADOOP-14338: Summary: Fix warnings from Spotbugs in hadoop-yarn Key: HADOOP-14338 URL: https://issues.apache.org/jira/browse/HADOOP-14338 Project: Hadoop Common Issue Type: Sub-task Reporter: Weiwei Yang Assignee: Weiwei Yang Fix warnings from Spotbugs in hadoop-yarn since switched from findbugs to spotbugs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14336) Cleanup findbugs warnings found by Spotbugs
Akira Ajisaka created HADOOP-14336: -- Summary: Cleanup findbugs warnings found by Spotbugs Key: HADOOP-14336 URL: https://issues.apache.org/jira/browse/HADOOP-14336 Project: Hadoop Common Issue Type: Bug Reporter: Akira Ajisaka HADOOP-14316 switched from Findbugs to Spotbugs and there are now about 60 warnings. Let's fix them. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-12940) Fix warnings from findbugs 3.0.1 in hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer reopened HADOOP-12940: --- It's effectively the same thing. Let's see if this patch applies still. > Fix warnings from findbugs 3.0.1 in hadoop-common > - > > Key: HADOOP-12940 > URL: https://issues.apache.org/jira/browse/HADOOP-12940 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-12940.01.patch, hadoop-common.20160715.html, > hadoop-common.html > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14333) HADOOP-14104 changed DFSClient API isHDFSEncryptionEnabled, impacted hacky hive code
Yongjun Zhang created HADOOP-14333: -- Summary: HADOOP-14104 changed DFSClient API isHDFSEncryptionEnabled, impacted hacky hive code Key: HADOOP-14333 URL: https://issues.apache.org/jira/browse/HADOOP-14333 Project: Hadoop Common Issue Type: Bug Reporter: Yongjun Zhang Though Hive should be fixed not to access DFSClient which is private to HADOOP, removing the throws added by HADOOP-14104 is a quicker solution to unblock hive. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/ [Apr 19, 2017 3:09:09 PM] (sunilg) YARN-6164. Expose Queue Configurations per Node Label through YARN [Apr 19, 2017 4:41:09 PM] (templedf) HADOOP-13997. Typo in metrics docs (Contributed by Ana Krasteva via [Apr 19, 2017 5:31:14 PM] (templedf) YARN-3427. Remove deprecated methods from ResourceCalculatorProcessTree [Apr 19, 2017 5:53:08 PM] (templedf) YARN-6438. Code can be improved in ContainersMonitorImpl.java [Apr 19, 2017 6:08:59 PM] (wang) YARN-6365. Get static SLS html resources from classpath. Contributed by [Apr 19, 2017 6:44:55 PM] (templedf) YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is [Apr 19, 2017 6:50:54 PM] (jzhuge) HADOOP-14241. Add ADLS sensitive config keys to default list. [Apr 19, 2017 7:13:40 PM] (aw) HADOOP-14316. Switch from Findbugs to Spotbugs (aw) [Apr 19, 2017 7:23:49 PM] (templedf) YARN-6302. Fail the node if Linux Container Executor is not configured [Apr 19, 2017 8:49:56 PM] (wang) HDFS-11671. [Apr 20, 2017 1:10:04 AM] (wang) HDFS-11660. TestFsDatasetCache#testPageRounder fails intermittently with [Apr 20, 2017 7:03:53 AM] (sunilg) YARN-6402. Move 'Long Running Services' to an independent tab at top -1 overall The following subsystems voted -1: compile mvninstall unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180 hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService hadoop.mapred.TestShuffleHandler hadoop.tools.TestHadoopArchiveLogsRunner hadoop.metrics2.impl.TestKafkaMetrics hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.yarn.server.timeline.TestRollingLevelDB hadoop.yarn.server.timeline.TestTimelineDataManager hadoop.yarn.server.timeline.TestLeveldbTimelineStore hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore Timed out junit tests : org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache mvninstall: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-mvninstall-root.txt [492K] compile: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-compile-root.txt [20K] cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-compile-root.txt [20K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-compile-root.txt [20K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-unit-hadoop-assemblies.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [332K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt [48K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-shuffle.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/294/artifact/out/patch-unit-hadoop-tools_hadoop-archive-logs.txt [8.0K]
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/383/ [Apr 19, 2017 8:49:56 PM] (wang) HDFS-11671. [Apr 20, 2017 1:10:04 AM] (wang) HDFS-11660. TestFsDatasetCache#testPageRounder fails intermittently with [Apr 20, 2017 7:03:53 AM] (sunilg) YARN-6402. Move 'Long Running Services' to an independent tab at top -1 overall The following subsystems voted -1: asflicense findbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-common-project/hadoop-minikdc Possible null pointer dereference in org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value of called method Dereferenced at MiniKdc.java:org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value of called method Dereferenced at MiniKdc.java:[line 368] FindBugs : module:hadoop-common-project/hadoop-auth org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.authenticate(HttpServletRequest, HttpServletResponse) makes inefficient use of keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:of keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:[line 192] FindBugs : module:hadoop-common-project/hadoop-common org.apache.hadoop.crypto.CipherSuite.setUnknownValue(int) unconditionally sets the field unknownValue At CipherSuite.java:unknownValue At CipherSuite.java:[line 44] org.apache.hadoop.crypto.CryptoProtocolVersion.setUnknownValue(int) unconditionally sets the field unknownValue At CryptoProtocolVersion.java:unknownValue At CryptoProtocolVersion.java:[line 67] Possible null pointer dereference in org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to return value of called method Dereferenced at FileUtil.java:org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to return value of called method Dereferenced at FileUtil.java:[line 118] Possible null pointer dereference in org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path, File, Path, File) due to return value of called method Dereferenced at RawLocalFileSystem.java:org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path, File, Path, File) due to return value of called method Dereferenced at RawLocalFileSystem.java:[line 387] Return value of org.apache.hadoop.fs.permission.FsAction.or(FsAction) ignored, but method has no side effect At FTPFileSystem.java:but method has no side effect At FTPFileSystem.java:[line 421] Useless condition:lazyPersist == true at this point At CommandWithDestination.java:[line 502] org.apache.hadoop.io.DoubleWritable.compareTo(DoubleWritable) incorrectly handles double value At DoubleWritable.java: At DoubleWritable.java:[line 78] org.apache.hadoop.io.DoubleWritable$Comparator.compare(byte[], int, int, byte[], int, int) incorrectly handles double value At DoubleWritable.java:int) incorrectly handles double value At DoubleWritable.java:[line 97] org.apache.hadoop.io.FloatWritable.compareTo(FloatWritable) incorrectly handles float value At FloatWritable.java: At FloatWritable.java:[line 71] org.apache.hadoop.io.FloatWritable$Comparator.compare(byte[], int, int, byte[], int, int) incorrectly handles float value At FloatWritable.java:int) incorrectly handles float value At FloatWritable.java:[line 89] Possible null pointer dereference in org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) due to return value of called method Dereferenced at IOUtils.java:org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) due to return value of called method Dereferenced at IOUtils.java:[line 350] org.apache.hadoop.io.erasurecode.ECSchema.toString() makes inefficient use of keySet iterator instead of entrySet iterator At ECSchema.java:keySet iterator instead of entrySet iterator At ECSchema.java:[line 193] Possible bad parsing of shift operation in org.apache.hadoop.io.file.tfile.Utils$Version.hashCode() At Utils.java:operation in org.apache.hadoop.io.file.tfile.Utils$Version.hashCode() At Utils.java:[line 398] org.apache.hadoop.metrics2.lib.DefaultMetricsFactory.setInstance(MutableMetricsFactory) unconditionally sets the field mmfImpl At DefaultMetricsFactory.java:mmfImpl At DefaultMetricsFactory.java:[line 49] org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.setMiniClusterMode(boolean) unconditionally sets the field miniClusterMode At DefaultMetricsSystem.java:miniClusterMode At DefaultMetricsSystem.java:[line 100] Useless object stored in variable seqOs of method
[jira] [Created] (HADOOP-14332) Document S3A SSE
Steve Loughran created HADOOP-14332: --- Summary: Document S3A SSE Key: HADOOP-14332 URL: https://issues.apache.org/jira/browse/HADOOP-14332 Project: Hadoop Common Issue Type: Sub-task Components: documentation, fs/s3 Affects Versions: 2.9.0 Reporter: Steve Loughran Priority: Minor Go into some detail about effective (secure) use of S3 SSE * features, benefits, limitations * how to lock down a bucket * what to avoid (SSE-C; mixed-security buckets) * performance impact * what the new stack traces mean -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/382/ [Apr 19, 2017 9:21:44 AM] (stevel) HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. [Apr 19, 2017 3:09:09 PM] (sunilg) YARN-6164. Expose Queue Configurations per Node Label through YARN [Apr 19, 2017 4:41:09 PM] (templedf) HADOOP-13997. Typo in metrics docs (Contributed by Ana Krasteva via [Apr 19, 2017 5:31:14 PM] (templedf) YARN-3427. Remove deprecated methods from ResourceCalculatorProcessTree [Apr 19, 2017 5:53:08 PM] (templedf) YARN-6438. Code can be improved in ContainersMonitorImpl.java [Apr 19, 2017 6:08:59 PM] (wang) YARN-6365. Get static SLS html resources from classpath. Contributed by [Apr 19, 2017 6:44:55 PM] (templedf) YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is [Apr 19, 2017 6:50:54 PM] (jzhuge) HADOOP-14241. Add ADLS sensitive config keys to default list. [Apr 19, 2017 7:13:40 PM] (aw) HADOOP-14316. Switch from Findbugs to Spotbugs (aw) [Apr 19, 2017 7:23:49 PM] (templedf) YARN-6302. Fail the node if Linux Container Executor is not configured -1 overall The following subsystems voted -1: asflicense findbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-common-project/hadoop-minikdc Possible null pointer dereference in org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value of called method Dereferenced at MiniKdc.java:org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value of called method Dereferenced at MiniKdc.java:[line 368] FindBugs : module:hadoop-common-project/hadoop-auth org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.authenticate(HttpServletRequest, HttpServletResponse) makes inefficient use of keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:of keySet iterator instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:[line 192] FindBugs : module:hadoop-common-project/hadoop-common org.apache.hadoop.crypto.CipherSuite.setUnknownValue(int) unconditionally sets the field unknownValue At CipherSuite.java:unknownValue At CipherSuite.java:[line 44] org.apache.hadoop.crypto.CryptoProtocolVersion.setUnknownValue(int) unconditionally sets the field unknownValue At CryptoProtocolVersion.java:unknownValue At CryptoProtocolVersion.java:[line 67] Possible null pointer dereference in org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to return value of called method Dereferenced at FileUtil.java:org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to return value of called method Dereferenced at FileUtil.java:[line 118] Possible null pointer dereference in org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path, File, Path, File) due to return value of called method Dereferenced at RawLocalFileSystem.java:org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path, File, Path, File) due to return value of called method Dereferenced at RawLocalFileSystem.java:[line 387] Return value of org.apache.hadoop.fs.permission.FsAction.or(FsAction) ignored, but method has no side effect At FTPFileSystem.java:but method has no side effect At FTPFileSystem.java:[line 421] Useless condition:lazyPersist == true at this point At CommandWithDestination.java:[line 502] org.apache.hadoop.io.DoubleWritable.compareTo(DoubleWritable) incorrectly handles double value At DoubleWritable.java: At DoubleWritable.java:[line 78] org.apache.hadoop.io.DoubleWritable$Comparator.compare(byte[], int, int, byte[], int, int) incorrectly handles double value At DoubleWritable.java:int) incorrectly handles double value At DoubleWritable.java:[line 97] org.apache.hadoop.io.FloatWritable.compareTo(FloatWritable) incorrectly handles float value At FloatWritable.java: At FloatWritable.java:[line 71] org.apache.hadoop.io.FloatWritable$Comparator.compare(byte[], int, int, byte[], int, int) incorrectly handles float value At FloatWritable.java:int) incorrectly handles float value At FloatWritable.java:[line 89] Possible null pointer dereference in org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) due to return value of called method Dereferenced at IOUtils.java:org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) due to return value of called method Dereferenced at IOUtils.java:[line 350] org.apache.hadoop.io.erasurecode.ECSchema.toString() makes inefficient use of keySet iterator instead of entrySet iterator At ECSchema.java:keySet iterator instead of entrySet
[jira] [Created] (HADOOP-14331) HadoopScheduledThreadPoolExecutor broken for periodic task running
Gabriel Reid created HADOOP-14331:
-
Summary: HadoopScheduledThreadPoolExecutor broken for periodic
task running
Key: HADOOP-14331
URL: https://issues.apache.org/jira/browse/HADOOP-14331
Project: Hadoop Common
Issue Type: Bug
Components: common
Affects Versions: 3.0.0-alpha3
Reporter: Gabriel Reid
The HadoopScheduledThreadPoolExecutor (introduced in HADOOP-12749) is broken
for the scheduling of periodic tasks (i.e. for tasks submitted with
{{scheduleAtFixedRate}} and {{scheduleAfterFixedDelay}}).
The behavior of the executor with these methods is that the underlying task is
executed once, and then blocks the running thread indefinitely in
{{ExecutorHelper::logThrowableFromAfterExecute}}, meaning further executions of
the task won't run, and will also block the running thread from running any
other tasks.
A quick scan of the source has shown me that these methods are used on
HadoopScheduledThreadPoolExecutor instances in several places in the code base,
at least in {{JobHistory}} and {{CleanerService}}, which appears to mean that
these classes also no longer function correctly.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
