[jira] [Resolved] (HADOOP-7675) Ant option to run disabled kerberos authentication tests.
[ https://issues.apache.org/jira/browse/HADOOP-7675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers resolved HADOOP-7675. Resolution: Won't Fix This ticket is ancient history. Resolving. > Ant option to run disabled kerberos authentication tests. > - > > Key: HADOOP-7675 > URL: https://issues.apache.org/jira/browse/HADOOP-7675 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Jitendra Nath Pandey >Priority: Major > > The kerberos tests, TestKerberosAuthenticator and > TestKerberosAuthenticationHandler, are disabled using @Ignore. A better > approach would be to have an ant option to run them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/ [Feb 22, 2018 7:32:32 PM] (xyao) HDFS-13136. Avoid taking FSN lock while doing group member lookup for [Feb 22, 2018 9:08:30 PM] (eyang) YARN-7836. Added error check for updating service components. [Feb 22, 2018 9:12:40 PM] (eyang) YARN-7942. Add check for JAAS configuration for Yarn Service. [Feb 23, 2018 2:12:12 AM] (carlo curino) YARN-7934. [GQ] Refactor preemption calculators to allow overriding for -1 overall The following subsystems voted -1: findbugs unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api org.apache.hadoop.yarn.api.records.Resource.getResources() may expose internal representation by returning Resource.resources At Resource.java:by returning Resource.resources At Resource.java:[line 234] Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.hdfs.TestDFSStripedOutputStreamWithFailure hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 hadoop.hdfs.TestErasureCodingPoliciesWithRandomECPolicy hadoop.hdfs.TestDFSStripedOutputStream hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.fs.http.server.TestHttpFSServerWebServer hadoop.yarn.server.nodemanager.webapp.TestContainerLogsPage hadoop.yarn.client.TestApplicationMasterServiceProtocolForTimelineV2 hadoop.yarn.applications.distributedshell.TestDistributedShell cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/diff-compile-javac-root.txt [280K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/diff-checkstyle-root.txt [17M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/whitespace-eol.txt [9.2M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/whitespace-tabs.txt [288K] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/diff-javadoc-javadoc-root.txt [760K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/patch-unit-hadoop-common-project_hadoop-kms.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [368K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt [24K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt [48K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt [20K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt [84K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/701/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core.txt [8.0K] Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12585) [Umbrella] Removing the usages of deprecated methods
[ https://issues.apache.org/jira/browse/HADOOP-12585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor resolved HADOOP-12585. --- Resolution: Done Target Version/s: (was: ) This umbrella is no longer in use. > [Umbrella] Removing the usages of deprecated methods > > > Key: HADOOP-12585 > URL: https://issues.apache.org/jira/browse/HADOOP-12585 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Tsuyoshi Ozawa >Priority: Major > > There are lots usages of deprecated methods in hadoop - we should avoid using > them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15260) Hive queries on tez are overwriting records on azure wasb storage.
Vineet Singh created HADOOP-15260: - Summary: Hive queries on tez are overwriting records on azure wasb storage. Key: HADOOP-15260 URL: https://issues.apache.org/jira/browse/HADOOP-15260 Project: Hadoop Common Issue Type: Bug Components: fs/azure Affects Versions: 2.7.3 Environment: This scenario occurs on hdp 2.5(hadoop 2.7.3) hdfs on WASB microsoft Azure platform. The same query yields proper result on regular hdfs on hdp 2.5(hadoop 2.7.3) on premise cluster. Reporter: Vineet Singh Attachments: On Premise Cluster.JPG, azure cloud.JPG, sample_query.txt When running multiple hive queries on Tez (see example ) the same mapper task number gets overwritten by the next union query. As seen in the azure snapshot the directories /1 ,2 ...,100 get overwritten again and again since the mapper numbers launch write again and again in the same directories. But in the on premise hadoop cluster version 2.7.3 . The directories are created as 1_copy_0,1_copy_2 and so on. Creating copies does not overwrite the data. The queries would be usually 600-1000 queries union together. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-2829) JT should consider the disk each task is on before scheduling jobs...
[ https://issues.apache.org/jira/browse/HADOOP-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor resolved HADOOP-2829. -- Resolution: Invalid It seems obsolete. > JT should consider the disk each task is on before scheduling jobs... > - > > Key: HADOOP-2829 > URL: https://issues.apache.org/jira/browse/HADOOP-2829 > Project: Hadoop Common > Issue Type: Improvement >Reporter: eric baldeschwieler >Priority: Major > > The DataNode can support a JBOD config, where blocks exist on explicit disks. > But this information is not exported or considered by the JT when assigning > tasks. This leads to non-optimal disk use. if 4 slots are used, 2 running > tasks will likely be on the same disk and we observe them running more slowly > then other tasks on the same machine. > We could follow a number of strategies to address this. > for example: The data nodes could support a what disk is this block on call. > Then the JT could discover the info and assign jobs accordingly. > Of course the TT itself uses disks for merge and temp space and the datanodes > on the same machine can be used by off node sources, so it is not clear > optimizing all of this is simple enough to be worth it. > This issue deserves study. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-6291) Confusing warn message from Configuration
[
https://issues.apache.org/jira/browse/HADOOP-6291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor resolved HADOOP-6291.
--
Resolution: Duplicate
> Confusing warn message from Configuration
> -
>
> Key: HADOOP-6291
> URL: https://issues.apache.org/jira/browse/HADOOP-6291
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Affects Versions: 0.21.0
>Reporter: Tsz Wo Nicholas Sze
>Priority: Major
>
> Starting a cluster without setting mapreduce.task.attempt.id and then
> {noformat}
> $ ./bin/hadoop fs -put README.txt r.txt
> 09/09/29 22:28:10 WARN conf.Configuration: mapred.task.id is deprecated.
> Instead, use mapreduce.task.attempt.id
> 09/09/29 22:28:10 INFO hdfs.DFSClient: Done flushing
> 09/09/29 22:28:10 INFO hdfs.DFSClient: Closing the streams...
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15257) Provide example docker compose file for developer builds
Elek, Marton created HADOOP-15257: - Summary: Provide example docker compose file for developer builds Key: HADOOP-15257 URL: https://issues.apache.org/jira/browse/HADOOP-15257 Project: Hadoop Common Issue Type: Sub-task Reporter: Elek, Marton Assignee: Elek, Marton This issue is about creating example docker-compose files which use the latest build from the hadoop-dist directory. These docker-compose files would help to run a specific hadoop cluster based on the latest custom build without the need to build customized docker image (with mounting hadoop fro hadoop-dist to the container -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15256) Create docker images for latest stable hadoop3 build
Elek, Marton created HADOOP-15256: - Summary: Create docker images for latest stable hadoop3 build Key: HADOOP-15256 URL: https://issues.apache.org/jira/browse/HADOOP-15256 Project: Hadoop Common Issue Type: Sub-task Reporter: Elek, Marton Assignee: Elek, Marton Similar to the hadoop2 image we can provide a developer hadoop image which contains the latest hadoop from the binary release. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15258) Create example docker-compse file for documentations
Elek, Marton created HADOOP-15258: - Summary: Create example docker-compse file for documentations Key: HADOOP-15258 URL: https://issues.apache.org/jira/browse/HADOOP-15258 Project: Hadoop Common Issue Type: Sub-task Reporter: Elek, Marton Assignee: Elek, Marton An other user case for docker is to use it in the documentation. For example in the HA documentation we can provide an example docker-compose file and configuration with all the required settings to getting started easily with an HA cluster. 1. I would add an example to a documetation page 2. It will use the hadoop3 image (which contains latest hadoop3) as the user of the documentation may not build a hadoop -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15255) Upper/Lower case conversion support for group names in LdapGroupsMapping
Nanda kumar created HADOOP-15255:
Summary: Upper/Lower case conversion support for group names in
LdapGroupsMapping
Key: HADOOP-15255
URL: https://issues.apache.org/jira/browse/HADOOP-15255
Project: Hadoop Common
Issue Type: Improvement
Reporter: Nanda kumar
On most LDAP servers the user and group names are case-insensitive. When we use
{{JniBasedUnixGroupsMappingWithFallback}} and have {{SSSD}} in place, it is
possible to configure {{SSSD}} to force the group names to be returned in
lowercase. If we use {{LdapGroupsMapping}}, we don't have any such option.
This jira proposes to introduce a new {{hadoop.security.group.mapping}}
implementation based on LdapGroupsMapping which supports force lower/upper case
group names.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
