YARN SLS improving idea

[Sichen Zhao]

Hi,
I am a developer from AliBaBa China, i recently used SLS for scheduling 
simulation, SLS currently supports multidimensional resource input(CPU, mem , 
other resources: disk). But SLS can't take scheduling request, which is 
currently widely used in YARN, as input, so Placement Constraints and 
attributes are not supported.

So what i wanna improve the SLS: Add scheduling emulation for scheduling 
request resource format.

The specific work is as follows:
1. Add input support for the scheduling request format.
2. Add support for scheduling request resource format in NMSim.
3. Adding scheduling request support for the Capacity Scheduler(maybe it is 
already done in current version).

What do you think about my ideas?


Best Regards
Sichen Zhao

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

[Apache Jenkins Server]

For more details, see https://builds.apache.org/job/hadoop-trunk-win/553/

[Aug 8, 2018 2:13:29 PM] (sunilg) YARN-8633. Update DataTables version in 
yarn-common in line with JQuery
[Aug 8, 2018 2:18:49 PM] (sunilg) Revert "YARN-8633. Update DataTables version 
in yarn-common in line with
[Aug 8, 2018 3:31:55 PM] (elek) HDFS-13447. Fix Typos - Node Not Chosen. 
Contributed by Beluga Behr.
[Aug 8, 2018 5:12:52 PM] (bibinchundatt) YARN-8601. Print ExecutionType in 
Container report CLI. Contributed by
[Aug 8, 2018 5:40:20 PM] (xiao) HDFS-13658. Expose HighestPriorityLowRedundancy 
blocks statistics.
[Aug 8, 2018 8:02:12 PM] (wangda) Make 3.1.1 awared by other branches
[Aug 8, 2018 8:05:24 PM] (wangda) Make 3.1.1 awared by other branches - adding 
missing files
[Aug 8, 2018 10:08:55 PM] (rkanter) YARN-8568. Replace the deprecated 
zk-address property in the HA config
[Aug 8, 2018 11:47:25 PM] (hanishakoneru) HDDS-267. Handle consistency issues 
during container update/close.
[Aug 9, 2018 5:57:24 AM] (stevel) HADOOP-15583. Stabilize S3A Assumed Role 
support. Contributed by Steve
[Aug 9, 2018 6:48:32 AM] (sunilg) YARN-8633. Update DataTables version in 
yarn-common in line with JQuery
[Aug 9, 2018 9:06:03 AM] (elek) HDDS-219. Genearate version-info.properties for 
hadoop and ozone.
[Aug 9, 2018 12:26:37 PM] (elek) HDDS-344. Remove multibyte characters from 
OzoneAcl. Contributed by
[Aug 9, 2018 3:17:34 PM] (jlowe) YARN-8331. Race condition in NM container 
launched after done.




-1 overall


The following subsystems voted -1:
compile mvninstall pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h 00m 00s)
unit


Specific tests:

Failed junit tests :

   hadoop.crypto.key.kms.server.TestKMS 
   hadoop.cli.TestAclCLI 
   hadoop.cli.TestAclCLIWithPosixAclInheritance 
   hadoop.cli.TestCacheAdminCLI 
   hadoop.cli.TestCryptoAdminCLI 
   hadoop.cli.TestDeleteCLI 
   hadoop.cli.TestErasureCodingCLI 
   hadoop.cli.TestHDFSCLI 
   hadoop.cli.TestXAttrCLI 
   hadoop.fs.contract.hdfs.TestHDFSContractAppend 
   hadoop.fs.contract.hdfs.TestHDFSContractConcat 
   hadoop.fs.contract.hdfs.TestHDFSContractCreate 
   hadoop.fs.contract.hdfs.TestHDFSContractDelete 
   hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus 
   hadoop.fs.contract.hdfs.TestHDFSContractMkdir 
   hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader 
   hadoop.fs.contract.hdfs.TestHDFSContractOpen 
   hadoop.fs.contract.hdfs.TestHDFSContractPathHandle 
   hadoop.fs.contract.hdfs.TestHDFSContractRename 
   hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory 
   hadoop.fs.contract.hdfs.TestHDFSContractSeek 
   hadoop.fs.contract.hdfs.TestHDFSContractSetTimes 
   hadoop.fs.loadGenerator.TestLoadGenerator 
   hadoop.fs.permission.TestStickyBit 
   hadoop.fs.shell.TestHdfsTextCommand 
   hadoop.fs.TestEnhancedByteBufferAccess 
   hadoop.fs.TestFcHdfsCreateMkdir 
   hadoop.fs.TestFcHdfsPermission 
   hadoop.fs.TestFcHdfsSetUMask 
   hadoop.fs.TestGlobPaths 
   hadoop.fs.TestHDFSFileContextMainOperations 
   hadoop.fs.TestHdfsNativeCodeLoader 
   hadoop.fs.TestResolveHdfsSymlink 
   hadoop.fs.TestSWebHdfsFileContextMainOperations 
   hadoop.fs.TestSymlinkHdfsDisable 
   hadoop.fs.TestSymlinkHdfsFileContext 
   hadoop.fs.TestSymlinkHdfsFileSystem 
   hadoop.fs.TestUnbuffer 
   hadoop.fs.TestUrlStreamHandler 
   hadoop.fs.TestWebHdfsFileContextMainOperations 
   hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot 
   hadoop.fs.viewfs.TestViewFileSystemHdfs 
   hadoop.fs.viewfs.TestViewFileSystemLinkFallback 
   hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash 
   hadoop.fs.viewfs.TestViewFileSystemWithAcls 
   hadoop.fs.viewfs.TestViewFileSystemWithTruncate 
   hadoop.fs.viewfs.TestViewFileSystemWithXAttrs 
   hadoop.fs.viewfs.TestViewFsAtHdfsRoot 
   hadoop.fs.viewfs.TestViewFsDefaultValue 
   hadoop.fs.viewfs.TestViewFsFileStatusHdfs 
   hadoop.fs.viewfs.TestViewFsHdfs 
   hadoop.fs.viewfs.TestViewFsWithAcls 
   hadoop.fs.viewfs.TestViewFsWithXAttrs 
   hadoop.hdfs.client.impl.TestBlockReaderLocal 
   hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy 
   hadoop.hdfs.client.impl.TestBlockReaderRemote 
   hadoop.hdfs.client.impl.TestClientBlockVerification 
   hadoop.hdfs.crypto.TestHdfsCryptoStreams 
   hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer 
   hadoop.hdfs.qjournal.client.TestEpochsAreUnique 
   hadoop.hdfs.qjournal.client.TestQJMWithFaults 
   hadoop.hdfs.qjournal.client.TestQuorumJournalManager 
   hadoop.hdfs.qjournal.server.TestJournal 
   hadoop.hdfs.qjournal.server.TestJournalNode 
   

[jira] [Created] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

[Thomas Marquardt (JIRA)]

Thomas Marquardt created HADOOP-15664:
-

 Summary: ABFS: Reduce test run time via parallelization and 
grouping
 Key: HADOOP-15664
 URL: https://issues.apache.org/jira/browse/HADOOP-15664
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Reporter: Thomas Marquardt


1) Let's reduce the total test runtime by improving parallelization of the 
tests.

2) Let's make it possible to select WASB tests, ABFS tests, or both so 
developers can run only the tests appropriate for the change they've made.

3) Update the testing-azure.md accordingly



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15663) ABFS: Simplify configuration

[Thomas Marquardt (JIRA)]

Thomas Marquardt created HADOOP-15663:
-

 Summary: ABFS: Simplify configuration
 Key: HADOOP-15663
 URL: https://issues.apache.org/jira/browse/HADOOP-15663
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Reporter: Thomas Marquardt


Configuration for WASB and ABFS is too complex.  The current approach is to use 
four files for test configuration. 

Both WASB and ABFS have basic test configuration which is committed to the repo 
(azure-test.xml and azure-bfs-test.xml).  Currently these contain the 
fs.AbstractFileSystem.[scheme].impl configuration, but otherwise are empty 
except for an include reference to a file containing the endpoint credentials. 

Both WASB and ABFS have endpoint credential configuration files 
(azure-auth-keys.xml and azure-bfs-auth-keys.xml).  These have been added to 
.gitignore to prevent them from accidentally being submitted in a patch, which 
would leak the developers storage account credentials.  These files contain 
account names, storage account keys, and service endpoints.

There is some overlap of the configuration for WASB and ABFS, where they use 
the same property name but use different values.  

1) Let's reduce the number of test configuration files to one, if possible.

2) Let's simplify the account name, key, and endpoint configuration for WASB 
and ABFS if possible, but still support the legacy way of doing it, which is 
very error prone.

3) Let's improve error handling, so that typos or misconfiguration are not so 
difficult to troubleshoot.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15662) ABFS: Better exception handling of DNS errors

[Thomas Marquardt (JIRA)]

Thomas Marquardt created HADOOP-15662:
-

 Summary: ABFS: Better exception handling of DNS errors
 Key: HADOOP-15662
 URL: https://issues.apache.org/jira/browse/HADOOP-15662
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Reporter: Thomas Marquardt


DNS errors are common during testing due to typos or misconfiguration.  They 
can also occur in production, as some transient DNS issues occur from time to 
time. 

1) Let's investigate if we can distinguish between the two and fail fast for 
the test issues, but continue to have retry logic for the transient DNS issues 
in production.

2) Let's improve the error handling of DNS failures, so the user has an 
actionable error message.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15661) ABFS: Add support for ACL

[Da Zhou (JIRA)]

Da Zhou created HADOOP-15661:


 Summary: ABFS: Add support for ACL
 Key: HADOOP-15661
 URL: https://issues.apache.org/jira/browse/HADOOP-15661
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Da Zhou
Assignee: Da Zhou


- Add support for ACL



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15659) ABFS: Code changes for bug fix and new tests

[Da Zhou (JIRA)]

Da Zhou created HADOOP-15659:


 Summary: ABFS: Code changes for bug fix and new tests
 Key: HADOOP-15659
 URL: https://issues.apache.org/jira/browse/HADOOP-15659
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Da Zhou
Assignee: Da Zhou


- add bug fixes.
- remove unnecessary dependencies.
- add new tests for code changes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[ANNOUNCE] Apache Hadoop 3.1.1 release

[Wangda Tan]

It gives me great pleasure to announce that the Apache Hadoop community has
voted to release Apache Hadoop 3.1.1.

Hadoop 3.1.1 is the first stable maintenance release for the year 2018 in
the Hadoop-3.1 line and brings a number of enhancements.

IMPORTANT NOTES

3.1.1 is the first stable release of 3.1 line which is production-ready.

The Hadoop community fixed 435 JIRAs [1] in total as part of the 3.1.1
release. Of these fixes:

   -

   60 in Hadoop Common
   -

   139 in HDFS
   -

   223 in YARN
   -

   13 in MapReduce

--

Apache Hadoop 3.1.1 contains a number of significant features and
enhancements. A few of them are noted below.


   -

   ENTRY_POINT support for Docker containers.
   -

   Restart policy support for YARN native services.
   -

   Capacity Scheduler: Intra-queue preemption for fairness ordering policy.
   -

   Stabilization works for schedulers, YARN service, docker support, etc.


Please see the Hadoop 3.1.1 CHANGES

for the detailed list of issues resolved. The release news is posted on the
Apache Hadoop website too, you can go to the downloads section.


--

Many thanks to everyone who contributed to the release, and everyone in the
Apache Hadoop community! The release is a result of direct and indirect
efforts from many contributors, listed below are the those who contributed
directly by submitting patches and reporting issues.

Abhishek Modi, Ajay Kumar, Akhil PB, Akira Ajisaka, Allen Wittenauer,
Anbang Hu, Andrew Wang, Arpit Agarwal, Atul Sikaria, BELUGA BEHR, Bharat
Viswanadham, Bibin A Chundatt, Billie Rinaldi, Bilwa S T, Botong Huang,
Brahma Reddy Battula, Brook Zhou, CR Hota, Chandni Singh, Chao Sun, Charan
Hebri, Chen Liang, Chetna Chaudhari, Chun Chen, Daniel Templeton, Davide
 Vergari, Dennis Huo, Dibyendu Karmakar, Ekanth Sethuramalingam, Eric
Badger, Eric Yang, Erik Krogen, Esfandiar Manii, Ewan Higgs, Gabor Bota,
Gang Li, Gang Xie, Genmao Yu, Gergely Novák, Gergo Repas, Giovanni Matteo
Fumarola, Gour Saha, Greg Senia, Haibo Yan, Hanisha Koneru, Hsin-Liang
Huang, Hu Ziqian, Istvan Fajth, Jack Bearden, Jason Lowe, Jeff Zhang, Jian
He, Jianchao Jia, Jiandan Yang , Jim Brennan, Jinglun, John Zhuge, Joseph
Fourny, K G Bakthavachalam, Karthik Palanisamy, Kihwal Lee, Kitti Nanasi,
Konstantin Shvachko, Lei (Eddy) Xu, LiXin Ge, Lokesh Jain, Lukas Majercak,
Miklos Szegedi, Mukul Kumar Singh, Namit Maheshwari, Nanda kumar, Nilotpal
Nandi, Pavel Avgustinov, Prabhu Joseph, Prasanth Jayachandran, Robert
Kanter, Rohith Sharma K S, Rushabh S Shah, Sailesh Patel, Sammi Chen, Sean
Mackrory, Sergey Shelukhin, Shane Kumpf, Shashikant Banerjee, Siyao Meng,
Sreenath Somarajapuram, Steve Loughran, Suma Shivaprasad, Sumana Sathish,
Sunil Govindan, Surendra Singh Lilhore, Szilard Nemeth, Takanobu Asanuma,
Tao Jie, Tao Yang, Ted Yu, Thomas Graves, Thomas Marquardt, Todd Lipcon,
Vinod Kumar Vavilapalli, Wangda Tan, Wei Yan, Wei-Chiu Chuang, Weiwei Yang,
Wilfred Spiegelenburg, Xiao Chen, Xiao Liang, Xintong Song, Xuan Gong, Yang
Wang, Yesha Vora, Yiqun Lin, Yiran Wu, Yongjun Zhang, Yuanbo Liu, Zian
Chen, Zoltan Haindrich, Zsolt Venczel, Zuoming Zhang, fang zhenyi, john
lilley, jwhitter, kyungwan nam, liaoyuxiangqin, liuhongtong, lujie, skrho,
yanghuafeng, yimeng, Íñigo Goiri.

Wangda Tan

[1] JIRA query: project in (YARN, HADOOP, MAPREDUCE, HDFS) AND resolution =
Fixed AND fixVersion = 3.1.1 ORDER BY key ASC, updated ASC, created DESC,
priority DESC

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/

[Aug 8, 2018 2:13:09 AM] (mackrorysd) HADOOP-15400. Improve S3Guard 
documentation on Authoritative Mode
[Aug 8, 2018 2:23:17 AM] (vinayakumarb) HDFS-13786. EC: Display erasure coding 
policy for sub-directories is not
[Aug 8, 2018 5:05:17 AM] (xiao) HDFS-13728. Disk Balancer should not fail if 
volume usage is greater
[Aug 8, 2018 7:12:20 AM] (vinayakumarb) HDFS-13785. EC: 'removePolicy' is not 
working for built-in/system
[Aug 8, 2018 11:50:23 AM] (ewan.higgs) HADOOP-15576. S3A Multipart Uploader to 
work with S3Guard and encryption
[Aug 8, 2018 2:13:29 PM] (sunilg) YARN-8633. Update DataTables version in 
yarn-common in line with JQuery
[Aug 8, 2018 2:18:49 PM] (sunilg) Revert "YARN-8633. Update DataTables version 
in yarn-common in line with
[Aug 8, 2018 3:31:55 PM] (elek) HDFS-13447. Fix Typos - Node Not Chosen. 
Contributed by Beluga Behr.
[Aug 8, 2018 5:12:52 PM] (bibinchundatt) YARN-8601. Print ExecutionType in 
Container report CLI. Contributed by
[Aug 8, 2018 5:40:20 PM] (xiao) HDFS-13658. Expose HighestPriorityLowRedundancy 
blocks statistics.
[Aug 8, 2018 8:02:12 PM] (wangda) Make 3.1.1 awared by other branches
[Aug 8, 2018 8:05:24 PM] (wangda) Make 3.1.1 awared by other branches - adding 
missing files
[Aug 8, 2018 10:08:55 PM] (rkanter) YARN-8568. Replace the deprecated 
zk-address property in the HA config
[Aug 8, 2018 11:47:25 PM] (hanishakoneru) HDDS-267. Handle consistency issues 
during container update/close.




-1 overall


The following subsystems voted -1:
asflicense findbugs pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed CTEST tests :

   test_test_libhdfs_threaded_hdfs_static 
   test_libhdfs_threaded_hdfspp_test_shim_static 

Failed junit tests :

   hadoop.security.TestRaceWhenRelogin 
   hadoop.util.TestBasicDiskValidator 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.mapred.TestMRTimelineEventHandling 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/diff-compile-javac-root.txt
  [332K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/diff-checkstyle-root.txt
  [4.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/diff-patch-shelldocs.txt
  [16K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/whitespace-eol.txt
  [9.4M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/whitespace-tabs.txt
  [1.1M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/xml.txt
  [4.0K]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [52K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-hdds_framework.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [16K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [28K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/863/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt
  [4.0K]
   

[jira] [Created] (HADOOP-15658) Memory leak in S3AOutputStream

[Piotr Nowojski (JIRA)]

Piotr Nowojski created HADOOP-15658:
---

 Summary: Memory leak in S3AOutputStream
 Key: HADOOP-15658
 URL: https://issues.apache.org/jira/browse/HADOOP-15658
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs/s3
Affects Versions: 2.8.4
Reporter: Piotr Nowojski


S3AOutputStream by calling 
{{org.apache.hadoop.fs.s3a.S3AFileSystem#createTmpFileForWrite}} indirectly 
calls {{java.io.File#deleteOnExit}} which is known for memory leaking:
 
[https://bugs.java.com/view_bug.do?bug_id=6664633]
[https://bugs.java.com/view_bug.do?bug_id=4872014]
 
Apparently it was even fixed for different component couple of years ago 
https://issues.apache.org/jira/browse/HADOOP-8635 . 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15657) Adding support to registering MutableQuantiles from Metric annotation

[Sushil Ks (JIRA)]

Sushil Ks created HADOOP-15657:
--

 Summary: Adding support to registering MutableQuantiles from 
Metric annotation
 Key: HADOOP-15657
 URL: https://issues.apache.org/jira/browse/HADOOP-15657
 Project: Hadoop Common
  Issue Type: Improvement
  Components: metrics
Reporter: Sushil Ks


Currently when creating new metrics we use @Metric annotation for registering 
the MutableMetric i.e
{code:java}
@Metric
private MutableInt foobarMetricCount
{code}
However, currently there's no support for registering MutableQuantiles via 
Metric annotation, hence creating this Jira to register MutableQuantiles vis 
Metric annotation. 

Example:

 
{code:java}
@Metric(about = "async PUT entities latency", valueName = "latency", interval = 
10)
private MutableQuantiles foobarAsyncLatency;
 
{code}
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15656) Support byteman in hadoop-runner baseimage

[Elek, Marton (JIRA)]

Elek, Marton created HADOOP-15656:
-

 Summary: Support byteman in hadoop-runner baseimage
 Key: HADOOP-15656
 URL: https://issues.apache.org/jira/browse/HADOOP-15656
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Elek, Marton
Assignee: Elek, Marton


[Byteman|http://byteman.jboss.org/] is an easy to use tool to instrument a java 
process with agent string.

For example [this 
script|https://gist.githubusercontent.com/elek/0589a91b4d55afb228279f6c4f04a525/raw/8bb4e03de7397c8a9d9bb74a5ec80028b42575c4/hadoop.btm]
 defines a rule to print out all the hadoop rpc traffic to the standard output 
(which is extremely useful for testing development).

This patch adds the byteman.jar to the baseimage and defines a simple logic to 
add agent instrumentation string to the HADOOP_OPTS (optional it also could 
download the byteman script from an url)





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org