[
https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799282#comment-16799282
]
Eric Yang commented on YARN-7129:
---------------------------------
Patch 033 includes [[email protected]] and [~jeagles] feedbacks.
Time benchmark for building application catalog on a 2015 i7 MacBook Pro with
Redhat 7 as guest vm.
Benchmark command in hadoop-yarn-applications-catalog folder:
{code}
mvn clean package -Pdocker
{code}
Fresh checkout with no docker cache:
{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] YARN Application Catalog ........................... SUCCESS [ 1.090 s]
[INFO] YARN Application Catalog Webapp .................... SUCCESS [ 23.444 s]
[INFO] YARN Application Catalog Docker Image .............. SUCCESS [02:19 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:44 min
[INFO] Finished at: 2019-03-22T15:16:19-04:00
[INFO] Final Memory: 108M/906M
[INFO] ------------------------------------------------------------------------
{code}
After maven and docker cache are built once:
{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] YARN Application Catalog ........................... SUCCESS [ 1.268 s]
[INFO] YARN Application Catalog Webapp .................... SUCCESS [ 24.019 s]
[INFO] YARN Application Catalog Docker Image .............. SUCCESS [ 32.326 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 58.348 s
[INFO] Finished at: 2019-03-22T14:31:19-04:00
[INFO] Final Memory: 106M/981M
[INFO] ------------------------------------------------------------------------
{code}
Skipping docker build (mvn clean package):
{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] YARN Application Catalog ........................... SUCCESS [ 1.151 s]
[INFO] YARN Application Catalog Webapp .................... SUCCESS [ 25.091 s]
[INFO] YARN Application Catalog Docker Image .............. SUCCESS [ 0.144 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.074 s
[INFO] Finished at: 2019-03-22T14:35:18-04:00
[INFO] Final Memory: 87M/917M
[INFO] ------------------------------------------------------------------------
{code}
Base on the result, it's good evidence to say that if developer can be patient
on fresh build once. Having docker build inline does not introduce much
overhead for developer that needs to rapidly reiterate building process.
When using profile to generate artifact, some part of code might not be
exercised daily, and hide bugs from surfacing until release time. I hope
Hadoop community will consider to make docker build process inline some day.
> Application Catalog for YARN applications
> -----------------------------------------
>
> Key: YARN-7129
> URL: https://issues.apache.org/jira/browse/YARN-7129
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: applications
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN Appstore.pdf, YARN-7129.001.patch,
> YARN-7129.002.patch, YARN-7129.003.patch, YARN-7129.004.patch,
> YARN-7129.005.patch, YARN-7129.006.patch, YARN-7129.007.patch,
> YARN-7129.008.patch, YARN-7129.009.patch, YARN-7129.010.patch,
> YARN-7129.011.patch, YARN-7129.012.patch, YARN-7129.013.patch,
> YARN-7129.014.patch, YARN-7129.015.patch, YARN-7129.016.patch,
> YARN-7129.017.patch, YARN-7129.018.patch, YARN-7129.019.patch,
> YARN-7129.020.patch, YARN-7129.021.patch, YARN-7129.022.patch,
> YARN-7129.023.patch, YARN-7129.024.patch, YARN-7129.025.patch,
> YARN-7129.026.patch, YARN-7129.027.patch, YARN-7129.028.patch,
> YARN-7129.029.patch, YARN-7129.030.patch, YARN-7129.031.patch,
> YARN-7129.032.patch, YARN-7129.033.patch
>
>
> YARN native services provides web services API to improve usability of
> application deployment on Hadoop using collection of docker images. It would
> be nice to have an application catalog system which provides an editorial and
> search interface for YARN applications. This improves usability of YARN for
> manage the life cycle of applications.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
