[
https://issues.apache.org/jira/browse/EAGLE-704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15624392#comment-15624392
]
ASF GitHub Bot commented on EAGLE-704:
--------------------------------------
Github user wujinhu commented on a diff in the pull request:
https://github.com/apache/incubator-eagle/pull/591#discussion_r85876323
--- Diff:
eagle-jpm/eagle-jpm-spark-history/src/main/resources/META-INF/providers/org.apache.eagle.jpm.spark.history.SparkHistoryJobAppProvider.xml
---
@@ -18,179 +18,127 @@
<application>
<type>SPARK_HISTORY_JOB_APP</type>
- <name>Spark History Job Monitoring</name>
+ <name>Spark History Job Monitor</name>
<version>0.5.0-incubating</version>
<appClass>org.apache.eagle.jpm.spark.history.SparkHistoryJobApp</appClass>
<configuration>
- <!-- org.apache.eagle.jpm.spark.history.SparkHistoryJobAppConfig
-->
+ <!-- topology config -->
<property>
- <name>basic.cluster</name>
- <displayName>cluster</displayName>
- <description>Cluster Name</description>
- <value>sandbox</value>
+ <name>workers</name>
+ <displayName>topology workers</displayName>
+ <description>topology workers</description>
+ <value>1</value>
</property>
<property>
- <name>basic.dataCenter</name>
- <displayName>dataCenter</displayName>
- <description>Data Center</description>
- <value>sandbox</value>
+ <name>topology.numOfSpoutExecutors</name>
+ <displayName>spout executors</displayName>
+ <description>Parallelism of sparkHistoryJobFetchSpout
</description>
+ <value>1</value>
</property>
<property>
- <name>basic.jobConf.additional.info</name>
- <displayName>jobConf.additional.info</displayName>
- <description>Additional info in Job Configs</description>
- <value></value>
+ <name>topology.numOfSpoutTasks</name>
+ <displayName>spout tasks</displayName>
+ <description>Tasks Num of sparkHistoryJobFetchSpout
</description>
+ <value>4</value>
+ </property>
+ <property>
+ <name>topology.numOfParseBoltExecutors</name>
+ <displayName>parser bolt parallelism hint</displayName>
+ <description>Parallelism of sparkHistoryJobParseBolt
</description>
+ <value>1</value>
+ </property>
+ <property>
+ <name>topology.numOfParserBoltTasks</name>
+ <displayName>parser bolt tasks</displayName>
+ <description>Tasks Num of
sparkHistoryJobParseBolt</description>
+ <value>4</value>
+ </property>
+ <property>
+ <name>topology.spoutCrawlInterval</name>
+ <displayName>spout crawl interval</displayName>
+ <description>Spout crawl interval (in
milliseconds)</description>
+ <value>10000</value>
</property>
<property>
- <name>dataSourceConfig.zkQuorum</name>
- <displayName>zkQuorum</displayName>
- <description>Zookeeper Quorum</description>
+ <name>topology.message.timeout.secs</name>
+ <displayName>topology message timeout (secs)</displayName>
+ <description>default timeout is 30s</description>
+ <value>300</value>
+ </property>
+ <!-- zookeeper config -->
+ <property>
+ <name>zkStateConfig.zkQuorum</name>
+ <displayName>zookeeper quorum list</displayName>
+ <description>zookeeper to store topology metadata</description>
--- End diff --
zookeeper configure is in eagle server config, we can reuse
> Update spark history config to integrate with the new application framework
> ---------------------------------------------------------------------------
>
> Key: EAGLE-704
> URL: https://issues.apache.org/jira/browse/EAGLE-704
> Project: Eagle
> Issue Type: Improvement
> Reporter: Zhao, Qingwen
> Assignee: Zhao, Qingwen
>
> This ticket will update SparkHistoryJobAppProvider.xml to integrate with
> application framework. In this way, users can manage this topology on the web
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
