[
https://issues.apache.org/jira/browse/KYLIN-4895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366381#comment-17366381
]
ASF GitHub Bot commented on KYLIN-4895:
---------------------------------------
zzcclp commented on a change in pull request #1568:
URL: https://github.com/apache/kylin/pull/1568#discussion_r655075865
##########
File path:
kylin-spark-project/kylin-spark-engine/src/main/java/org/apache/kylin/engine/spark/job/NSparkExecutable.java
##########
@@ -382,6 +387,8 @@ protected String generateSparkCmd(KylinConfig config,
String hadoopConf, String
if (StringUtils.isNotBlank(sparkUploadFiles)) {
sb.append("--files ").append(sparkUploadFiles).append(" ");
}
+ sb.append("--principal
").append(config.getKerberosPrincipal()).append(" ");
+ sb.append("--keytab ").append(config.getKerberosKeytabPath()).append("
");
Review comment:
why it needs these two lines directly ? please check the code in the
method 'replaceSparkNodeJavaOpsConfIfNeeded', it will set the parameters about
kerberos according to 'config.isKerberosEnabled()'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> change spark deploy mode of kylin4.0 engine from local to cluster
> -----------------------------------------------------------------
>
> Key: KYLIN-4895
> URL: https://issues.apache.org/jira/browse/KYLIN-4895
> Project: Kylin
> Issue Type: New Feature
> Components: Job Engine
> Affects Versions: v4.0.0-alpha
> Reporter: tianhui
> Assignee: tianhui
> Priority: Major
> Fix For: v4.0.0
>
>
> In cloud native environment, the memory of pod is quite limited. But the
> spark Driver can use a huge amount of memory, in job engine pod, which is
> quite difficult to manage when there are more and more projects.
> So it's better to put spark driver on yarn, and kylin only maintain its
> status.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
