[
https://issues.apache.org/jira/browse/HUDI-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prathit Malik updated HUDI-8406:
--------------------------------
Description:
Currently spark.ui.port is defaulting to 8090
For ref :
[https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java#L330]
[https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java#L347]
Due to which custom spark ui port cannot be provided.
As part of this ticket, a new config will be added to
[https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/HoodieStreamer.java#L222]
to accept --spark-ui-port which will default to 8090
was:
Currently spark.ui.port is hardcoded to 8090
For ref :
[https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java#L330]
[https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java#L347]
Due to which custom spark ui port cannot be provided.
As part of this ticket, a new config will be added to
[https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/HoodieStreamer.java#L222]
to accept --spark-ui-port which will default to 8090
> Support to provide custom spark ui port for HoodieStreamer
> ----------------------------------------------------------
>
> Key: HUDI-8406
> URL: https://issues.apache.org/jira/browse/HUDI-8406
> Project: Apache Hudi
> Issue Type: Task
> Components: deltastreamer
> Reporter: Prathit Malik
> Priority: Minor
> Fix For: 1.0.0
>
>
> Currently spark.ui.port is defaulting to 8090
> For ref :
> [https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java#L330]
> [https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java#L347]
> Due to which custom spark ui port cannot be provided.
> As part of this ticket, a new config will be added to
> [https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/HoodieStreamer.java#L222]
> to accept --spark-ui-port which will default to 8090
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
