milenkovicm commented on code in PR #1351:
URL:
https://github.com/apache/datafusion-ballista/pull/1351#discussion_r2678827078
##########
ballista/core/proto/ballista.proto:
##########
@@ -642,6 +642,7 @@ message JobStatus {
message GetJobStatusResult {
JobStatus status = 1;
+ optional string flight_endpoint = 2;
Review Comment:
would it make sense to name this as "flight proxy" or similar?
##########
ballista/scheduler/src/scheduler_process.rs:
##########
@@ -131,7 +180,24 @@ pub async fn start_server(
) -> ballista_core::error::Result<()> {
info!("Ballista v{BALLISTA_VERSION} Scheduler listening on {address:?}");
let scheduler =
- create_scheduler::<LogicalPlanNode, PhysicalPlanNode>(cluster,
config).await?;
+ create_scheduler::<LogicalPlanNode, PhysicalPlanNode>(cluster,
config.clone())
+ .await?;
- start_grpc_service(address, scheduler).await
+ info!(
+ "advertise_flight_sql_endpoint: {:?}",
+ config.advertise_flight_sql_endpoint
+ );
+ match config.advertise_flight_sql_endpoint {
+ Some(_) => {
+ info!("Starting flight proxy");
+ let flight_proxy = start_flight_proxy_server(config,
scheduler.state.clone());
Review Comment:
Would it make sense to run proxy as a service on the same port with a
scheduler service? It would simplify configuration.
##########
ballista/core/src/execution_plans/distributed_query.rs:
##########
@@ -327,7 +328,10 @@ async fn execute_query(
let mut prev_status: Option<job_status::Status> = None;
loop {
- let GetJobStatusResult { status } = scheduler
+ let GetJobStatusResult {
+ status,
+ flight_endpoint,
Review Comment:
would it make sense to support Some("") in which case client should use
scheduler address and port ? This way scheduler should not relly care about its
public port?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
