thinkharderdev commented on code in PR #93:
URL: https://github.com/apache/arrow-ballista/pull/93#discussion_r929088794
##########
ballista/rust/scheduler/src/flight_sql.rs:
##########
@@ -0,0 +1,402 @@
+use std::collections::HashMap;
+use std::sync::{Arc, Mutex};
+use std::time::Duration;
+use arrow_flight::{FlightData, FlightDescriptor, FlightEndpoint, FlightInfo,
Location, Ticket};
+use arrow_flight::flight_descriptor::DescriptorType;
+use arrow_flight::flight_service_server::FlightService;
+use arrow_flight::sql::{ActionClosePreparedStatementRequest,
ActionCreatePreparedStatementRequest, ActionCreatePreparedStatementResult,
CommandGetCatalogs, CommandGetCrossReference, CommandGetDbSchemas,
CommandGetExportedKeys, CommandGetImportedKeys, CommandGetPrimaryKeys,
CommandGetSqlInfo, CommandGetTables, CommandGetTableTypes,
CommandPreparedStatementQuery, CommandPreparedStatementUpdate,
CommandStatementQuery, CommandStatementUpdate, SqlInfo, TicketStatementQuery};
+use arrow_flight::sql::server::FlightSqlService;
+use log::{debug, error};
+use tonic::{Response, Status, Streaming};
+
+use crate::scheduler_server::SchedulerServer;
+use datafusion_proto::protobuf::LogicalPlanNode;
+use ballista_core::{
+ serde::protobuf::{PhysicalPlanNode},
+};
+use ballista_core::config::BallistaConfig;
+use arrow_flight::SchemaAsIpc;
+use datafusion::arrow;
+use datafusion::arrow::datatypes::Schema;
+use datafusion::arrow::ipc::writer::{IpcDataGenerator, IpcWriteOptions};
+use datafusion::logical_expr::LogicalPlan;
+use tokio::time::sleep;
+use uuid::{Uuid};
+use crate::scheduler_server::event::QueryStageSchedulerEvent;
+use ballista_core::serde::protobuf::job_status;
+use ballista_core::serde::protobuf::JobStatus;
+use ballista_core::serde::protobuf;
+use prost::Message;
+
+#[derive(Clone)]
+pub struct FlightSqlServiceImpl {
+ server: SchedulerServer<LogicalPlanNode, PhysicalPlanNode>,
+ statements: Arc<Mutex<HashMap<Uuid, LogicalPlan>>>,
Review Comment:
Sort of. There is a bit more structure imposed by the `StateBackendClient`
to try and avoid lock contention in the scheduler. So you have to define a new
`Keyspace` enum for the new data to live in. The it's probably best to add
methods in `TaskManager` to save/fetch the logical plans. That can wrap the
serde logic and interaction with the state backend.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
