joellubi commented on code in PR #38385:
URL: https://github.com/apache/arrow/pull/38385#discussion_r1424138793
##########
format/FlightSql.proto:
##########
@@ -1779,9 +1796,60 @@ message CommandPreparedStatementUpdate {
}
/*
- * Returned from the RPC call DoPut when a CommandStatementUpdate
- * CommandPreparedStatementUpdate was in the request, containing
- * results from the update.
+ * Represents a bulk ingestion request. Used in the command member of
FlightDescriptor
+ * for the the RPC call DoPut to cause the server load the contents of the
stream's
+ * FlightData into the target destination.
+ */
+message CommandStatementIngest {
Review Comment:
I've given some thought to potentially splitting up the "create" and
"append" operations. Having creation be a separate operation would be nice, but
I do also see the slippery slope of adding more configuration over time for
various edge cases.
It would definitely be valuable to have _some_ method of abstracting table
creation logic in a way that's independent of the backend. I did some research
and learned that Substrait already has a
[relation](https://substrait.io/relations/logical_relations/#ddl-data-definition-language-operator)
defined for this. For this reason, it seems that independent "create" and
"append" operations are more aligned with Substrait's stated goals rather than
those of Flight SQL. I think that past a certain level of detail, a Substrait
DDL + Write plan should be the supported/preferred approach to handle these
kinds of scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
