alexandreyc commented on code in PR #1725: URL: https://github.com/apache/arrow-adbc/pull/1725#discussion_r1568710105
########## rust2/core/src/lib.rs: ########## @@ -0,0 +1,520 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +//! ADBC: Arrow Database Connectivity +//! +//! ADBC is a set of APIs and libraries for [Arrow](https://arrow.apache.org/)-native +//! access to databases. Execute SQL and [Substrait](https://substrait.io/) +//! queries, query database catalogs, and more, all using Arrow data to +//! eliminate unnecessary data copies, speed up access, and make it more +//! convenient to build analytical applications. +//! +//! Read more about ADBC at <https://arrow.apache.org/adbc/> +//! +//! This library currently provides: +//! - An abstract Rust API to be implemented by vendor-specific drivers. +//! - A driver manager which implements this same API, but dynamically loads +//! drivers internally and forwards calls appropriately using the [C API](https://github.com/apache/arrow-adbc/blob/main/adbc.h). +//! +//! We hope to provide the ability to expose Rust drivers as C drivers in the +//! near future permitting other languages to use Rust drivers. +//! +//! # Native Rust drivers +//! +//! Native Rust drivers will implement the abstract API consisting of the traits: +//! - [Driver] +//! - [Database] +//! - [Connection] +//! - [Statement] +//! +//! For drivers implemented in Rust, using these will be more efficient and +//! safe, since it avoids the overhead of going through C FFI. +//! +//! # Driver Manager +//! +//! The [driver_manager] module allows loading drivers exposing the C API, +//! either from an initialization function (link-time, either static or dynamic) +//! or by dynamically finding such a function in a dynamic library (run-time). + +// TODO(alexandreyc): uncomment these lines during follow-up PRs +// pub mod driver_exporter; +// pub mod driver_manager; +// pub use ffi::FFI_AdbcDriverInitFunc as AdbcDriverInitFunc; + +pub mod error; +pub mod ffi; +pub mod options; +pub mod schemas; + +use arrow::datatypes::Schema; +use arrow::record_batch::{RecordBatch, RecordBatchReader}; + +use error::Result; +use options::{OptionConnection, OptionDatabase, OptionStatement, OptionValue}; + +/// Ability to configure an object by setting/getting options. +pub trait Optionable { + type Option: AsRef<str>; + + /// Set a post-init option. + fn set_option(&mut self, key: Self::Option, value: OptionValue) -> Result<()>; + + /// Get a string option value by key. + fn get_option_string(&self, key: Self::Option) -> Result<String>; + + /// Get a bytes option value by key. + fn get_option_bytes(&self, key: Self::Option) -> Result<Vec<u8>>; + + /// Get an integer option value by key. + fn get_option_int(&self, key: Self::Option) -> Result<i64>; + + /// Get a float option value by key. + fn get_option_double(&self, key: Self::Option) -> Result<f64>; +} + +/// A handle to an ADBC driver. +pub trait Driver { + type DatabaseType: Database; + + /// Allocate and initialize a new database without pre-init options. + fn new_database(&self) -> Result<Self::DatabaseType>; + + /// Allocate and initialize a new database with pre-init options. + fn new_database_with_opts( + &self, + opts: impl Iterator<Item = (OptionDatabase, OptionValue)>, + ) -> Result<Self::DatabaseType>; +} + +/// A handle to an ADBC database. +/// +/// Databases hold state shared by multiple connections. This typically means +/// configuration and caches. For in-memory databases, it provides a place to +/// hold ownership of the in-memory database. +/// +/// Databases must be kept alive as long as any connections exist. +pub trait Database: Optionable<Option = OptionDatabase> { + type ConnectionType: Connection; + + /// Allocate and initialize a new connection without pre-init options. + fn new_connection(&self) -> Result<Self::ConnectionType>; + + /// Allocate and initialize a new connection with pre-init options. + fn new_connection_with_opts( + &self, + opts: impl Iterator<Item = (options::OptionConnection, OptionValue)>, Review Comment: > We could also define a non-fieldless enum and accept an iterator, but that allows duplicate values for the same key, so I really think a product type is better here. That's what I did in a first version but it wasn't a great fit because we also need to query option values with `get_option_*` and in that case we only need the option key and not the value (that would force us to provide a sentinel in the non-fieldless enum variant). Regarding the product type, IMO it's not a great fit either: 1. Databases not necessarily need all options. For instance, SQLite only needs `uri`, it even does not recognize `username` nor `password` (we could wrap all the fields in `Option` though). 2. Everytime a new "standard" key is added we'll have to add a new field to this struct. 3. It's harder for the user to split between pre-init and post-init options, because in that case he would need to define two instances of the struct with non-overlapping non-`None` fields. The only downside I see with the current solution is the possibility to provide duplicated keys... We could document that "last write win" strategy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
