tbar4 commented on code in PR #1100:
URL:
https://github.com/apache/datafusion-ballista/pull/1100#discussion_r1835165509
##########
python/src/lib.rs:
##########
@@ -15,18 +15,59 @@
// specific language governing permissions and limitations
// under the License.
+use ballista::prelude::*;
+use datafusion::prelude::*;
+use datafusion_python::context::PySessionContext as
DataFusionPythonSessionContext;
+use datafusion_python::utils::wait_for_future;
+use std::collections::HashMap;
+
use pyo3::prelude::*;
-pub mod context;
mod utils;
-pub use crate::context::PySessionContext;
-
#[pymodule]
-fn pyballista_internal(_py: Python, m: Bound<'_, PyModule>) -> PyResult<()> {
+fn ballista_internal(_py: Python, m: Bound<'_, PyModule>) -> PyResult<()> {
pyo3_log::init();
// Ballista structs
- m.add_class::<PySessionContext>()?;
+ m.add_class::<PyBallista>()?;
// DataFusion structs
m.add_class::<datafusion_python::dataframe::PyDataFrame>()?;
Ok(())
}
+
+#[pyclass(name = "Ballista", module = "ballista", subclass)]
+pub struct PyBallista(pub Option<HashMap<String, String>>);
+
+#[pymethods]
+impl PyBallista {
+ #[staticmethod]
+ #[pyo3(signature = (config=None))]
+ pub fn config(config: Option<HashMap<String, String>>) -> Self {
Review Comment:
I will set a default empty config if none is provided at Class creation, but
you can also provide it with class creation. I also set it up so you can adjust
the config after class creation. I was kind of thinking this might be
interesting because in the context of spark a lot of times you will set your
config to optimize for a particular type of workload initially, then you have
to start a completely new config type to optimize for a separate workload.
Regardless, the implementation was simple and allows for options when
defining config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
