liurenjie1024 commented on code in PR #1827:
URL: https://github.com/apache/iceberg-rust/pull/1827#discussion_r2512426868
##########
crates/integrations/datafusion/src/task_writer.rs:
##########
@@ -139,17 +132,29 @@ impl<B: IcebergWriterBuilder> TaskWriter<B> {
SupportedWriter::Clustered(ClusteredWriter::new(writer_builder))
};
+ // Initialize partition splitter in constructor for partitioned tables
+ let partition_splitter = if !partition_spec.is_unpartitioned() {
+ Some(
+ RecordBatchPartitionSplitter::new_with_precomputed_values(
+ schema.clone(),
+ partition_spec.clone(),
+ )
+ .expect("Failed to create partition splitter"),
Review Comment:
We shoud not panic, but throwing error instead.
##########
crates/integrations/datafusion/src/physical_plan/write.rs:
##########
@@ -277,31 +265,40 @@ impl ExecutionPlan for IcebergWriteExec {
);
let data_file_writer_builder =
DataFileWriterBuilder::new(rolling_writer_builder);
+ // Create TaskWriter
+ // TODO: Make fanout_enabled configurable via table properties
Review Comment:
Please create an issue to track this.
##########
crates/integrations/datafusion/tests/integration_datafusion_test.rs:
##########
@@ -810,3 +814,147 @@ async fn test_insert_into_nested() -> Result<()> {
Ok(())
}
+
+#[tokio::test]
+async fn test_insert_into_partitioned() -> Result<()> {
Review Comment:
Could we move this test to newly added sqllogictests? I think we should be
able to read/write now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
