xushiyan commented on code in PR #162:
URL: https://github.com/apache/hudi-rs/pull/162#discussion_r1799914632
##########
crates/datafusion/src/lib.rs:
##########
@@ -287,4 +330,73 @@ mod tests {
verify_data_with_replacecommits(&ctx, &sql,
test_table.as_ref()).await
}
}
+
+ #[tokio::test]
+ async fn datafusion_read_external_hudi_table() {
+ for test_table in &[
+ V6ComplexkeygenHivestyle,
+ V6Nonpartitioned,
+ V6SimplekeygenNonhivestyle,
+ V6SimplekeygenHivestyleNoMetafields,
+ V6TimebasedkeygenNonhivestyle,
+ ] {
+ println!(">>> testing for {}", test_table.as_ref());
+ let ctx = prepare_session_context_with_table_factory().await;
+ let base_path = test_table.path();
+
+ let create_table_sql = format!(
+ "CREATE EXTERNAL TABLE {} STORED AS HUDITABLE LOCATION '{}'",
+ test_table.as_ref(),
+ base_path
+ );
+
+ let _ = ctx
+ .sql(create_table_sql.as_str())
+ .await
+ .expect("Failed to register table");
+
+ let sql = format!(
+ r#"
+ SELECT id, name, isActive, structField.field2
+ FROM {} WHERE id % 2 = 0
+ AND structField.field2 > 30 ORDER BY name LIMIT 10"#,
+ test_table.as_ref()
+ );
+
+ verify_data(&ctx, &sql, test_table.as_ref()).await
+ }
+ }
+
+ #[tokio::test]
+ async fn datafusion_read_external_hudi_table_with_replacecommits() {
+ for (test_table, planned_input_partitions) in
+ &[(V6SimplekeygenNonhivestyleOverwritetable, 1)]
+ {
+ println!(">>> testing for {}", test_table.as_ref());
Review Comment:
avoid print statement; tests don't say anything unless they fail
##########
crates/datafusion/src/lib.rs:
##########
@@ -287,4 +330,73 @@ mod tests {
verify_data_with_replacecommits(&ctx, &sql,
test_table.as_ref()).await
}
}
+
+ #[tokio::test]
+ async fn datafusion_read_external_hudi_table() {
+ for test_table in &[
+ V6ComplexkeygenHivestyle,
+ V6Nonpartitioned,
+ V6SimplekeygenNonhivestyle,
+ V6SimplekeygenHivestyleNoMetafields,
+ V6TimebasedkeygenNonhivestyle,
+ ] {
+ println!(">>> testing for {}", test_table.as_ref());
+ let ctx = prepare_session_context_with_table_factory().await;
+ let base_path = test_table.path();
+
+ let create_table_sql = format!(
+ "CREATE EXTERNAL TABLE {} STORED AS HUDITABLE LOCATION '{}'",
Review Comment:
i think `HUDITABLE` is a bit redundant - we already know it's creating a
table. More importantly, in other sql engine like spark/flink/hive, we all use
`HUDI` to indicate the format. So let's be consistent and simplify this as
`HUDI`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
