GitHub user krishna-prasad-s closed a discussion: Does Datafusion support
versions for delta table
I'm working on a merge activity where I want to replicate the checkpoint
behaviour of structured spark streaming.
I am working with delta tables and I have access to table versioning.
I am thinking to apply the version number as an alternative to structured spark
streaming's checkpoint.
Does datafusion support creating an queriying between versions?
for e.g.
` pub async fn get_df_from_version(&mut self, table_name: &str, version:
i64) -> DataFrame {
let sql = &format!("SELECT * FROM {} VERSION BETWEEN {} AND current",
table_name, version);
self.ctx.sql(sql).await.unwrap()
}
`
GitHub link: https://github.com/apache/datafusion/discussions/10041
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
