alitrack commented on issue #20758:
URL: https://github.com/apache/superset/issues/20758#issuecomment-1211540895
@LittleYmada @manojrustagi79
I have two solutions for this,
### [DuckDB
version](https://mp.weixin.qq.com/s?__biz=MzU1NTg2ODQ5Nw==&mid=2247487993&idx=1&sn=0068a1f79c386f76a3cb14bb79637c9d&chksm=fbcc99c7ccbb10d1be9143be8f1af35f994ff8414ac1b550763603f6e182bb43a443baa9bc8e&token=1997429354&lang=zh_CN#rd)
- install packages
```bash
pip install duckdb-engine deltalake
# if want to support partition, needs duckdb-0.4.1.dev1057 at least.
pip install -U --pre duckdb
```
- superset-config.py
```python
# activate Jinja templating
FEATURE_FLAGS = {
# . . . . .
"ENABLE_TEMPLATE_PROCESSING": True
}
def delta_table(table_name):
from deltalake import DeltaTable
dt = DeltaTable(table_name)
return dt.file_uris()
JINJA_CONTEXT_ADDONS = {
'delta_table': delta_table
}
```
- in SQL Lab
```sql
SELECT * from read_parquet({{delta_table('/Users/steven/data/iris')}})
```
want to support partitons,
```sql
SELECT * from
parquet_scan({{delta_table('/Users/steven/data/iris1')}},HIVE_PARTITIONING=true)
```
### [PostgreSQL version
](https://mp.weixin.qq.com/s?__biz=MzU1NTg2ODQ5Nw==&mid=2247487935&idx=1&sn=cd893d1f5ce5acc7cce1f65fa4984d8c&chksm=fbcc9981ccbb1097a5b61c2298536401186d072395935c82ea32402b31b379f2a0d1888d4b26&token=1997429354&lang=zh_CN#rd)
tested on Postgres 14 and PL/Python (Python3.9)
- install packages
```bash
pip install deltalake typing-extensions
```
- plpython function
```sql
create or replace function read_iris(iris_path text)
returns table(sepal_length REAL,sepal_width REAL
,petal_length REAL,petal_width REAL
,species text)
as $$
from deltalake import DeltaTable
dt = DeltaTable(iris_path)
df= dt.to_pandas()
return df.values.tolist()
$$ language plpython3u;
```
- test function
```sql
select * from read_iris('/Users/steven/data/iris');
```
- then you can create view
```
create or replace view iris as
select * from read_iris('/Users/steven/data/iris');
```
- or materialized view
```
create materialized view iris1 as
select * from read_iris('/Users/steven/data/iris');
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
