[
https://issues.apache.org/jira/browse/HUDI-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tao meng updated HUDI-1662:
---------------------------
Description:
step1: prepare raw DataFrame with DateType, and insert it to HudiMorTable
df_raw.withColumn("date", lit(Date.valueOf("2020-11-10")))
merge(df_raw, "bulk_insert", "huditest.bulkinsert_mor_10g")
step2: prepare update DataFrame with DateType, and upsert into HudiMorTable
df_update = sql("select * from
huditest.bulkinsert_mor_10g_rt").withColumn("date",
lit(Date.valueOf("2020-11-11")))
merge(df_update, "upsert", "huditest.bulkinsert_mor_10g")
step3: use hive-beeeline/ spark-sql query mor_rt table
!image-2021-03-05-10-06-11-949.png!
was:
step1: prepare raw DataFrame with DateType, and insert it to HudiMorTable
df_raw.withColumn("date", lit(Date.valueOf("2020-11-10")))
merge(df_raw, "bulk_insert", "huditest.bulkinsert_mor_10g")
step2: prepare update DataFrame with DateType, and upsert into HudiMorTable
df_update = sql("select * from
huditest.bulkinsert_mor_10g_rt").withColumn("date",
lit(Date.valueOf("2020-11-11")))
merge(df_update, "upsert", "huditest.bulkinsert_mor_10g")
step3: use hive-beeeline/ spark-sql query mor_rt table
!file:///C:/Users/m00443775/AppData/Roaming/eSpace_Desktop/UserData/m00443775/imagefiles/847287BC-86FF-4EFF-9BDC-C2A60FFD3F47.png|id=847287BC-86FF-4EFF-9BDC-C2A60FFD3F47,vspace=3!
> Failed to query real-time view use hive/spark-sql when hudi mor table
> contains dateType
> ----------------------------------------------------------------------------------------
>
> Key: HUDI-1662
> URL: https://issues.apache.org/jira/browse/HUDI-1662
> Project: Apache Hudi
> Issue Type: Bug
> Components: Hive Integration
> Affects Versions: 0.7.0
> Environment: hive 3.1.1
> spark 2.4.5
> hadoop 3.1.1
> suse os
> Reporter: tao meng
> Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> step1: prepare raw DataFrame with DateType, and insert it to HudiMorTable
> df_raw.withColumn("date", lit(Date.valueOf("2020-11-10")))
> merge(df_raw, "bulk_insert", "huditest.bulkinsert_mor_10g")
> step2: prepare update DataFrame with DateType, and upsert into HudiMorTable
> df_update = sql("select * from
> huditest.bulkinsert_mor_10g_rt").withColumn("date",
> lit(Date.valueOf("2020-11-11")))
> merge(df_update, "upsert", "huditest.bulkinsert_mor_10g")
>
> step3: use hive-beeeline/ spark-sql query mor_rt table
> !image-2021-03-05-10-06-11-949.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
