[
https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085380#comment-17085380
]
Sasikumar Venkatesh edited comment on HUDI-773 at 4/17/20, 3:38 AM:
--------------------------------------------------------------------
Hi [~vinoth] and [~garyli1019]
I am experimenting Hudi on ADLS Gen 2,
{code:java}
outputPath = abfss://tpch@<account>.dfs.core.windows.net/hudi-tables
df.write.format("hudi").options(hudiOptions)...
.save(outputPath){code}
I have changed the output dir to the above path and when I write using Hudi
format. It is showing the error, _*Configuration property
[<<account>>.dfs.core.windows.net|http://re230bb35spstg.dfs.core.windows.net/]
not found*_. I have configured ADLS FS and FSImpl class in spark using OAUTH
and I can write to ADLS2 using the below code.
{code:java}
df.write.format("parquet").save(outputPath){code}
Is there any specific config related to ADLS2 in Hudi for connecting the ALDS
storage account using OAuth.
Thanks and regards,
Sasikumar
was (Author: sasikumar.venkat):
Hi [~vinoth] and [~garyli1019]
I am experimenting Hudi on ADLS Gen 2,
{code:java}
outputPath = abfss://tpch@<account>.dfs.core.windows.net/hudi-tables
df.write.format("hudi").options(hudiOptions)...
.save(outputPath){code}
I have changed the output dir to the above path and when I write using Hudi
format. It is showing the error, _*Configuration property
[<<account>>.dfs.core.windows.net|http://re230bb35spstg.dfs.core.windows.net/]
not found*_. I have configured ADLS FS and FSImpl class in spark using OAUTH
and I can write to ADLS2 using the below code.
{code:java}
df.write.format("parquet").save(outputPath){code}
Is there any specific config related to ADLS2 for connecting the storage
account using OAuth while using Hudi.
Thanks and regards,
Sasikumar
> Hudi On Azure Data Lake Storage V2
> ----------------------------------
>
> Key: HUDI-773
> URL: https://issues.apache.org/jira/browse/HUDI-773
> Project: Apache Hudi (incubating)
> Issue Type: New Feature
> Components: Usability
> Reporter: Yanjia Gary Li
> Assignee: Yanjia Gary Li
> Priority: Minor
> Fix For: 0.6.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
