[jira] [Commented] (HUDI-146) Impala Support

2019-11-05 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968043#comment-16968043 ] Yanjia Gary Li commented on HUDI-146: - Hello [~vinoth], Yuanbin finished his internship a few months

[jira] [Commented] (HUDI-146) Impala Support

2019-11-06 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968638#comment-16968638 ] Yanjia Gary Li commented on HUDI-146: - [~vinoth] is there any hudi related code in the Hive code base? 

[jira] [Created] (HUDI-318) Update Migration Guide to Include Delta Streamer

2019-10-31 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-318: --- Summary: Update Migration Guide to Include Delta Streamer Key: HUDI-318 URL: https://issues.apache.org/jira/browse/HUDI-318 Project: Apache Hudi (incubating)

[jira] [Created] (HUDI-415) HoodieSparkSqlWriter Commit time not representing the Spark job starting time

2019-12-16 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-415: --- Summary: HoodieSparkSqlWriter Commit time not representing the Spark job starting time Key: HUDI-415 URL: https://issues.apache.org/jira/browse/HUDI-415 Project:

[jira] [Commented] (HUDI-259) Hadoop 3 support for Hudi writing

2019-12-12 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995153#comment-16995153 ] Yanjia Gary Li commented on HUDI-259: - Hello, I recently started using Hadoop 3 and Spark 2.4. 

[jira] [Updated] (HUDI-415) HoodieSparkSqlWriter Commit time not representing the Spark job starting time

2019-12-20 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-415: Status: Closed (was: Patch Available) > HoodieSparkSqlWriter Commit time not representing the Spark

[jira] [Updated] (HUDI-415) HoodieSparkSqlWriter Commit time not representing the Spark job starting time

2019-12-20 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-415: Status: Patch Available (was: In Progress) > HoodieSparkSqlWriter Commit time not representing the

[jira] [Commented] (HUDI-415) HoodieSparkSqlWriter Commit time not representing the Spark job starting time

2019-12-20 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001084#comment-17001084 ] Yanjia Gary Li commented on HUDI-415: - PR merged. Issue resolved. > HoodieSparkSqlWriter Commit time

[jira] [Created] (HUDI-610) Impala nea real time table support

2020-02-13 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-610: --- Summary: Impala nea real time table support Key: HUDI-610 URL: https://issues.apache.org/jira/browse/HUDI-610 Project: Apache Hudi (incubating) Issue Type:

[jira] [Created] (HUDI-611) Impala sync tool

2020-02-13 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-611: --- Summary: Impala sync tool Key: HUDI-611 URL: https://issues.apache.org/jira/browse/HUDI-611 Project: Apache Hudi (incubating) Issue Type: New Feature

[jira] [Created] (HUDI-644) Enable to retrieve checkpoint from previous commits in Delta Streamer

2020-02-26 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-644: --- Summary: Enable to retrieve checkpoint from previous commits in Delta Streamer Key: HUDI-644 URL: https://issues.apache.org/jira/browse/HUDI-644 Project: Apache Hudi

[jira] [Updated] (HUDI-644) Enable to retrieve checkpoint from previous commits in Delta Streamer

2020-03-03 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-644: Status: Open (was: New) > Enable to retrieve checkpoint from previous commits in Delta Streamer >

[jira] [Updated] (HUDI-644) Enable to retrieve checkpoint from previous commits in Delta Streamer

2020-03-03 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-644: Status: In Progress (was: Open) > Enable to retrieve checkpoint from previous commits in Delta

[jira] [Updated] (HUDI-644) Enable to retrieve checkpoint from previous commits in Delta Streamer

2020-03-03 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-644: Fix Version/s: 0.6.0 > Enable to retrieve checkpoint from previous commits in Delta Streamer >

[jira] [Closed] (HUDI-315) Reimplement statistics/workload profile collected during writes using Spark 2.x custom accumulators

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li closed HUDI-315. --- Resolution: Won't Fix > Reimplement statistics/workload profile collected during writes using Spark >

[jira] [Commented] (HUDI-315) Reimplement statistics/workload profile collected during writes using Spark 2.x custom accumulators

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047137#comment-17047137 ] Yanjia Gary Li commented on HUDI-315: - Agree. Closing this ticket.  > Reimplement statistics/workload

[jira] [Resolved] (HUDI-415) HoodieSparkSqlWriter Commit time not representing the Spark job starting time

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-415. - Resolution: Fixed > HoodieSparkSqlWriter Commit time not representing the Spark job starting time

[jira] [Reopened] (HUDI-415) HoodieSparkSqlWriter Commit time not representing the Spark job starting time

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reopened HUDI-415: - > HoodieSparkSqlWriter Commit time not representing the Spark job starting time >

[jira] [Commented] (HUDI-315) Reimplement statistics/workload profile collected during writes using Spark 2.x custom accumulators

2020-02-26 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045986#comment-17045986 ] Yanjia Gary Li commented on HUDI-315: - I will take a look at this ticket > Reimplement

[jira] [Assigned] (HUDI-315) Reimplement statistics/workload profile collected during writes using Spark 2.x custom accumulators

2020-02-26 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reassigned HUDI-315: --- Assignee: Yanjia Gary Li > Reimplement statistics/workload profile collected during writes

[jira] [Updated] (HUDI-597) Enable incremental pulling from defined partitions

2020-03-01 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-597: Description: For the use case that I only need to pull the incremental part of certain partitions,

[jira] [Resolved] (HUDI-597) Enable incremental pulling from defined partitions

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-597. - Resolution: Fixed PR merged. Will update the DOC after 0.5.2 release > Enable incremental pulling

[jira] [Updated] (HUDI-597) Enable incremental pulling from defined partitions

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-597: Fix Version/s: 0.5.2 > Enable incremental pulling from defined partitions >

[jira] [Updated] (HUDI-597) Enable incremental pulling from defined partitions

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-597: Status: Open (was: New) > Enable incremental pulling from defined partitions >

[jira] [Updated] (HUDI-611) Add Impala Guide to Doc

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-611: Status: Open (was: New) > Add Impala Guide to Doc > --- > >

[jira] [Resolved] (HUDI-611) Add Impala Guide to Doc

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-611. - Resolution: Fixed > Add Impala Guide to Doc > --- > > Key:

[jira] [Updated] (HUDI-611) Add Impala Guide to Doc

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-611: Status: In Progress (was: Open) > Add Impala Guide to Doc > --- > >

[jira] [Updated] (HUDI-597) Enable incremental pulling from defined partitions

2020-02-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-597: Status: In Progress (was: Open) > Enable incremental pulling from defined partitions >

[jira] [Created] (HUDI-597) Enable incremental pulling from defined partitions

2020-02-03 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-597: --- Summary: Enable incremental pulling from defined partitions Key: HUDI-597 URL: https://issues.apache.org/jira/browse/HUDI-597 Project: Apache Hudi (incubating)

[jira] [Resolved] (HUDI-146) Impala Support

2020-02-11 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-146. - Resolution: Done read optimized table now support by Impala. Fixed by: 

[jira] [Updated] (HUDI-611) Add Impala Guide to Doc

2020-02-21 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-611: Summary: Add Impala Guide to Doc (was: Impala sync tool) > Add Impala Guide to Doc >

[jira] [Updated] (HUDI-611) Add Impala Guide to Doc

2020-02-21 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-611: Priority: Minor (was: Major) > Add Impala Guide to Doc > --- > >

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-01-02 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Description: I am using the manual build master after 

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-01-02 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Attachment: Screen Shot 2020-01-02 at 8.53.44 PM.png > [DEBUGGING] Huge amount of tasks when writing

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-01-02 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Attachment: Screen Shot 2020-01-02 at 8.53.24 PM.png > [DEBUGGING] Huge amount of tasks when writing

[jira] [Created] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-01-02 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-494: --- Summary: [DEBUGGING] Huge amount of tasks when writing files into HDFS Key: HUDI-494 URL: https://issues.apache.org/jira/browse/HUDI-494 Project: Apache Hudi

[jira] [Commented] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-01-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008199#comment-17008199 ] Yanjia Gary Li commented on HUDI-494: - Hello [~lamber-ken], Thanks for trying this out. This behavior

[jira] [Commented] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-01-07 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010031#comment-17010031 ] Yanjia Gary Li commented on HUDI-494: - [~vinoth] Thanks for the feedback. The code snippets were

[jira] [Updated] (HUDI-644) checkpoint generator tool for delta streamer

2020-03-11 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-644: Summary: checkpoint generator tool for delta streamer (was: Enable to retrieve checkpoint from

[jira] [Updated] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-08 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-773: Summary: Hudi On Azure Data Lake Storage V2 (was: Hudi On Azure Data Lake Storage) > Hudi On Azure

[jira] [Created] (HUDI-773) Hudi On Azure Data Lake Storage

2020-04-08 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-773: --- Summary: Hudi On Azure Data Lake Storage Key: HUDI-773 URL: https://issues.apache.org/jira/browse/HUDI-773 Project: Apache Hudi (incubating) Issue Type: New

[jira] [Resolved] (HUDI-759) Integrate checkpoint provider

2020-04-14 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-759. - Resolution: Fixed > Integrate checkpoint provider > - > >

[jira] [Comment Edited] (HUDI-69) Support realtime view in Spark datasource #136

2020-04-14 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082773#comment-17082773 ] Yanjia Gary Li edited comment on HUDI-69 at 4/14/20, 10:11 PM: --- After a closer

[jira] [Assigned] (HUDI-765) Implement OrcReaderIterator

2020-04-15 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reassigned HUDI-765: --- Assignee: Yanjia Gary Li > Implement OrcReaderIterator > --- > >

[jira] [Commented] (HUDI-791) Replace null by Option in Delta Streamer

2020-04-17 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085942#comment-17085942 ] Yanjia Gary Li commented on HUDI-791: - [~tison] Thanks for looking into this ticket! The initiative

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-17 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086087#comment-17086087 ] Yanjia Gary Li commented on HUDI-773: - Hello [~sasikumar.venkat], I am very new to Azure. How is your

[jira] [Created] (HUDI-805) Verify which types of Azure storage support Hudi

2020-04-17 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-805: --- Summary: Verify which types of Azure storage support Hudi Key: HUDI-805 URL: https://issues.apache.org/jira/browse/HUDI-805 Project: Apache Hudi (incubating)

[jira] [Created] (HUDI-804) Add Azure Support to Hudi Doc

2020-04-17 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-804: --- Summary: Add Azure Support to Hudi Doc Key: HUDI-804 URL: https://issues.apache.org/jira/browse/HUDI-804 Project: Apache Hudi (incubating) Issue Type:

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-16 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085409#comment-17085409 ] Yanjia Gary Li commented on HUDI-773: - Hello [~sasikumar.venkat], thanks for sharing! I am able to

[jira] [Commented] (HUDI-69) Support realtime view in Spark datasource #136

2020-04-13 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082773#comment-17082773 ] Yanjia Gary Li commented on HUDI-69: After a closer look, I think Spark datasource support for realtime

[jira] [Created] (HUDI-791) Replace null by Option in Delta Streamer

2020-04-13 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-791: --- Summary: Replace null by Option in Delta Streamer Key: HUDI-791 URL: https://issues.apache.org/jira/browse/HUDI-791 Project: Apache Hudi (incubating) Issue

[jira] [Updated] (HUDI-791) Replace null by Option in Delta Streamer

2020-04-13 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-791: Issue Type: Improvement (was: New Feature) > Replace null by Option in Delta Streamer >

[jira] [Assigned] (HUDI-30) Explore support for Spark Datasource V2

2020-04-12 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-30?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reassigned HUDI-30: -- Assignee: Yanjia Gary Li > Explore support for Spark Datasource V2 >

[jira] [Updated] (HUDI-30) Explore support for Spark Datasource V2

2020-04-12 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-30?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-30: --- Status: In Progress (was: Open) > Explore support for Spark Datasource V2 >

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-20 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087994#comment-17087994 ] Yanjia Gary Li commented on HUDI-773: - [~sasikumar.venkat] I haven't tried Databricks Spark myself, but

[jira] [Created] (HUDI-822) Decouple hoodie related methods with Hoodie Input Formats

2020-04-20 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-822: --- Summary: Decouple hoodie related methods with Hoodie Input Formats Key: HUDI-822 URL: https://issues.apache.org/jira/browse/HUDI-822 Project: Apache Hudi (incubating)

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-10 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081030#comment-17081030 ] Yanjia Gary Li commented on HUDI-773: - surprisingly easy...I tried the following test using Spark2.4

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-10 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081032#comment-17081032 ] Yanjia Gary Li commented on HUDI-773: - Any extra tests needed? What tests have you guys done for AWS

[jira] [Updated] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-10 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-773: Status: In Progress (was: Open) > Hudi On Azure Data Lake Storage V2 >

[jira] [Updated] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-10 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-773: Fix Version/s: 0.6.0 > Hudi On Azure Data Lake Storage V2 > -- > >

[jira] [Commented] (HUDI-69) Support realtime view in Spark datasource #136

2020-03-31 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072382#comment-17072382 ] Yanjia Gary Li commented on HUDI-69: [~vinoth] I am happy to work on this ticket. Please assign to me >

[jira] [Created] (HUDI-759) Integrate checkpoint provider

2020-04-03 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-759: --- Summary: Integrate checkpoint provider Key: HUDI-759 URL: https://issues.apache.org/jira/browse/HUDI-759 Project: Apache Hudi (incubating) Issue Type: New

[jira] [Updated] (HUDI-759) Integrate checkpoint provider

2020-04-03 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-759: Status: Open (was: New) > Integrate checkpoint provider > - > >

[jira] [Updated] (HUDI-759) Integrate checkpoint provider

2020-04-03 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-759: Status: In Progress (was: Open) > Integrate checkpoint provider > - > >

[jira] [Resolved] (HUDI-644) checkpoint generator tool for delta streamer

2020-04-03 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-644. - Resolution: Fixed > checkpoint generator tool for delta streamer >

[jira] [Updated] (HUDI-69) Support realtime view in Spark datasource #136

2020-04-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-69: --- Status: In Progress (was: Open) > Support realtime view in Spark datasource #136 >

[jira] [Commented] (HUDI-69) Support realtime view in Spark datasource #136

2020-04-05 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076023#comment-17076023 ] Yanjia Gary Li commented on HUDI-69: Hello [~bhasudha], I found your commit 

[jira] [Updated] (HUDI-644) checkpoint generator tool for delta streamer

2020-03-27 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-644: Description: This ticket is to resolve the following problem: The user has finished the initial

[jira] [Updated] (HUDI-69) Support realtime view in Spark datasource #136

2020-04-25 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-69: --- Description: [https://github.com/uber/hudi/issues/136] RFC: 

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

2020-04-23 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091042#comment-17091042 ] Yanjia Gary Li commented on HUDI-773: - Hello [~sasikumar.venkat], could you try the following: mount

[jira] [Updated] (HUDI-69) Support realtime view in Spark datasource #136

2020-04-21 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-69: --- Description: [https://github.com/uber/hudi/issues/136] RFC: 

[jira] [Reopened] (HUDI-69) Support realtime view in Spark datasource #136

2020-05-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reopened HUDI-69: > Support realtime view in Spark datasource #136 > -- > >

[jira] [Issue Comment Deleted] (HUDI-69) Support realtime view in Spark datasource #136

2020-05-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-69: --- Comment: was deleted (was: Can anyone reopen this ticket? I accidentally closed this :)) > Support

[jira] [Updated] (HUDI-69) Support realtime view in Spark datasource #136

2020-05-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-69: --- Status: Closed (was: Patch Available) > Support realtime view in Spark datasource #136 >

[jira] [Commented] (HUDI-69) Support realtime view in Spark datasource #136

2020-05-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099507#comment-17099507 ] Yanjia Gary Li commented on HUDI-69: Can anyone reopen this ticket? I accidentally closed this :) >

[jira] [Updated] (HUDI-822) Decouple hoodie related methods with Hoodie Input Formats

2020-05-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-822: Status: In Progress (was: Open) > Decouple hoodie related methods with Hoodie Input Formats >

[jira] [Updated] (HUDI-69) Support realtime view in Spark datasource #136

2020-05-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-69: --- Description: [https://github.com/uber/hudi/issues/136] RFC: 

[jira] [Updated] (HUDI-69) Support realtime view in Spark datasource #136

2020-05-04 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-69: --- Status: Patch Available (was: In Progress) > Support realtime view in Spark datasource #136 >

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-12 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Fix Version/s: 0.5.3 > [DEBUGGING] Huge amount of tasks when writing files into HDFS >

[jira] [Updated] (HUDI-528) Incremental Pull fails when latest commit is empty

2020-05-12 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-528: Fix Version/s: 0.5.3 > Incremental Pull fails when latest commit is empty >

[jira] [Assigned] (HUDI-318) Update Migration Guide to Include Delta Streamer

2020-05-12 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reassigned HUDI-318: --- Assignee: (was: Yanjia Gary Li) > Update Migration Guide to Include Delta Streamer >

[jira] [Updated] (HUDI-110) Better defaults for Partition extractor for Spark DataSOurce and DeltaStreamer

2020-05-17 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-110: Status: In Progress (was: Open) > Better defaults for Partition extractor for Spark DataSOurce and

[jira] [Commented] (HUDI-890) Prepare for 0.5.3 patch release

2020-05-17 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109805#comment-17109805 ] Yanjia Gary Li commented on HUDI-890: - Hi [~bhavanisudha] , #1602 HUDI-494 fix incorrect record size

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-17 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Fix Version/s: (was: 0.5.3) > [DEBUGGING] Huge amount of tasks when writing files into HDFS >

[jira] [Created] (HUDI-905) Support native filter pushdown for Spark Datasource

2020-05-17 Thread Yanjia Gary Li (Jira)
Yanjia Gary Li created HUDI-905: --- Summary: Support native filter pushdown for Spark Datasource Key: HUDI-905 URL: https://issues.apache.org/jira/browse/HUDI-905 Project: Apache Hudi (incubating)

[jira] [Commented] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-06 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101207#comment-17101207 ] Yanjia Gary Li commented on HUDI-494: -   Commit 1: {code:java} "partitionToWriteStats" : {

[jira] [Assigned] (HUDI-528) Incremental Pull fails when latest commit is empty

2020-05-10 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reassigned HUDI-528: --- Assignee: Yanjia Gary Li > Incremental Pull fails when latest commit is empty >

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-10 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Status: In Progress (was: Open) > [DEBUGGING] Huge amount of tasks when writing files into HDFS >

[jira] [Updated] (HUDI-528) Incremental Pull fails when latest commit is empty

2020-05-10 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-528: Status: In Progress (was: Open) > Incremental Pull fails when latest commit is empty >

[jira] [Resolved] (HUDI-528) Incremental Pull fails when latest commit is empty

2020-05-15 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-528. - Resolution: Fixed > Incremental Pull fails when latest commit is empty >

[jira] [Commented] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-06 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100967#comment-17100967 ] Yanjia Gary Li commented on HUDI-494: - Hi folks, this issue seems coming back again...

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-06 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Status: Open (was: New) > [DEBUGGING] Huge amount of tasks when writing files into HDFS >

[jira] [Assigned] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-06 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li reassigned HUDI-494: --- Assignee: Yanjia Gary Li (was: Vinoth Chandar) > [DEBUGGING] Huge amount of tasks when

[jira] [Commented] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-06 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101055#comment-17101055 ] Yanjia Gary Li commented on HUDI-494: - Ok, I see what happened here. Root cause is 

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-06 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Attachment: example2_hdfs.png > [DEBUGGING] Huge amount of tasks when writing files into HDFS >

[jira] [Updated] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-06 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-494: Attachment: example2_sparkui.png > [DEBUGGING] Huge amount of tasks when writing files into HDFS >

[jira] [Comment Edited] (HUDI-494) [DEBUGGING] Huge amount of tasks when writing files into HDFS

2020-05-07 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101055#comment-17101055 ] Yanjia Gary Li edited comment on HUDI-494 at 5/8/20, 1:38 AM: -- -Ok, I see what

[jira] [Updated] (HUDI-905) Support PrunedFilteredScan for Spark Datasource

2020-05-20 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-905: Priority: Minor (was: Major) > Support PrunedFilteredScan for Spark Datasource >

[jira] [Updated] (HUDI-905) Support PrunedFilteredScan for Spark Datasource

2020-05-20 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-905: Status: Open (was: New) > Support PrunedFilteredScan for Spark Datasource >

[jira] [Updated] (HUDI-905) Support PrunedFilteredScan for Spark Datasource

2020-05-20 Thread Yanjia Gary Li (Jira)
[ https://issues.apache.org/jira/browse/HUDI-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-905: Component/s: Spark Integration > Support PrunedFilteredScan for Spark Datasource >

  1   2   3   4   5   6   7   8   9   >