vinothchandar commented on a change in pull request #1200: [HUDI-514] A schema
provider to get metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200#discussion_r366183041
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java
vinothchandar commented on a change in pull request #1200: [HUDI-514] A schema
provider to get metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200#discussion_r366183326
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java
vinothchandar commented on a change in pull request #1200: [HUDI-514] A schema
provider to get metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200#discussion_r366184290
##
File path:
hudi-utilities/src/test/resources/delta-streamer-config/source-jdbc.avsc
vinothchandar commented on a change in pull request #1200: [HUDI-514] A schema
provider to get metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200#discussion_r366182135
##
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java
vinothchandar commented on a change in pull request #1200: [HUDI-514] A schema
provider to get metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200#discussion_r366183554
##
File path:
zhedoubushishi opened a new pull request #1226: [HUDI-238] Make Hudi support
Scala 2.12
URL: https://github.com/apache/incubator-hudi/pull/1226
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening
OpenOpened commented on issue #1200: [HUDI-514] A schema provider to get
metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200#issuecomment-574033535
@vinothchandar
Please review. I mainly did something:
1. Added test cases
2. All logic is implemented
[
https://issues.apache.org/jira/browse/HUDI-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang updated HUDI-531:
--
Description: Currently, the general classes (under src/main dir) has no
java docs. We should add doc for those
vinoyang created HUDI-532:
-
Summary: Add java doc for hudi test suite test classes
Key: HUDI-532
URL: https://issues.apache.org/jira/browse/HUDI-532
Project: Apache Hudi (incubating)
Issue Type:
vinoyang created HUDI-531:
-
Summary: Add java doc for hudi test suite general classes
Key: HUDI-531
URL: https://issues.apache.org/jira/browse/HUDI-531
Project: Apache Hudi (incubating)
Issue Type:
vinothchandar commented on issue #1157: [HUDI-332]Add operation type
(insert/upsert/bulkinsert/delete) to HoodieCommitMetadata
URL: https://github.com/apache/incubator-hudi/pull/1157#issuecomment-574027065
@hddong thanks! avro works based on field positions, so reordering them was
my
vinothchandar commented on issue #1208: [HUDI-304] Bring back spotless plugin
URL: https://github.com/apache/incubator-hudi/pull/1208#issuecomment-574024869
>ocument that developers could use checkstyle.xml file in style folder in
checkstyle plugin and things will go well
I was
bhasudha commented on issue #1225: [MINOR] Adding util methods to assist in
adding deletion support to Quick Start
URL: https://github.com/apache/incubator-hudi/pull/1225#issuecomment-574024580
@nsivabalan will merge this once you are able to verify this method with
quickstart steps.
[
https://issues.apache.org/jira/browse/HUDI-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang updated HUDI-503:
--
Status: Open (was: New)
> Add hudi test suite documentation into the README file of the test suite
> module
>
[
https://issues.apache.org/jira/browse/HUDI-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang updated HUDI-503:
--
Status: In Progress (was: Open)
> Add hudi test suite documentation into the README file of the test suite
>
vinothchandar commented on issue #1223: [HUDI-530] Fix conversion of Spark
struct type to Avro schema
URL: https://github.com/apache/incubator-hudi/pull/1223#issuecomment-574015931
@umehrot2 is this by any way related to the quickstart breakage that
@nsivabalan reported?
vinothchandar commented on issue #1225: Adding util methods to assist in adding
deletion support to Quick Start
URL: https://github.com/apache/incubator-hudi/pull/1225#issuecomment-574005188
@nsivabalan can you add a `[MINOR]` prefix to your commit and PR?
OpenOpened closed pull request #1200: [HUDI-514] A schema provider to get
metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200
This is an automated message from the Apache Git Service.
To respond to
OpenOpened opened a new pull request #1200: [HUDI-514] A schema provider to get
metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200
## What is the purpose of the pull request
In our production environment, we usually need to synchronize data from
mysql,
nsivabalan opened a new pull request #1225: Adding util methods to assist in
adding deletion support to Quick Start
URL: https://github.com/apache/incubator-hudi/pull/1225
Adding util methods to assist in adding deletion support to Quick Start
## Verify this pull request
yanghua commented on issue #1100: [HUDI-289] Implement a test suite to support
long running test for Hudi writing and querying end-end
URL: https://github.com/apache/incubator-hudi/pull/1100#issuecomment-573978162
@n3nash This PR has conflicts, I have rebased.
[
https://issues.apache.org/jira/browse/HUDI-523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hong dongdong updated HUDI-523:
---
Description: May be need spark3 (was: As spark upgrade to 2.4, we can
upgrade to datasource api v2
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a change to branch hudi_test_suite_refactor
in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git.
omit 09c34a0 [HUDI-442] Fix
TestComplexKeyGenerator#testSingleValueKeyGenerator and
[
https://issues.apache.org/jira/browse/HUDI-523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hong dongdong reassigned HUDI-523:
--
Assignee: (was: hong dongdong)
> Upgrade Hudi to Spark DataSource V2
>
wangxianghu opened a new pull request #1224: [HUDI-397] Normalize log print
statement
URL: https://github.com/apache/incubator-hudi/pull/1224
## What is the purpose of the pull request
*Normalize log print statement*
*Redo hudi-test-suite log statements using SLF4J*
##
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git.
from c1f8aca [HUDI-526] fix the HoodieAppendHandle
add fd8f1c7 [MINOR] Reuse random object (#1222)
No new
vinothchandar merged pull request #1222: [MINOR] Reuse random object
URL: https://github.com/apache/incubator-hudi/pull/1222
This is an automated message from the Apache Git Service.
To respond to the message, please log on
wangxianghu commented on issue #1220: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1220#issuecomment-573958871
@hmatu Roll back, It closed automatically
This is an automated
hmatu commented on issue #1220: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1220#issuecomment-573957902
Close again?
This is an automated message from the Apache Git Service.
To
wangxianghu closed pull request #1220: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1220
This is an automated message from the Apache Git Service.
To respond to the message,
wangxianghu commented on issue #1220: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1220#issuecomment-573955655
Hi @n3nash, This PR covered all the logs in test-suite module. Besides, I
found the wrong email was used. I will fix it.
wangxianghu commented on issue #1220: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1220#issuecomment-573953742
Hi @hmatu, Thanks for your advice, I will pay attention next time.
umehrot2 commented on issue #1223: [HUDI-530] Fix conversion of Spark struct
type to Avro schema
URL: https://github.com/apache/incubator-hudi/pull/1223#issuecomment-573951118
@vinothchandar @bvaradar The migration to `spark-avro` has introduced this
issue which was earlier reported for
[
https://issues.apache.org/jira/browse/HUDI-530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-530:
Labels: pull-request-available (was: )
> Datasource Writer throws error on resolving struct fields
umehrot2 opened a new pull request #1223: [HUDI-530] Fix conversion of Spark
struct type to Avro schema
URL: https://github.com/apache/incubator-hudi/pull/1223
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review
lamber-ken opened a new pull request #1222: [MINOR] Reuse random object
URL: https://github.com/apache/incubator-hudi/pull/1222
## What is the purpose of the pull request
Reuse random object.
## Brief change log
- *Reuse random object.*
## Verify this pull
Udit Mehrotra created HUDI-530:
--
Summary: Datasource Writer throws error on resolving struct fields
Key: HUDI-530
URL: https://issues.apache.org/jira/browse/HUDI-530
Project: Apache Hudi (incubating)
[
https://issues.apache.org/jira/browse/HUDI-530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Udit Mehrotra reassigned HUDI-530:
--
Assignee: Udit Mehrotra
> Datasource Writer throws error on resolving struct fields
>
bvaradar commented on a change in pull request #1194: [HUDI-326] Add support to
delete records with only record_key
URL: https://github.com/apache/incubator-hudi/pull/1194#discussion_r366091392
##
File path:
bvaradar commented on a change in pull request #1194: [HUDI-326] Add support to
delete records with only record_key
URL: https://github.com/apache/incubator-hudi/pull/1194#discussion_r366088953
##
File path:
hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.java
bvaradar commented on issue #1109: [HUDI-238] - Migrating to Scala 2.12
URL: https://github.com/apache/incubator-hudi/pull/1109#issuecomment-573923932
> Sure. I will send another PR. Currently our work only supports 2.12, but I
can try to see if it is possible to support both 2.11 and
bvaradar commented on issue #1109: [HUDI-238] - Migrating to Scala 2.12
URL: https://github.com/apache/incubator-hudi/pull/1109#issuecomment-573920277
@ezhux : just see that you have pushed some changes. Let us know when you
want us to review the code.
Prashant Wason created HUDI-529:
---
Summary: Enable cobertura coverage reporting
Key: HUDI-529
URL: https://issues.apache.org/jira/browse/HUDI-529
Project: Apache Hudi (incubating)
Issue Type:
Javier Vega created HUDI-528:
Summary: Incremental Pull fails when latest commit is empty
Key: HUDI-528
URL: https://issues.apache.org/jira/browse/HUDI-528
Project: Apache Hudi (incubating)
garyli1019 commented on issue #1128: [HUDI-453] Fix throw failed to archive
commits error when writing data to MOR/COW table
URL: https://github.com/apache/incubator-hudi/pull/1128#issuecomment-573880940
Thanks for the discussion. I have been running into the same issue and
manually
gfn9cho removed a comment on issue #894: Getting java.lang.NoSuchMethodError
while doing Hive sync
URL: https://github.com/apache/incubator-hudi/issues/894#issuecomment-573861890
Hi, I am using EMR 5.28 with built in hudi support.
I was able to use hudi through spark-shell. However,
ezhux commented on a change in pull request #1109: [HUDI-238] - Migrating to
Scala 2.12
URL: https://github.com/apache/incubator-hudi/pull/1109#discussion_r366029872
##
File path: hudi-utilities/pom.xml
##
@@ -28,14 +28,52 @@
${project.parent.basedir}
+
TadasSugintasYields commented on a change in pull request #1109: [HUDI-238] -
Migrating to Scala 2.12
URL: https://github.com/apache/incubator-hudi/pull/1109#discussion_r366029309
##
File path: hudi-utilities/pom.xml
##
@@ -28,14 +28,52 @@
gfn9cho commented on issue #894: Getting java.lang.NoSuchMethodError while
doing Hive sync
URL: https://github.com/apache/incubator-hudi/issues/894#issuecomment-573861890
Hi, I am using EMR 5.28 with built in hudi support.
I was able to use hudi through spark-shell. However, when
[
https://issues.apache.org/jira/browse/HUDI-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014641#comment-17014641
]
Prashant Wason commented on HUDI-527:
-
[WARNING] The POM for org.jamon:jamon-runtime:jar:2.3.1 is
[
https://issues.apache.org/jira/browse/HUDI-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-527:
Labels: pull-request-available (was: )
> Fix warning in project compilation
>
prashantwason opened a new pull request #1221: [HUDI-527]
scalastyle-maven-plugin moved to pluginManagement as it ionly used in
hoodie-spark and hoodie-cli modules
URL: https://github.com/apache/incubator-hudi/pull/1221
## What is the purpose of the pull request
This fixes
zhedoubushishi commented on issue #1109: [HUDI-238] - Migrating to Scala 2.12
URL: https://github.com/apache/incubator-hudi/pull/1109#issuecomment-573847670
> @zhedoubushishi : As you had mentioned that AWS EMR has internally made it
possible to package hudi jars using scala 2.12, can you
[
https://issues.apache.org/jira/browse/HUDI-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014616#comment-17014616
]
Prashant Wason commented on HUDI-527:
-
scalastyle-maven-plugin emits missing file warnings if there are
[
https://issues.apache.org/jira/browse/HUDI-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason updated HUDI-527:
Status: In Progress (was: Open)
> Fix warning in project compilation
>
[
https://issues.apache.org/jira/browse/HUDI-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason updated HUDI-527:
Status: Open (was: New)
> Fix warning in project compilation
> --
>
Prashant Wason created HUDI-527:
---
Summary: Fix warning in project compilation
Key: HUDI-527
URL: https://issues.apache.org/jira/browse/HUDI-527
Project: Apache Hudi (incubating)
Issue Type:
n3nash commented on issue #1220: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1220#issuecomment-573811899
@wangxianghu Thanks for the PR, does this exhaustively take care of all the
logs in the test-suite module ?
This is an automated email from the ASF dual-hosted git repository.
nagarwal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git
The following commit(s) were added to refs/heads/master by this push:
new c1f8aca [HUDI-526] fix the
n3nash merged pull request #1218: [HUDI-526] add parition meta file in
HoodieAppendHandle
URL: https://github.com/apache/incubator-hudi/pull/1218
This is an automated message from the Apache Git Service.
To respond to the
n3nash commented on issue #1216: [HUDI-525] lack of insert info in delta_commit
inflight
URL: https://github.com/apache/incubator-hudi/pull/1216#issuecomment-573809598
@liujianhuiouc What functionality are we going to enhance by adding this
information to the inflight workload profile
xushiyan commented on issue #1187: [HUDI-499] Allow update partition path with
GLOBAL_BLOOM
URL: https://github.com/apache/incubator-hudi/pull/1187#issuecomment-573781552
@nsivabalan Thank you for the thorough review! I'll try to address these in
the next few days.
hmatu commented on issue #1220: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1220#issuecomment-573730094
Please use `git config --list` to check whether your email is right or not.
hmatu commented on issue #1220: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1220#issuecomment-573729421
If you modify some changes, you can continue to commit to same branch, no
need to create a new pr, like
wangxianghu opened a new pull request #1220: [HUDI-397] Normalize log print
statement
URL: https://github.com/apache/incubator-hudi/pull/1220
## What is the purpose of the pull request
*Redo hudi-test-suite log statements using SLF4J*
*Normalize log print statement*
##
wangxianghu closed pull request #1219: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1219
This is an automated message from the Apache Git Service.
To respond to the message,
wangxianghu opened a new pull request #1219: [HUDI-397] Normalize log print
statement
URL: https://github.com/apache/incubator-hudi/pull/1219
## What is the purpose of the pull request
*Normalize log print statement*
*Redo hudi-test-suite log statements using SLF4J*
##
wangxianghu closed pull request #1217: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1217
This is an automated message from the Apache Git Service.
To respond to the message,
[
https://issues.apache.org/jira/browse/HUDI-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liujianhui closed HUDI-517.
---
Resolution: Fixed
> compact error when hoodie.compact.inline is true
>
[
https://issues.apache.org/jira/browse/HUDI-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liujianhui resolved HUDI-518.
-
Resolution: Fixed
> compact error when hoodie.compact.inline is true
>
[
https://issues.apache.org/jira/browse/HUDI-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liujianhui updated HUDI-518:
Status: Open (was: New)
> compact error when hoodie.compact.inline is true
>
wangxianghu opened a new pull request #1217: [HUDI-397] Normalize log print
statement
URL: https://github.com/apache/incubator-hudi/pull/1217
## What is the purpose of the pull request
*Normalize log print statement*
*Redo hudi-test-suite log statements using SLF4J*
##
[
https://issues.apache.org/jira/browse/HUDI-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liujianhui updated HUDI-517:
Status: Open (was: New)
> compact error when hoodie.compact.inline is true
>
[
https://issues.apache.org/jira/browse/HUDI-526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-526:
Labels: pull-request-available (was: )
> inline compact not work
> ---
>
>
wangxianghu closed pull request #1217: [HUDI-397] Normalize log print statement
URL: https://github.com/apache/incubator-hudi/pull/1217
This is an automated message from the Apache Git Service.
To respond to the message,
liujianhuiouc opened a new pull request #1218: [HUDI-526] add parition meta
file in HoodieAppendHandle
URL: https://github.com/apache/incubator-hudi/pull/1218
compact will retrieve the partition path which has partition meta file at
that path
add partition meta file in the
[
https://issues.apache.org/jira/browse/HUDI-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-397:
Labels: pull-request-available (was: )
> Normalize log print statement
>
wangxianghu opened a new pull request #1217: [HUDI-397] Normalize log print
statement
URL: https://github.com/apache/incubator-hudi/pull/1217
## What is the purpose of the pull request
*Normalize log print statement*
*Redo udi-test-suite log statements using SLF4J*
##
[
https://issues.apache.org/jira/browse/HUDI-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangxianghu updated HUDI-397:
-
Status: In Progress (was: Open)
> Normalize log print statement
> -
>
>
[
https://issues.apache.org/jira/browse/HUDI-397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014226#comment-17014226
]
wangxianghu commented on HUDI-397:
--
Hi [~jotarada], 3 days has passed since you asked for this issue, and
liujianhui created HUDI-526:
---
Summary: inline compact not work
Key: HUDI-526
URL: https://issues.apache.org/jira/browse/HUDI-526
Project: Apache Hudi (incubating)
Issue Type: Bug
leesf commented on issue #1216: [HUDI-525]
URL: https://github.com/apache/incubator-hudi/pull/1216#issuecomment-573590408
Thanks for opening the contribution @liujianhuiouc ! would you please change
the title and follow the guide
OpenOpened closed pull request #1200: [HUDI-514] A schema provider to get
metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200
This is an automated message from the Apache Git Service.
To respond to
OpenOpened opened a new pull request #1200: [HUDI-514] A schema provider to get
metadata through Jdbc
URL: https://github.com/apache/incubator-hudi/pull/1200
## What is the purpose of the pull request
In our production environment, we usually need to synchronize data from
mysql,
[
https://issues.apache.org/jira/browse/HUDI-525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-525:
Labels: pull-request-available (was: )
> inserts info miss in delta_commit_inflight meta file
>
liujianhuiouc opened a new pull request #1216: [HUDI-525]
URL: https://github.com/apache/incubator-hudi/pull/1216
add insert info with insert records num in the HoodieCommitMeta.
because of the file id is unknown, set it as empty string
testing result:
```
zhedoubushishi commented on a change in pull request #1109: [HUDI-238] -
Migrating to Scala 2.12
URL: https://github.com/apache/incubator-hudi/pull/1109#discussion_r365680930
##
File path: hudi-utilities/pom.xml
##
@@ -28,14 +28,52 @@
${project.parent.basedir}
[
https://issues.apache.org/jira/browse/HUDI-295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014104#comment-17014104
]
Pratyaksh Sharma commented on HUDI-295:
---
[~vbalaji] Few of my commits are not getting counted in my
88 matches
Mail list logo
