n3nash commented on issue #624: (WIP) Test suite documentation
URL: https://github.com/apache/incubator-hudi/pull/624#issuecomment-478822267
Closing this PR since it was opened against master branch
This is an automated
n3nash closed pull request #624: (WIP) Test suite documentation
URL: https://github.com/apache/incubator-hudi/pull/624
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
arunlakshman commented on issue #622: [HUDI-61] Remove payload class
dependency in SpillableMap
URL: https://github.com/apache/incubator-hudi/pull/622#issuecomment-478884206
I just saw this on JIRA board and started working on this
- https://issues.apache.org/jira/browse/HUDI-61
-
SemanticBeeng edited a comment on issue #128: Evaluate feasibility of
scheduling hoodie pipelines w/o need for external workflow managers
URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-478900724
Do we want to consider what this means more exactly from perspective of
n3nash opened a new pull request #626: Adding documentation for hudi test suite
URL: https://github.com/apache/incubator-hudi/pull/626
This is an automated message from the Apache Git Service.
To respond to the message,
SemanticBeeng commented on issue #128: Evaluate feasibility of scheduling
hoodie pipelines w/o need for external workflow managers
URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-478900724
Do we want to consider what this means more exactly from perspective of
vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for
writing through spark
URL: https://github.com/apache/incubator-hudi/pull/620#issuecomment-478077857
@leilinen did you have the same problem with this patch applied as well?
Some background. FileSystem
vinothchandar commented on issue #128: Evaluate feasibility of scheduling
hoodie pipelines w/o need for external workflow managers
URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-479134137
@SemanticBeeng this is orthogonal to connection management I believe.. This
lyogev commented on issue #498: Is there any record delete examples?
URL: https://github.com/apache/incubator-hudi/issues/498#issuecomment-479127330
Does anyone have an example of how to hard delete in spark SQL?
I can't seem to find a way to do this.
vinothchandar commented on issue #128: Evaluate feasibility of scheduling
hoodie pipelines w/o need for external workflow managers
URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-479134748
Once we approach beam, we can consider ProcessContext. for now, I think
lyogev commented on issue #498: Is there any record delete examples?
URL: https://github.com/apache/incubator-hudi/issues/498#issuecomment-479134812
Sorry figured it out.
Created a class:
```
class EmptyHoodieRecordPayload extends
HoodieRecordPayload[EmptyHoodieRecordPayload] {
n3nash commented on issue #621: Follow up HUDI-27 : Call super.close() in
HoodieWraperFileSystem::close()
URL: https://github.com/apache/incubator-hudi/pull/621#issuecomment-479254921
Okay, fair enough.
This is an automated
n3nash commented on a change in pull request #599: Fixing small file handling,
inline compaction defaults
URL: https://github.com/apache/incubator-hudi/pull/599#discussion_r271531565
##
File path: hoodie-spark/src/test/scala/DataSourceTest.scala
##
@@ -131,6 +131,7 @@
thesuperzapper opened a new issue #628: Replace Databricks spark-avro with
native spark-avro
URL: https://github.com/apache/incubator-hudi/issues/628
I believe that we should bump to Spark 2.4 minimum for the native spark-avro
package. Its just going to become problematic the longer we
vinothchandar commented on issue #599: Fixing small file handling, inline
compaction defaults
URL: https://github.com/apache/incubator-hudi/pull/599#issuecomment-479337017
I think repush got rid of it.. fished it out of email notification
> I see, put a sleep(1s) for now ?
vinothchandar commented on issue #628: Replace Databricks spark-avro with
native spark-avro
URL: https://github.com/apache/incubator-hudi/issues/628#issuecomment-479336469
Agree.. IIRC someone was working on spark 2.4 upgrade. Mind joining the
mailing list and kick off a thread there?
vinothchandar commented on issue #599: Fixing small file handling, inline
compaction defaults
URL: https://github.com/apache/incubator-hudi/pull/599#issuecomment-479336163
I think I resolved both your comments? @n3nash did I miss something? I added
the comment to denoet the actual MB
vinothchandar merged pull request #621: Follow up HUDI-27 : Call super.close()
in HoodieWraperFileSystem::close()
URL: https://github.com/apache/incubator-hudi/pull/621
This is an automated message from the Apache Git
BigdataGit commented on issue #607: The difference between `_hoodie_record_key`
and `_row_key`？
URL: https://github.com/apache/incubator-hudi/issues/607#issuecomment-475094145
> yes. to add, all merges/compaction etc happens based on _hoodie_record_key
once its extracted out from the
ambition119 commented on issue #608: [HUDI-63] Removed unused BucketedIndex code
URL: https://github.com/apache/incubator-hudi/pull/608#issuecomment-476446712
> @ambition119 squashing is different.. I think those are for rebasing
(original error you reported)..
>
>
n3nash commented on issue #612: Fixing source schema and writer schema
distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#issuecomment-476390837
@bvaradar Fixed the code and added a small test case as well, please take a
look.
@vinothchandar please review
vinothchandar closed pull request #535: Build with Error Prone
URL: https://github.com/apache/incubator-hudi/pull/535
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
vinothchandar commented on issue #535: Build with Error Prone
URL: https://github.com/apache/incubator-hudi/pull/535#issuecomment-476417963
closing due to inactivity
This is an automated message from the Apache Git Service.
n3nash edited a comment on issue #612: Fixing source schema and writer schema
distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#issuecomment-476390837
@bvaradar Updated and added a small test case as well, please take a look.
@vinothchandar please review
vinothchandar commented on issue #608: [HUDI-63] Removed unused BucketedIndex
code
URL: https://github.com/apache/incubator-hudi/pull/608#issuecomment-476408168
@ambition119 squashing is different.. I think those are for rebasing
(original error you reported)..
vinothchandar commented on a change in pull request #612: Fixing source schema
and writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r268887126
##
File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java
vinothchandar commented on a change in pull request #612: Fixing source schema
and writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r26636
##
File path:
vinothchandar commented on a change in pull request #612: Fixing source schema
and writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r268886753
##
File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java
vinothchandar commented on a change in pull request #612: Fixing source schema
and writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r26490
##
File path:
vinothchandar commented on a change in pull request #612: Fixing source schema
and writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r268887669
##
File path:
leilinen commented on issue #615: how change HoodieDeltaStreamer with
Real-time calculation
URL: https://github.com/apache/incubator-hudi/issues/615#issuecomment-477158609
I noticed that the dstream consumed from kafka topic was empty, for
dstream.foreachRDD resulted nothing. But the
leilinen opened a new issue #615: how change HoodieDeltaStreamer with
Real-time calculation
URL: https://github.com/apache/incubator-hudi/issues/615
Hi, In my project, I want to consume data from kafka topic and upsert data
with hoodie. In hoodie, the HoodieDeltaStreamer is
cdmikechen commented on issue #547: The question about Hudi transform timestamp
and decimal's type
URL: https://github.com/apache/incubator-hudi/issues/547#issuecomment-476621181
@vinothchandar
Sorry for delay. I've changed my job in February, and my new job works for
an automobile
vinothchandar commented on issue #615: how change HoodieDeltaStreamer with
Real-time calculation
URL: https://github.com/apache/incubator-hudi/issues/615#issuecomment-477248903
lets continue on the mailing list.. !
This
vinothchandar commented on issue #615: how change HoodieDeltaStreamer with
Real-time calculation
URL: https://github.com/apache/incubator-hudi/issues/615#issuecomment-477248778
@leilinen please post on the mailing list for faster response.
https://hudi.apache.org/community.html
vinothchandar commented on a change in pull request #617: Fixes HUDI-9 : Check
precondition minInstantsToKeep > cleanerCommitsR…
URL: https://github.com/apache/incubator-hudi/pull/617#discussion_r269683323
##
File path:
bvaradar commented on a change in pull request #616: Fixes HUDI-38: Reduce
memory overhead of WriteStatus
URL: https://github.com/apache/incubator-hudi/pull/616#discussion_r269691293
##
File path: hoodie-client/src/main/java/com/uber/hoodie/WriteStatus.java
##
@@ -48,6
vinothchandar commented on issue #616: Fixes HUDI-38: Reduce memory overhead of
WriteStatus
URL: https://github.com/apache/incubator-hudi/pull/616#issuecomment-477284353
yes @ovj @n3nash the custom WriteStatus need to change post release. its
okay for you to ignore the args though.. Once
vinothchandar merged pull request #617: Fixes HUDI-9 : Check precondition
minInstantsToKeep > cleanerCommitsR…
URL: https://github.com/apache/incubator-hudi/pull/617
This is an automated message from the Apache Git Service.
n3nash commented on a change in pull request #612: Fixing source schema and
writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269236754
##
File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java
bvaradar commented on issue #581: ClassNotFoundException:HoodieInputFormat
URL: https://github.com/apache/incubator-hudi/issues/581#issuecomment-476779092
@Achuth17 : This was done on purpose. There are lot of different
hadoop/hive environments out there. With Hudi having to interoperate
bvaradar commented on issue #581: ClassNotFoundException:HoodieInputFormat
URL: https://github.com/apache/incubator-hudi/issues/581#issuecomment-476779549
@daikon12 : Is run_hive_sync tool working fine now ?
This is an
n3nash commented on a change in pull request #612: Fixing source schema and
writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269237813
##
File path:
n3nash commented on a change in pull request #612: Fixing source schema and
writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269237737
##
File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java
n3nash commented on a change in pull request #612: Fixing source schema and
writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269236754
##
File path: hoodie-client/src/main/java/com/uber/hoodie/io/HoodieIOHandle.java
n3nash commented on a change in pull request #612: Fixing source schema and
writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269236502
##
File path:
hoodie-client/src/main/java/com/uber/hoodie/io/HoodieCreateHandle.java
n3nash commented on issue #612: Fixing source schema and writer schema
distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#issuecomment-476771521
@vinothchandar Addressed and replied to comments.
n3nash commented on a change in pull request #612: Fixing source schema and
writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r269238293
##
File path:
vinothchandar commented on issue #547: The question about Hudi transform
timestamp and decimal's type
URL: https://github.com/apache/incubator-hudi/issues/547#issuecomment-476752594
Congrats on the new gig! :)
This is an
vinothchandar commented on issue #547: The question about Hudi transform
timestamp and decimal's type
URL: https://github.com/apache/incubator-hudi/issues/547#issuecomment-476752493
@cdmikechen no worries.. it would help if you can file a JIRA on hive/hadoop
version gaps preventing you
vinothchandar merged pull request #608: [HUDI-63] Removed unused BucketedIndex
code
URL: https://github.com/apache/incubator-hudi/pull/608
This is an automated message from the Apache Git Service.
To respond to the message,
vinothchandar commented on issue #608: [HUDI-63] Removed unused BucketedIndex
code
URL: https://github.com/apache/incubator-hudi/pull/608#issuecomment-476753401
awesome .. @ambition119 . congrats on your first PR :)
This is
Zhujun-Vungle commented on issue #143: Tracking ticket for folks to be added to
slack group
URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-475150950
Hi would you add me to slack please~
jun@vungle.com
vinothchandar closed issue #580: Change serializer for DiskBasedMap to handle
non serializable java objects
URL: https://github.com/apache/incubator-hudi/issues/580
This is an automated message from the Apache Git Service.
vinothchandar commented on issue #596: Unable to find previous checkpoint.
Please double check if this table was indeed built via delta streamer
URL: https://github.com/apache/incubator-hudi/issues/596#issuecomment-475820514
Closing this due to inactivity
vinothchandar closed issue #596: Unable to find previous checkpoint. Please
double check if this table was indeed built via delta streamer
URL: https://github.com/apache/incubator-hudi/issues/596
This is an automated
vinothchandar opened a new pull request #611: Refreshing site content
URL: https://github.com/apache/incubator-hudi/pull/611
This is an automated message from the Apache Git Service.
To respond to the message, please log on
n3nash opened a new pull request #612: (WIP) Fixing source schema and writer
schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612
This is an automated message from the Apache Git Service.
To
vinothchandar commented on issue #493: Hive partition column defaults to String
URL: https://github.com/apache/incubator-hudi/issues/493#issuecomment-475820882
@AndrewKL non partitioned tables must be supported now.. (@bvaradar fyi)..
close this?
vinothchandar closed issue #540: `COMBINE_BEFORE_INSERT_PROP` and
`COMBINE_BEFORE_UPSERT_PROP` default values
URL: https://github.com/apache/incubator-hudi/issues/540
This is an automated message from the Apache Git
vinothchandar commented on issue #547: The question about Hudi transform
timestamp and decimal's type
URL: https://github.com/apache/incubator-hudi/issues/547#issuecomment-475820698
@cdmikechen there is a thread on this on the mailing list. Are you still
working/hitting this? If so, could
vinothchandar commented on issue #540: `COMBINE_BEFORE_INSERT_PROP` and
`COMBINE_BEFORE_UPSERT_PROP` default values
URL: https://github.com/apache/incubator-hudi/issues/540#issuecomment-475820737
Closing this due to inactivity
bvaradar commented on a change in pull request #612: (WIP) Fixing source schema
and writer schema distinction in payloads
URL: https://github.com/apache/incubator-hudi/pull/612#discussion_r268368530
##
File path:
Achuth17 commented on issue #581: ClassNotFoundException:HoodieInputFormat
URL: https://github.com/apache/incubator-hudi/issues/581#issuecomment-476058892
I faced the antlr recognition exception as well, Fixed it by adding a few
jars (like the antlr jar) manually. Just thinking out loud,
vinothchandar commented on issue #613: Too many S3 Connections being opened
while df.write
URL: https://github.com/apache/incubator-hudi/issues/613#issuecomment-476034123
https://issues.apache.org/jira/browse/HUDI-80 tracks a potential fix for
this..
smdahmed opened a new issue #613: Too many S3 Connections being opened while
df.write
URL: https://github.com/apache/incubator-hudi/issues/613
1. Table clean is called for every dataframe write
ambition119 commented on issue #608: [HUDI-63] Removed unused BucketedIndex code
URL: https://github.com/apache/incubator-hudi/pull/608#issuecomment-475876976
> awesome.. can you please squash the commits again? :)
I use these command :
```shell
$ git remote add apache
vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for
writing through spark
URL: https://github.com/apache/incubator-hudi/pull/620#issuecomment-479576736
Also lets take this discussion to the mailing list, so others also benefit
:)
vinothchandar commented on issue #620: Fix HUDI-27 : Support num_cores > 1 for
writing through spark
URL: https://github.com/apache/incubator-hudi/pull/620#issuecomment-479576565
@leilinen so may be with increased parallelism, hitting the race condition
becomes more probable.
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271905565
##
File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieBaseClient.java
##
@@
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271905880
##
File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieBaseClient.java
##
@@
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271912046
##
File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieWriteClient.java
##
@@
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271915076
##
File path:
hoodie-client/src/main/java/com/uber/hoodie/io/HoodieAppendHandle.java
##
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271922431
##
File path:
hoodie-client/src/test/java/com/uber/hoodie/io/strategy/TestHoodieDataFile.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271927078
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/table/TableFileSystemView.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271931617
##
File path:
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271905131
##
File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieBaseClient.java
##
@@
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271914607
##
File path:
hoodie-client/src/main/java/com/uber/hoodie/func/EmbeddedTimelineServer.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271925460
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieTimeline.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271926261
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieView.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271925979
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieView.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271918483
##
File path:
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271928501
##
File path:
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271906279
##
File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieBaseClient.java
##
@@
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271911256
##
File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieWriteClient.java
##
@@
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271913114
##
File path:
hoodie-client/src/main/java/com/uber/hoodie/func/EmbeddedTimelineServer.java
n3nash opened a new pull request #629: 1. Minor changes to fix compaction 2.
Adding 2 compaction policies 3. Adding a Hbase index property
URL: https://github.com/apache/incubator-hudi/pull/629
This is an automated message
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271917051
##
File path:
hoodie-client/src/main/java/com/uber/hoodie/io/HoodieCleanHelper.java
##
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271923395
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/model/HoodieDataFile.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271923395
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/model/HoodieDataFile.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271928350
##
File path:
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271948421
##
File path:
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271922608
##
File path:
hoodie-client/src/test/java/com/uber/hoodie/table/TestMergeOnReadTable.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271925460
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieTimeline.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271925335
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/table/HoodieTimeline.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271927580
##
File path:
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271910725
##
File path: hoodie-client/src/main/java/com/uber/hoodie/HoodieWriteClient.java
##
@@
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271913778
##
File path:
hoodie-client/src/main/java/com/uber/hoodie/func/EmbeddedTimelineServer.java
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271921122
##
File path:
n3nash commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r271924427
##
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/model/HoodieLogFile.java
1 - 100 of 25981 matches
Mail list logo
