[GitHub] [incubator-hudi] arunlakshman commented on issue #622: [HUDI-61] Remove payload class dependency in SpillableMap

arunlakshman commented on issue #622: [HUDI-61] Remove payload class dependency in SpillableMap URL: https://github.com/apache/incubator-hudi/pull/622#issuecomment-478884206 I just saw this on JIRA board and started working on this - https://issues.apache.org/jira/browse/HUDI-61 -

[GitHub] [incubator-hudi] SemanticBeeng edited a comment on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

SemanticBeeng edited a comment on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-478900724 Do we want to consider what this means more exactly from perspective of

[GitHub] [incubator-hudi] SemanticBeeng commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

SemanticBeeng commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-478900724 Do we want to consider what this means more exactly from perspective of

[GitHub] [incubator-hudi] vinothchandar commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

vinothchandar commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-479134137 @SemanticBeeng this is orthogonal to connection management I believe.. This

[GitHub] [incubator-hudi] lyogev commented on issue #498: Is there any record delete examples?

lyogev commented on issue #498: Is there any record delete examples? URL: https://github.com/apache/incubator-hudi/issues/498#issuecomment-479127330 Does anyone have an example of how to hard delete in spark SQL? I can't seem to find a way to do this.

[GitHub] [incubator-hudi] vinothchandar commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers

vinothchandar commented on issue #128: Evaluate feasibility of scheduling hoodie pipelines w/o need for external workflow managers URL: https://github.com/apache/incubator-hudi/issues/128#issuecomment-479134748 Once we approach beam, we can consider ProcessContext. for now, I think

[GitHub] [incubator-hudi] lyogev commented on issue #498: Is there any record delete examples?

lyogev commented on issue #498: Is there any record delete examples? URL: https://github.com/apache/incubator-hudi/issues/498#issuecomment-479134812 Sorry figured it out. Created a class: ``` class EmptyHoodieRecordPayload extends HoodieRecordPayload[EmptyHoodieRecordPayload] {

[GitHub] [incubator-hudi] n3nash commented on issue #621: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close()

n3nash commented on issue #621: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close() URL: https://github.com/apache/incubator-hudi/pull/621#issuecomment-479254921 Okay, fair enough. This is an automated

[GitHub] [incubator-hudi] n3nash commented on a change in pull request #599: Fixing small file handling, inline compaction defaults

n3nash commented on a change in pull request #599: Fixing small file handling, inline compaction defaults URL: https://github.com/apache/incubator-hudi/pull/599#discussion_r271531565 ## File path: hoodie-spark/src/test/scala/DataSourceTest.scala ## @@ -131,6 +131,7 @@

[GitHub] [incubator-hudi] thesuperzapper opened a new issue #628: Replace Databricks spark-avro with native spark-avro

thesuperzapper opened a new issue #628: Replace Databricks spark-avro with native spark-avro URL: https://github.com/apache/incubator-hudi/issues/628 I believe that we should bump to Spark 2.4 minimum for the native spark-avro package. Its just going to become problematic the longer we

[GitHub] [incubator-hudi] vinothchandar commented on issue #599: Fixing small file handling, inline compaction defaults

vinothchandar commented on issue #599: Fixing small file handling, inline compaction defaults URL: https://github.com/apache/incubator-hudi/pull/599#issuecomment-479337017 I think repush got rid of it.. fished it out of email notification > I see, put a sleep(1s) for now ?

[GitHub] [incubator-hudi] vinothchandar commented on issue #628: Replace Databricks spark-avro with native spark-avro

vinothchandar commented on issue #628: Replace Databricks spark-avro with native spark-avro URL: https://github.com/apache/incubator-hudi/issues/628#issuecomment-479336469 Agree.. IIRC someone was working on spark 2.4 upgrade. Mind joining the mailing list and kick off a thread there?

[incubator-hudi] branch master updated: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close()

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 51f4908 Follow up HUDI-27 : Call

[GitHub] [incubator-hudi] vinothchandar commented on issue #599: Fixing small file handling, inline compaction defaults

vinothchandar commented on issue #599: Fixing small file handling, inline compaction defaults URL: https://github.com/apache/incubator-hudi/pull/599#issuecomment-479336163 I think I resolved both your comments? @n3nash did I miss something? I added the comment to denoet the actual MB

[GitHub] [incubator-hudi] vinothchandar merged pull request #621: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close()

vinothchandar merged pull request #621: Follow up HUDI-27 : Call super.close() in HoodieWraperFileSystem::close() URL: https://github.com/apache/incubator-hudi/pull/621 This is an automated message from the Apache Git