Re: Reg: Hudi Jira Ticket Conventions

2019-08-28 Thread Vinoth Chandar
+1 can we add this to contributing/community pages. As well On Wed, Aug 28, 2019 at 2:33 PM vbal...@apache.org wrote: > To all contributors of Hudi: > Dear folks, > When filing or updating a JIRA for Apache Hudi, kindly make sure the issue > type and versions (when resolving the ticket) are set

Reg: Hudi Jira Ticket Conventions

2019-08-28 Thread vbal...@apache.org
To all contributors of Hudi: Dear folks, When filing or updating a JIRA for Apache Hudi, kindly make sure the issue type and versions (when resolving the ticket) are set correctly. Also, the summary needs to be descriptive enough to catch the essence of the problem/features. This greatly helps

Re: Upsert after Delete

2019-08-28 Thread vbal...@apache.org
Hi Kabeer, I have requested some information in the github ticket.  Balaji.VOn Wednesday, August 28, 2019, 10:46:04 AM PDT, Kabeer Ahmed wrote: Thanks for the quick response Vinoth. That is what I would have thought that there is nothing complex or different in upsert after a delete.

[For Mentors] Readiness for IP Clearance

2019-08-28 Thread vbal...@apache.org
Dear Mentors, We are able to setup nightly snapshot builds. At this moment, we have the following steps done (Master Jira: https://jira.apache.org/jira/browse/HUDI-121)  - Software Grant : Software grant from Uber to Apache has been completed - Contributor CLA : Done - License

Re: Upsert after Delete

2019-08-28 Thread Kabeer Ahmed
Thanks for the quick response Vinoth. That is what I would have thought that there is nothing complex or different in upsert after a delete. Yes, I can reproduce the issue with simple example that I have written in the email. I have dug into the issue in detail and it seems it is a bug. I have

Re: [Hudi Improvement]: Introduce secondary source-ordering-field for breaking ties while writing

2019-08-28 Thread vbal...@apache.org
Sure Pratyaksh, Whatever field works for your use-case is good enough. You do have the flexibility to generate a derived field or use one of the source fields  Balaji.VOn Wednesday, August 28, 2019, 06:48:44 AM PDT, Pratyaksh Sharma wrote: Hi Balaji, Sure I can do that. However

Re: [Hudi Improvement]: Introduce secondary source-ordering-field for breaking ties while writing

2019-08-28 Thread Pratyaksh Sharma
Hi Balaji, Sure I can do that. However after a considerable amount of time, the bin-log position will get exhausted. To handle this, we can have secondary ordering field as the ingestion_timestamp (the time when I am pushing the event to Kafka to be consumed by DeltaStreamer) which will work