Re: [DISCUSS] Release 0.6.0 timelines

2020-08-19 Thread Allen Underwood
nteer > > > > to > > > > > >> RM > > > > > >> > > this > > > > > >> > > > >> > > release. > > > > > >> > > > >> > > > > > > > > > > > > >> > > > >> > > > > > > > Understand 0.6.0 release is delayed than > > what > &

Re: Handling delta

2020-07-16 Thread Allen Underwood
Step 3: I was expecting only 10 records will be written but it writes whole > 50 records is this a normal behaviour? Which means do I need to determine > the delta myself and write them alone? > > Am I missing something? > -- *Allen Underwood* Principal Software Engineer B

Re: Extracting a partition field from something like a timestamp using Deltastreamer / Configurations?

2020-05-07 Thread Allen Underwood
e can add this to `writing data` page. > > > > > On Wed, May 6, 2020 at 12:20 PM Allen Underwood > wrote: > > > Hello Vinoth, > > > > As promised, here's a PR into 0.5.2 - I think it might be worth bringing > > that into master / 0.5.3 as well.

Re: Extracting a partition field from something like a timestamp using Deltastreamer / Configurations?

2020-05-06 Thread Allen Underwood
e only one that needed it! :-) Allen On Tue, May 5, 2020 at 11:22 AM Vinoth Chandar wrote: > Great! > > On Mon, May 4, 2020 at 5:43 PM Allen Underwood > wrote: > > > Hi Vinoth, > > > > Yes I was going to set some things up in the morning. I’ll let you know

Re: Extracting a partition field from something like a timestamp using Deltastreamer / Configurations?

2020-05-04 Thread Allen Underwood
On Mon, May 4, 2020 at 3:08 PM Allen Underwood > wrote: > > > Hi Vinoth - that was extremely helpful...I almost had it working, > HOWEVER, > > it appears I have dates that some have the ms on the end and others > > don'tso if I pick adding a time format with them, th

Re: Extracting a partition field from something like a timestamp using Deltastreamer / Configurations?

2020-05-04 Thread Allen Underwood
se > your timestamp using Java SimpleDateFormat, you can specify the datetime > field and a pattern to parse it into.. > > Happy to make this work for you. > > community, any volunteers to faq/document this? :) > > > On Mon, May 4, 2020 at 9:11 AM Allen Underwood >

Extracting a partition field from something like a timestamp using Deltastreamer / Configurations?

2020-05-04 Thread Allen Underwood
configuration of the spark job / hudi configuration? hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.java <https://github.com/apache/incubator-hudi/blob/master/hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.java> -- *Allen Underwood* smime.p7s Descr