sure. sg. thanks!
On Tue, 23 May 2023 at 23:22, 孔维 <18701146...@163.com> wrote:
>
> Hi, Sivabalan,
>
>
> Great to hear from you. Then I will create a JIRA ticket to track this feature
>
>
> Best Regards
>
> At 2023-05-24 02:22:36, "Sivabalan" wrote:
> >I could not see the image you have
Hi folks,
I haven't received your reply for a long time. I think you must have
something more important to do. Am I right? : )
At 2023-03-31 21:40:46, "吕虎" wrote:
>Hi Vinoth, I'm glad to receive your letter. Here are some of my thoughts.
>At 2023-03-31 10:17:52, "Vinoth Chandar"
Pulled in another reviewer as well. Left a comment. We can move the
discussion to the PR?
Thanks for the useful contribution!
On Thu, Apr 6, 2023 at 12:34 AM 孔维 <18701146...@163.com> wrote:
> Hi, vinoth,
>
> I created a PR(https://github.com/apache/hudi/pull/8376) for this
> feature, could you
Look forward to this! could really help backfill/rebootstrap scenarios.
On Tue, Apr 4, 2023 at 9:18 AM Vinoth Chandar wrote:
> Thinking out loud.
>
> 1. For insert operations, it should not matter anyway.
> 2. For upsert etc, the preCombine would handle the ordering problems.
>
> Is that what
Thinking out loud.
1. For insert operations, it should not matter anyway.
2. For upsert etc, the preCombine would handle the ordering problems.
Is that what you are saying? I feel we don't want to leak any Kafka
specific logic or force use of special payloads etc. thoughts?
I assigned the jira
left some comments. thanks!
On Fri, 31 Mar 2023 at 00:59, 符其军 <18889897...@163.com> wrote:
> Hi community, we have submitted RFC-65 Partition TTL Management in this
> pr: https://github.com/apache/hudi/pull/8062.Let me know if you
> have any questions or concerns with this proposal.
> At
Hi Vinoth, I'm glad to receive your letter. Here are some of my thoughts.
At 2023-03-31 10:17:52, "Vinoth Chandar" wrote:
>I think we can focus more on validating the hash index + bloom filter vs
>consistent hash index more first. Have you looked at RFC-08, which is a
>kind of hash index as well,
I think we can focus more on validating the hash index + bloom filter vs
consistent hash index more first. Have you looked at RFC-08, which is a
kind of hash index as well, except it stores the key => file group mapping
externally.
On Fri, Mar 24, 2023 at 2:14 AM 吕虎 wrote:
> Hi Vinoth, I am
>but when it is used for data expansion, it still involves the need to
redistribute the data records of some data files, thus affecting the
performance.
but expansion of the consistent hash index is an optional operation right?
Sorry, not still fully understanding the differences here,
>Because
er is like an app, just like the delta
>> > streamer. If we want to build another Flink writer, we can still share
>> the
>> > same flink client right? Does the flink client also have to use the new
>> > feature only available on Flink 1.12?
>> >
>> > Than
> > same flink client right? Does the flink client also have to use the new
> > feature only available on Flink 1.12?
> >
> > Thanks,
> > Gary Li
> > ____
> > From: Danny Chan
> > Sent: Thursday, January 7, 2021 10:19 AM
>
available on Flink 1.12?
>
> Thanks,
> Gary Li
>
> From: Danny Chan
> Sent: Thursday, January 7, 2021 10:19 AM
> To: dev@hudi.apache.org
> Subject: Re: Re: [DISCUSS] New Flink Writer Proposal
>
> Thanks vino yang ~
>
> IMO, we shou
;
> Thanks,
> Gary Li
>
> From: Danny Chan
> Sent: Thursday, January 7, 2021 10:19 AM
> To: dev@hudi.apache.org
> Subject: Re: Re: [DISCUSS] New Flink Writer Proposal
>
> Thanks vino yang ~
>
> IMO, we should not
From: Danny Chan
Sent: Thursday, January 7, 2021 10:19 AM
To: dev@hudi.apache.org
Subject: Re: Re: [DISCUSS] New Flink Writer Proposal
Thanks vino yang ~
IMO, we should not put much put too much energy for current Flink writer,
it is not production-ready in the long run
Thanks vino yang ~
IMO, we should not put much put too much energy for current Flink writer,
it is not production-ready in the long run. There are so many features need
to add/support for the Flink write/read(MOR write, COR read, MOR read, the
new index), we should focus on one version first,
Hi Danny,
As we discussed in the doc, we should agree on if we should be compatible
with the version less than Flink 1.11/1.12.
We all know that there are some bottlenecks in the current plan. You
proposed some improvements, yes it is great, but it radically uses the
newer features provided by
Hi Danny,
You should have cwiki edit permission now.
Any problems let me know.
Best,
Vino
Danny Chan 于2021年1月6日周三 下午12:05写道：
> Sorry ~
>
> Forget to say that my Confluence ID is danny0405.
>
> It would be nice if any of you can help on this.
>
> Best,
> Danny Chan
>
> Danny Chan 于2021年1月6日周三
Sorry ~
Forget to say that my Confluence ID is danny0405.
It would be nice if any of you can help on this.
Best,
Danny Chan
Danny Chan 于2021年1月6日周三 下午12:00写道：
> Hi, can someone give me the CWIKI permission so that i can update the
> design details to that (maybe as a new RFC though ~).
>
>
Hi, can someone give me the CWIKI permission so that i can update the
design details to that (maybe as a new RFC though ~).
wangxianghu 于2021年1月5日周二 下午2:43写道：
> + 1, Thanks Danny!
> I believe this new feature OperatorConrdinator in flink-1.11 will help
> improve the current implementation
>
>
Hi, vc
I also want to do some tests against the release branch.
957029...@qq.com
From: Vinoth Chandar
Date: 2020-08-19 12:51
To: dev
Subject: Re: [DISCUSS] Release 0.6.0 timelines
We still have 1 blocker issue from TimestampKeyGenerator issue with joda
DateTimeFormatter. Sudha (RM) and
Hi Vinoth,
Yes, it's incorrect to draw the conclusion from only one test.
It's just an new idea to improve the merge performance, it's not the best.
e.g when read old record, series of conversion operations (Row to GenericRecord
to HoodieRecord) etc..
> Also let's separate the RDD vs
Hi Lamber-ken,
If you agree reduceByKey() will shuffle data, then it would serialize and
deserialize anyway correct?
I am not denying that this may be a valid approach.. But we need much more
rigorous testing and potentially implement both approaches side-by-side to
compare.. IMO We cannot
Does n't this move the problem to tuning spark simply? the
ExternalSpillableMap itself was not the issue right, the serialization
was. This map is also used on the query side btw, where we need something
like that.
I took a pass at the code. I think we are shuffling data again for the
Thank you!
On Mon, Feb 24, 2020 at 10:47 AM Suneel Marthi wrote:
> https://issues.apache.org/jira/browse/HUDI-580
>
> On Mon, Feb 24, 2020 at 1:42 PM Vinoth Chandar wrote:
>
> > Hi,
> >
> > Have a filed a JIRA for this, tagged with 0.5.2?
> >
> > Thanks
> > Vinoth
> >
> > On Sat, Feb 22, 2020
https://issues.apache.org/jira/browse/HUDI-580
On Mon, Feb 24, 2020 at 1:42 PM Vinoth Chandar wrote:
> Hi,
>
> Have a filed a JIRA for this, tagged with 0.5.2?
>
> Thanks
> Vinoth
>
> On Sat, Feb 22, 2020 at 6:58 AM lamberken wrote:
>
> >
> >
> > Right, will do.
> >
> >
> > Thanks,
> >
If there are no more comments/objections, we could re work the PR based on
the discussion here..
Points made by Udit are also pretty valid..
Thanks for the constructive conversation. :)
On Wed, Feb 19, 2020 at 3:12 PM lamberken wrote:
>
>
> @Vinoth, glad to see your reply.
>
>
> >>
Thanks @Vino :)
Thanks
Lamber-Ken
At 2020-01-24 10:18:28, "vino yang" wrote:
>Hi Lamber,
>
>+1 from my side.
>
>Best,
>Vino
>
>lamberken 于2020年1月24日周五 上午7:11写道：
>
>>
>>
>> Thanks you all. :)
>>
>>
>> Hi @nishith, good catch. I fixed it.
>>
>>
Hi Lamber,
+1 from my side.
Best,
Vino
lamberken 于2020年1月24日周五 上午7:11写道：
>
>
> Thanks you all. :)
>
>
> Hi @nishith, good catch. I fixed it.
>
> https://github.com/apache/incubator-hudi/pull/1276/files?short_path=55fa8a8#diff-55fa8a81e6bf8c8d9d11d293b41511b5
>
>
> Thanks
> Lamber-Ken
>
>
>
>
+1 nice work.
On Thu, Jan 23, 2020 at 12:43 PM nishith agarwal
wrote:
> +1 looks great
>
> Nit : I see that the old diagram has "Raw Ingest Tables" vs the new one
> "Row Ingest Tables". IMO, "Raw Ingest Tables" sounds more logical.
>
> -Nishith
>
> On Thu, Jan 23, 2020 at 10:57 AM Vinoth
+1 looks great
Nit : I see that the old diagram has "Raw Ingest Tables" vs the new one
"Row Ingest Tables". IMO, "Raw Ingest Tables" sounds more logical.
-Nishith
On Thu, Jan 23, 2020 at 10:57 AM Vinoth Chandar wrote:
> +1. on that :)
>
> On Thu, Jan 23, 2020 at 10:22 AM hmatu
The whole site looks better than old currently, big thanks for your work!
Thanks,
Hmatu
-- Original --
From:"Balaji Varadarajan"https://hudi.apache.org
[2] https://lamber-ken.github.io
Thanks
Lamber-Ken
+1. on that :)
On Thu, Jan 23, 2020 at 10:22 AM hmatu <3480388...@qq.com> wrote:
> The whole site looks better than old currently, big thanks for your work!
>
>
> Thanks,
> Hmatu
>
>
>
> -- Original --
> From:"Balaji Varadarajan" Date:Fri, Jan 24, 2020 01:21 AM
>
Looks great. +1
--Original--
From:"Balaji Varadarajan"https://hudi.apache.org
[2] https://lamber-ken.github.io
Thanks
Lamber-Ken
hi @Vinoth Chandar,
Got it, thanks.
best,
lamber-ken
At 2020-01-09 23:52:52, "Vinoth Chandar" wrote:
>Hi lamber-ken,
>
>A ConfigOption class would be good indeed. +1 on starting incrementally
>with DataSource first and then iterating..
>
>Thanks
>Vinoth
>
>On Tue, Jan 7, 2020 at 6:58
Hi @Y Ethan Guo,
Thanks for your enthusiasm, most of the translation work has been done.
The next work is that sync chinese translation docs in old site to new site
after the new site fully released.
Best,
Lamber-Ken
At 2020-01-09 14:12:27, "Y Ethan Guo" wrote:
>@lamber-ken Great
Hi lamber-ken,
A ConfigOption class would be good indeed. +1 on starting incrementally
with DataSource first and then iterating..
Thanks
Vinoth
On Tue, Jan 7, 2020 at 6:58 PM lamberken wrote:
>
>
> Hi @Vinoth,
>
>
> It's time to pick up this topic. Based on the content we talked about,
> here
@lamber-ken Great to hear that you've led the comms with ApacheCN! Let me
know if any help is needed. I'm also willing to help the translation work.
On Wed, Jan 8, 2020 at 3:58 PM lamberken wrote:
> Hello @Sudha,
>
>
> You are welcome, no need to say sorry :) . I just did something within my
Hello @Sudha,
You are welcome, no need to say sorry :) . I just did something within my
ability to promote hudi project, thanks.
Best,
Lamber-Ken
At 2020-01-09 05:41:11, "Bhavani Sudha Saktheeswaran"
wrote:
>Sorry for the late response. Just catching up on mailing list thread after
Hi @Y Ethan Guo,
Thanks, I've already been in touch with ApacheCN. http://hudi.apachecn.org is
coming.
Best,
Lamber-Ken
At 2020-01-08 15:21:51, "Y Ethan Guo" wrote:
>@lamber-ken
>
>Got it. It would be great if the ApacheCN organization can also help
>translation and promotion.
>
>The
@lamber-ken
Got it. It would be great if the ApacheCN organization can also help
translation and promotion.
The reason I'm asking about the Chinese docs is that the pages under
"Documentation" (e.g.,
http://hudi.apache.org/newsite-content/docs/writing_data.html) already have
the companion
Hi @Y Ethan Guo,
Thank you very much for your advice, I'll consider adjusting the font size.
For Chinese docs, I talked with @leesf about the chinese docs before, our
initial aim is to help user to learn hudi quickly, we should not translate the
whole site, it doesn't work very well.
@lamber-ken, Thanks for the great effort! The new website looks slick,
with a much better browsing experience.
One thing I noticed is that there seems to be no link to the Chinese
version of the docs on the new website. Wondering where I can find them.
Another minor thing is that the font
Hi @Vinoth,
It's time to pick up this topic. Based on the content we talked about, here are
my thoughts
1, Initial proposal aims to rework configuration framework includes(DataSource
and WriteClient level),
for compatibility, we can introduce a ConfigOption class and rework it on
Gread job, the new website has a much better experience of reading.
----
??:"Vinoth Chandar"http://hudi.apache.org/newsite-content/ if you are
wondering.. Based on the feedback, we plan to deprecate the old in the next
week or so. So please chime in.
Hi lamberken,
Thank you for your efforts. The new website definitely looks a lot better.
I found a minor issue. At the top where we are giving link to go back to
old site, the language seems incorrect. It says "Click here back to old
site". Rather it should be "Click here to go back to old
The new site is here http://hudi.apache.org/newsite-content/ if you are
wondering.. Based on the feedback, we plan to deprecate the old in the next
week or so. So please chime in.
Thanks Lamber-ken for the champion effort! As someone who shouldered the
site design since the beginning of the
hello everyone,
The new site has been merged into asf-site branch, the official website has
been updated, hope you all enjoy the new site style.
Please visit the new web site, if you notice any issues, feel free provide
feedback.
btw, thanks @Vinoth for reviewing carefully.
best,
Hi Nicholas,
We discussed about incremental mode for arbitrary queries at our end
and we came up with an idea. We can put a filtering criteria using
where clause and provide placeholder for incrementing conditions.
Thanks,
Rushiraj
On Mon, Jan 6, 2020 at 4:50 PM rushiraj chavan wrote:
>
> Hi
Hi Nicholas,
I guess it can be made pluggable. Sorry I didn't understand what you
meant by ... SPI
Purushotham(pushpavant...@gmail.com) and I are working on JDBC Delta
Streamer. It is WIP.
Thanks,
Rushiraj
On Mon, Jan 6, 2020 at 4:20 PM 蒋晓峰 wrote:
>
> Hi Rushiraj,
> As you said, Is the
Lamber,
Thanks for explaining, makes sense.
-Nishith
On Thu, Dec 19, 2019 at 5:29 PM lamberken wrote:
>
> Hi nishith,
>
> Thank you for your affirmation. The content in the blue box is to help us
> understand the highlighted content.
> It is different from the body content, so we need it.
Hi lamber,
Given we have enough +1s on the look and feel aspects, I propose we open a
PR and iron out the content/remaining issues there one by one.
I think a full line by line review is the best way to go, as with any code
change
Please share the PR here once you have it
Thanks
Vinoth
On
Hi leesf,
Thank you for your affirmation.
best,
lamber-ken
At 2019-12-21 07:28:50, "leesf" wrote:
Hi lamber,
Thanks for your great work, the new website looks much better.
Also if you guys have other companies(logos) needed to add to powered by(Hudi
Users)[1], please let
Hi lamber,
Thanks for your great work, the new website looks much better.
Also if you guys have other companies(logos) needed to add to powered
by(Hudi Users)[1], please let lamberken/me know before using new website.
Best,
Leesf
[1] https://lamber-ken.github.io/
lamberken 于2019年12月20日周五
Great job Lamber!
The website looks really slick and has a much better experience of moving
from one page to another (mostly I think because it's faster), also find it
the text much more conducive to absorb.
While going through the quick start, I noticed that under the highlighted
box in dark
Hi Lamber,
Awesome! Thanks for your hard work.
Best,
Vino
lamberken 于2019年12月19日周四 下午2:11写道：
>
>
> Hi everyone,
>
>
> I finished the rework of the new UI, if you have time, please visit the
> website[1].
> Any questions are welcome.
>
>
>
Hi everyone,
I finished the rework of the new UI, if you have time, please visit the
website[1].
Any questions are welcome.
[1]https://lamber-ken.github.io/docs/quick-start-guide/
best,
lamber-ken
At 2019-12-19 07:38:47, "lamberken" wrote:
>
>
>Hi @Shiyan Xu
>
>
>Thanks. :)
>best,
Sounds good.. This scoped down version per se, does not need a RFC.
On Wed, Dec 18, 2019 at 3:09 PM lamberken wrote:
>
>
> Hi @Vinoth
>
>
> I understand what you mean, I will continue to work on this when I finish
> reworking the new UI. :)
>
>
> best,
> lamber-ken
>
>
>
>
> At 2019-12-18
Hi @Shiyan Xu
Thanks. :)
best,
lamber-ken
At 2019-12-19 00:53:51, "Shiyan Xu" wrote:
>Thank you @lamber-ken for the work! It is definitely a greater browsing
>experience.
>
>On Tue, Dec 17, 2019 at 8:28 PM lamberken wrote:
>
>>
>> Hi, @Vinoth
>>
>>
>>
>> I'm glad to hear your thoughts on
Hi @Vinoth
I understand what you mean, I will continue to work on this when I finish
reworking the new UI. :)
best,
lamber-ken
At 2019-12-18 11:39:30, "Vinoth Chandar" wrote:
>Expect most users to use inputDF.write() approach... Uber uses the lower
>level RDD apis, like the
Thank you @lamber-ken for the work! It is definitely a greater browsing
experience.
On Tue, Dec 17, 2019 at 8:28 PM lamberken wrote:
>
> Hi, @Vinoth
>
>
>
> I'm glad to hear your thoughts on the new UI, thanks. So we keep its style
> as it is now.
> The development of new UI can be completed
Hi, @Vinoth
I'm glad to hear your thoughts on the new UI, thanks. So we keep its style as
it is now.
The development of new UI can be completed these days, any questions are
welcome.
best,
lamber-ken
At 2019-12-18 11:44:27, "Vinoth Chandar" wrote:
>The case for right navigation for me,
The case for right navigation for me, is mainly from pages like
https://lamber-ken.github.io/docs/docker_demo
https://lamber-ken.github.io/docs/querying_data
https://lamber-ken.github.io/docs/writing_data
which often have commands/text you want to selectively copy paste from a
section.
For
Expect most users to use inputDF.write() approach... Uber uses the lower
level RDD apis, like the DeltaStreamer tool does..
If we don't rename configs and still support a builder, it should be fine.
I think we can scope this down to introducing a ConfigOption class that
ties, the key,value,
hi, allOne more thing that is missing.In the new UI, I put a "BACK TO TOP"
button at the bottom of all pages to help us back to top.
We can also discuss whether we need the right navigation at the community
meeting today.best,
lamber-ken
At 2019-12-18 08:41:49, "lamberken" wrote:
>
One more thing that is missing.
Current site has a navigation links on the right, which lets you jump to
the right section directly. This is also a must-have IMHO.
I would suggest wait for more folks to come back from vacation, before we
finalize anything on this, as there could be more feedback
Hi Lamber,
+1 on the look and feel. Definitely feels slick and fast. Love the syntax
highlighting.
Few things :
- Can we just update the site content as-is? ( I'd rather change just the
look-and-feel and evolve the content from there, per usual means)
- Can we keep the theming blue and white,
https://cwiki.apache.org/confluence/display/HUDI/Community+Support Anyone
interested in giving the rotation a shot, please add your name next to the
a slot.
Let's see how this goes in Jan and we can learn
On Mon, Dec 16, 2019 at 5:27 AM Vinoth Chandar wrote:
> Thanks everyone for the
Thanks everyone for the suggestions. Always great to see a healthy
constructive discussion!
Overall it seems like
- At-least 3-4 are willing to give the support rotation a shot. As concrete
followup, will document the support responsibilities clearly in a wiki and
share it again here. To be
Hi, @vinoth
Okay, I see. If we don't want existing users to do any upgrading or
reconfigurations, then this refactor work will not make much sense.
This issue can be closed, because ConfigOptions and these builders do the same
things.
From another side, if we finish this work before a stable
Hi,
Are you saying these classes needs to change? If so, understood. But are
you planning on renaming configs or relocating them? We dont want existing
users to do any upgrading or reconfigurations
On Fri, Dec 13, 2019 at 10:28 AM lamberken wrote:
>
>
> Hi,
>
>
> They need to change due to
Hi,
They need to change due to this, because only HoodieWriteConfig and *Options
will be kept.
best,
lamber-ken
At 2019-12-14 01:23:35, "Vinoth Chandar" wrote:
>Hi,
>
>We are trying to understand if existing jobs (datasource, deltastreamer,
>anything else) needs to change due to this.
>
Hi,
We are trying to understand if existing jobs (datasource, deltastreamer,
anything else) needs to change due to this.
On Wed, Dec 11, 2019 at 7:18 PM lamberken wrote:
>
>
> Hi, @vinoth
>
>
> 1, Hoodie*Config classes are only used to set default value when call
> their build method
Hi, @vinoth
1, Hoodie*Config classes are only used to set default value when call their
build method currently.
They will be replaced by HoodieMemoryOptions, HoodieIndexOptions,
HoodieHBaseIndexOptions, etc...
2, I don't understand the question "It is not clear to me whether there is any
I actually prefer the builder pattern for making the configs, because I can
do `builder.` in the IDE and actually see all the options... That said,
most developers program against the Spark datasource and so this may not be
useful, unless we expose a builder for that.. I will concede that since
Let me summarize your initial proposal and then will get into details.
- Introduce ConfigOptions for ease of handling of default values.
- Remove all Hoodie*Config classes and just have HoodieWriteConfig. What
this means is that, every other config file will be replaced by
ConfigOptions. eg,
Hi Lamber-ken,
I looked at the sample PR you put up as well.
On 1,2 => Seems your intent is to replace these with moving the getter to
the component level Config class itself? I am fine with that (although I
think its not that big of a hurdle really to use atm). But, once we do that
we could
Hi lamberken,
Reasonable. Will change the description of jira issue soon.
Best,
Vino
lamberken 于2019年11月21日周四 下午4:22写道：
> Hi, vino. I think we can set severity to info level, if so, use can check
> style themself.
>
>
>
>
>
>
> At 2019-11-21 15:43:34, "vino yang" wrote:
> >Hi all,
> >
> >The
78 matches
Mail list logo
