Re:[DISCUSS] Relocate spark-avro dependency by maven-shade-plugin

2020-02-05 Thread hmatu
Thanks for raising this! +1 to @Udit Mehrotra's point.


 It's right that recommend users to actually build their  own hudi jars, with 
the spark version they use. It avoid the compatibility issues 

between user's local jars and pre-built hudi spark version(2.4.4).

Or can remove "org.apache.spark:spark-avro_2.11:2.4.4"? Because user local env 
will contains that external dependency if they use avro.

If not, to run hudi(release-0.5.1) is more complex for me, when using Delta 
Lake, it's more simpler:
just "bin/spark-shell --packages io.delta:delta-core_2.11:0.5.0"







-- Original --
From: "lamberken"http://spark.apache.org/docs/latest/sql-data-sources-avro.html
>[2] 
http://mirror.bit.edu.cn/apache/spark/spark-2.4.4/spark-2.4.4-bin-hadoop2.7.tgz 
>[3] https://github.com/apache/incubator-hudi/pull/1290
>

Re: [DISCUSS] Unify Hudi code cleanup and improvement

2020-01-30 Thread hmatu
Hi,



I think these "MINOR" issues are important, a good project requires not 
only functions, but also good coding style and  habits.


Best
Hmatu




-- Original --
From: "Vinoth Chandar"https://github.com/apache/incubator-hudi/pull/1237
> [2]: https://github.com/apache/incubator-hudi/pull/1139
> [3]: https://github.com/apache/incubator-hudi/pull/1137
> [4]: https://github.com/apache/incubator-hudi/pull/1136
>
> Vinoth Chandar 

Re: [DISCUSS] Remove HoodieWriteClient

2020-01-27 Thread hmatu
Thanks. 



IMO, we should focus more on SparkDatasource level, not the compatibility with 
the HoodieClient level.


Thanks,
Hmatu





-- Original --
From: "Vinoth Chandar"

[DISCUSS] Remove HoodieWriteClient

2020-01-26 Thread hmatu
Hi guys,


As we know, hudi project contains HoodieWriteClient and HoodieSparkSource level 
framework. But may 99% user just use HoodieSparkSource except for uber. So I 
suggest remove HoodieWriteClient. WDYT?


Thanks
Hmatu

Re: Re: [DISCUSS] Redraw of hudi data lake architecture diagram on langing page

2020-01-23 Thread hmatu
The whole site looks better than old currently, big thanks for your work!


Thanks,
Hmatu



-- Original --
From: "Balaji Varadarajan"https://hudi.apache.org
> [2] https://lamber-ken.github.io
>
>
> Thanks
> Lamber-Ken
 

Re: Re: [DISCUSS] Redraw of hudi data lake architecture diagram on langing page

2020-01-23 Thread hmatu
Looks great. +1


-- Original --
From: "Balaji Varadarajan"https://hudi.apache.org
> [2] https://lamber-ken.github.io
>
>
> Thanks
> Lamber-Ken
 

?????? Re: Re: Re: Re:Re: Re: Re:Re: Re: Re: [DISCUSS] Rework of new web site

2020-01-07 Thread hmatu
Gread job, the new website has a much better experience of reading.




--  --
??: "Vinoth Chandar"http://hudi.apache.org/newsite-content/ if you are
wondering.. Based on the feedback, we plan to deprecate the old in the next
week or so. So please chime in.

Thanks Lamber-ken for the champion effort! As someone who shouldered the
site design since the beginning of the project, I am very happy to see
something better finally replace it :D

On Mon, Jan 6, 2020 at 10:38 PM lamberken https://lamber-ken.github.io/
> >>
> >>
> >> lamberken https://lamber-ken.github.io/docs/quick-start-guide/
> >> >> >
> >> >> >
> >> >> > best,
> >> >> > lamber-ken
> >> >> >
> >> >> >
> >> >> >
> >> >> > At 2019-12-19 07:38:47, "lamberken" 
https://lamber-ken.github.io/docs/docker_demo
> >> >> > >>> 
>https://lamber-ken.github.io/docs/querying_data
> >> >> > >>> 
>https://lamber-ken.github.io/docs/writing_data
> >> >> > >>> >
> >> >> > >>> >which often have commands/text you 
want to selectively copy
> paste
> >> >> > from a
> >> >> > >>> >section.
> >> >> > >>> >For content you read sequentially, 
it matters less. I agree..
> >> >> > >>> >
> >> >> > >>> >BTW the new site looks very 
sleek.. :)
> >> >> > >>> >
> >> >> > >>> >
> >> >> > >>> >
> >> >> > >>> >On Tue, Dec 17, 2019 at 4:50 PM 
lamberken https://lamber-ken.github.io/docs/admin_guide
> >> >> > >>> >> >[2] 
https://lamber-ken.github.io/docs/writing_data
> >> >> > >>> >> >[3] 
https://lamber-ken.github.io/docs/quick-start-guide/
> >> >> > >>> >> >
> >> >> > >>> >> >
> >> >> > >>> >> >best,
> >> >> > >>> >> >lamber-ken
> >> >> > >>> >> >
> >> >> > >>> >> >
> >> >> > >>> >> >
> >> >> > >>> >> >
> >> >> > >>> >> >At 2019-12-18 04:44:04, 
"Vinoth Chandar" <
> vin...@apache.org>
> >> >> > wrote:
> >> >> > >>> >> >>One more thing that 
is missing.
> >> >> > >>> >> >>
> >> >> > >>> >> >>Current site has a 
navigation links on the right, which
> lets
> >> you
> >> >> > jump
> >> >> > >>> to
> >> >> > >>> >> >>the right section 
directly. This is also a must-have IMHO.
> >> >> > >>> >> >>I would suggest wait 
for more folks to come back from
> >> vacation,
> >> >> > >>> before we
> >> >> > >>> >> >>finalize anything on 
this, as there could be more feedback
> >> >> > >>> >> >>
> >> >> > >>> >> >>
> >> >> > >>> >> >>
> >> >> > >>> >> >>On Mon, Dec 16, 2019 
at 9:15 PM lamberken <
> lamber...@163.com
> >> >
> >> >> > wrote:
> >> >> > >>> >> >>
> >> >> > >>> >> >>>
> >> >> > >>> >> >>> Hi Vinoth,
> >> >> > >>> >> >>>
> >> >> > >>> >> >>>
> >> >> > >>> >> >>> 1, I'll update 
the site content this week, clean some
> >> useless
> >> >> > >>> templete
> >> >> > >>> >> >>> codes, adjust 
the content etc...
> >> >> > >>> >> 
>>> It will take a little long time for 
syncing the
> >> content.
> >> >> > >>> >> >>> 2, I will adjust 
the style as much as I can to keep the
> >> >> theming
> >> >> > blue
> >> >> > >>> >> and
> >> >> > >>> >> >>> white.
> >> >> > >>> >> >>>
> >> >> > >>> >> >>>
> >> >> > >>> >> >>> When the above 
work is completed, I will notify you all
> >> again.
> >> >> > >>> >> >>> best,
> >> >> > >>> >> >>> lamber-ken
> >> >> > >>> >> >>>
> >> >> > >>> >> >>>
> >> >> > >>> >> >>> At 2019-12-17 
12:49:23, "Vinoth Chandar" <
> >> vin...@apache.org>
> >> >> > wrote:
> >> >> > >>> >> >>> >Hi Lamber,
> >> >> > >>> >> >>> >
> >> >> > >>> >> >>> >+1 on the 
look and feel. Definitely feels slick and
> fast.
> >> >> Love
> >> >> > the
> >> >> > >>> >> syntax
> >> >> > >>> >> >>> >highlighting.
> >> >> > >>> >> >>> >
> >> >> > >>> >> >>> >
> >> >> > >>> >> >>> >Few things :
> >> >> > >>> >> >>> >- Can we 
just update the site content as-is? ( I'd
> rather
> >> >> > change
> >> >> > >>> just
> >> >> > >>> >> the
> >> >> > >>> >> >>> 
>look-and-feel and evolve the content from there, per
> usual
> >> >> > means)
> >> >> > >>> >> >>> >- Can we 
keep the theming blue and white, like now,
> since
> >> it
> >> >> > gels
> >> >> > >>> well
> >> >> > >>> >> >>> with
> >> >> > >>> >> >>> >the logo and 
images.
> >> >> > >>> >> >>> >
> >> >> > >>> >> >>> >
> >> >> > >>> >> >>> >On Mon, Dec 
16, 2019 at 8:02 AM lamberken <
> >> lamber...@163.com
> >> >> >
> >> >> > >>> wrote:
> >> >> > >>> >> >>> >
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >> Thanks 
for your reply @lees @vino @vinoth :)
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >> best,
> >> >> > >>> >> >>> >> 
lamber-ken
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >>
> >> >> > >>> >> >>> >> ?? 
2019-12-16 12:24:26??"leesf" https://lamber-ken.github.io
> >> >> > >>> >> >>> >> 
>> > > for a quick overview.
> >> >> > >>> >> >>> >> 
>> > >
> >> >> > >>> >> >>> >> 
>> > >
> >> >> > >>> >> >>> >> 
>> > > I??m looking forward to your reply, thanks!
> >> >> > >>> >> >>> >> 
>> > >
> >> >> > >>> >> >>> >> 
>> > >
> >> >> > >>> >> >>> >> 
>> > > [1] https://