Re: [build system] experiencing network issues, git fetch timeouts likely

Thanks Shane for taking care of this! On Mon, Apr 2, 2018 at 9:12 PM shane knapp wrote: > the problem was identified and fixed, and we should be good as of about an > hour ago. > > sorry for any inconvenience! > > On Mon, Apr 2, 2018 at 4:15 PM, shane knapp

Re: [build system] experiencing network issues, git fetch timeouts likely

the problem was identified and fixed, and we should be good as of about an hour ago. sorry for any inconvenience! On Mon, Apr 2, 2018 at 4:15 PM, shane knapp wrote: > hey all! > > we're having network issues on campus right now, and the jenkins workers > are experiencing

答复: Welcome Zhenhua Wang as a Spark committer

Thanks everyone! It’s my great pleasure to be part of such a professional and innovative community! best regards, -Zhenhua(Xander)

Re: Hadoop 3 support

Yes, the main blocking issue is the hive version used in Spark (1.2.1.spark) doesn't support run on Hadoop 3. Hive will check the Hadoop version in the runtime [1]. Besides this I think some pom changes should be enough to support Hadoop 3. If we want to use Hadoop 3 shaded client jar, then the

[build system] experiencing network issues, git fetch timeouts likely

hey all! we're having network issues on campus right now, and the jenkins workers are experiencing up to 40% packet loss on our pings to github. this can cause builds to time out when attempting to git fetch. i'll post an update on the network status when i find out more about what's going on.

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congratulations Zhenhua! On Mon, Apr 2, 2018 at 12:01 PM, ron8hu wrote: > Congratulations, Zhenhua! Well deserved!! > > Ron > > > > -- > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ > >

Re: Hadoop 3 support

I haven't looked at it in detail... Somebody's been trying to do that in https://github.com/apache/spark/pull/20659, but that's kind of a huge change. The parts where I'd be concerned are: - using Hive's original hive-exec package brings in a bunch of shaded dependencies, which may break Spark

Re: Hadoop 3 support

Is it difficult to upgrade Hive execution version to the latest version? The metastore used to be an issue but now that part had been separated from the execution part. On Mon, Apr 2, 2018 at 1:57 PM, Marcelo Vanzin wrote: > Saisai filed SPARK-23534, but the main blocking

Re: Hadoop 3 support

Saisai filed SPARK-23534, but the main blocking issue is really SPARK-18673. On Mon, Apr 2, 2018 at 1:00 PM, Reynold Xin wrote: > Does anybody know what needs to be done in order for Spark to support Hadoop > 3? > -- Marcelo

Re: Hadoop 3 support

That's just a nice to have improvement right? I'm more curious what is the minimal amount of work required to support 3.0, without all the bells and whistles. (Of course we can also do the bells and whistles, but those would come after we can actually get 3.0 running). On Mon, Apr 2, 2018 at

Re: Hadoop 3 support

Specifically to run spark with hadoop 3 docker support, I have filed a few jira's tracked under [1]. Regards, Mridul [1] https://issues.apache.org/jira/browse/SPARK-23717 On Mon, Apr 2, 2018 at 1:00 PM, Reynold Xin wrote: > Does anybody know what needs to be done in order

Hadoop 3 support

Does anybody know what needs to be done in order for Spark to support Hadoop 3?

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congratulations, Zhenhua! Well deserved!! Ron -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Welcome Zhenhua Wang as a Spark committer

Welcome! On Mon, Apr 2, 2018 at 11:00 AM, Takuya UESHIN wrote: > Congratulations! > > On Mon, Apr 2, 2018 at 10:34 AM, Dongjoon Hyun > wrote: > >> Congratulations! >> >> Bests, >> Dongjoon. >> >> On Mon, Apr 2, 2018 at 07:57 Cody Koeninger

Re: [Kubernetes] Resource requests and limits for Driver and Executor Pods

In summary, it looks like a combination of David's (#20943 ) and Yinan's PR (#20553 ) are good solutions here. Agreed on the importance of requesting memoryoverhead up front. I'm also wondering if we should

Re: Welcome Zhenhua Wang as a Spark committer

Congratulations! On Mon, Apr 2, 2018 at 10:34 AM, Dongjoon Hyun wrote: > Congratulations! > > Bests, > Dongjoon. > > On Mon, Apr 2, 2018 at 07:57 Cody Koeninger wrote: > >> Congrats! >> >> On Mon, Apr 2, 2018 at 12:28 AM, Wenchen Fan

Re: Welcome Zhenhua Wang as a Spark committer

Congratulations! Bests, Dongjoon. On Mon, Apr 2, 2018 at 07:57 Cody Koeninger wrote: > Congrats! > > On Mon, Apr 2, 2018 at 12:28 AM, Wenchen Fan wrote: > > Hi all, > > > > The Spark PMC recently added Zhenhua Wang as a committer on the project. > >

Re: Welcome Zhenhua Wang as a Spark committer

Congrats! On Mon, Apr 2, 2018 at 12:28 AM, Wenchen Fan wrote: > Hi all, > > The Spark PMC recently added Zhenhua Wang as a committer on the project. > Zhenhua is the major contributor of the CBO project, and has been > contributing across several areas of Spark for a while,

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congratulations! Xiao On Mon, Apr 2, 2018 at 4:57 AM Hadrien Chicault wrote: > Congrats > > Le lun. 2 avr. 2018 à 12:06, Weichen Xu a > écrit : > >> Congrats Zhenhua! >> >> On Mon, Apr 2, 2018 at 5:32 PM, Gengliang

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congrats Le lun. 2 avr. 2018 à 12:06, Weichen Xu a écrit : > Congrats Zhenhua! > > On Mon, Apr 2, 2018 at 5:32 PM, Gengliang wrote: > >> Congrats, Zhenhua! >> >> >> >> On Mon, Apr 2, 2018 at 5:19 PM, Marco Gaido >> wrote: >>

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congrats Zhenhua! On Mon, Apr 2, 2018 at 5:32 PM, Gengliang wrote: > Congrats, Zhenhua! > > > > On Mon, Apr 2, 2018 at 5:19 PM, Marco Gaido > wrote: > >> Congrats Zhenhua! >> >> 2018-04-02 11:00 GMT+02:00 Saisai Shao : >> >>>

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congrats, Zhenhua! On Mon, Apr 2, 2018 at 5:19 PM, Marco Gaido wrote: > Congrats Zhenhua! > > 2018-04-02 11:00 GMT+02:00 Saisai Shao : > >> Congrats, Zhenhua! >> >> 2018-04-02 16:57 GMT+08:00 Takeshi Yamamuro : >> >>>

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congrats Zhenhua! 2018-04-02 11:00 GMT+02:00 Saisai Shao : > Congrats, Zhenhua! > > 2018-04-02 16:57 GMT+08:00 Takeshi Yamamuro : > >> Congrats, Zhenhua! >> >> On Mon, Apr 2, 2018 at 4:13 PM, Ted Yu wrote: >> >>>

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congrats, Zhenhua! 2018-04-02 16:57 GMT+08:00 Takeshi Yamamuro : > Congrats, Zhenhua! > > On Mon, Apr 2, 2018 at 4:13 PM, Ted Yu wrote: > >> Congratulations, Zhenhua >> >> Original message >> From: 雨中漫步 <601450...@qq.com> >> Date:

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congrats, Zhenhua! On Mon, Apr 2, 2018 at 4:13 PM, Ted Yu wrote: > Congratulations, Zhenhua > > Original message > From: 雨中漫步 <601450...@qq.com> > Date: 4/1/18 11:30 PM (GMT-08:00) > To: Yuanjian Li , Wenchen Fan

Re: 回复： Welcome Zhenhua Wang as a Spark committer

Congratulations, Zhenhua  Original message From: 雨中漫步 <601450...@qq.com> Date: 4/1/18 11:30 PM (GMT-08:00) To: Yuanjian Li , Wenchen Fan Cc: dev Subject: 回复： Welcome Zhenhua Wang as a Spark committer

回复： Welcome Zhenhua Wang as a Spark committer

Congratulations Zhenhua Wang -- 原始邮件 -- 发件人: "Yuanjian Li"; 发送时间: 2018年4月2日(星期一) 下午2:26 收件人: "Wenchen Fan"; 抄送: "Spark dev list"; 主题: Re: Welcome Zhenhua Wang as a Spark committer

Re: Welcome Zhenhua Wang as a Spark committer

Congratulations Zhenhua!! 2018-04-02 13:28 GMT+08:00 Wenchen Fan : > Hi all, > > The Spark PMC recently added Zhenhua Wang as a committer on the project. > Zhenhua is the major contributor of the CBO project, and has been > contributing across several areas of Spark for a

Re: Welcome Zhenhua Wang as a Spark committer

Welcome, Zhenhua! Matei > On Apr 1, 2018, at 10:28 PM, Wenchen Fan wrote: > > Hi all, > > The Spark PMC recently added Zhenhua Wang as a committer on the project. > Zhenhua is the major contributor of the CBO project, and has been > contributing across several areas of