Re: Tachyon in Spark

Thanks the response. I got the point - sounds like todays Spark linage dose not push to Tachyon linage. Would be good to see how it works. Jun Feng Liu. Haoyuan Li

Re: Tachyon in Spark

is not ready yet if tachyon linage dose not work right now? Best Regards Jun Feng Liu IBM China Systems Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: liuj...@cn.ibm.com BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193 China

Re: HA support for Spark

Interesting, you saying StreamContext checkpoint can regenerate DAG stuff? Best Regards Jun Feng Liu IBM China Systems Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: liuj...@cn.ibm.com BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193

Re: HA support for Spark

Well, it should not be mission impossible thinking there are so many HA solution existing today. I would interest to know if there is any specific difficult. Best Regards Jun Feng Liu IBM China Systems Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: liuj...@cn.ibm.com

Tachyon in Spark

Dose Spark today really leverage Tachyon linage to process data? It seems like the application should call createDependency function in TachyonFS to create a new linage node. But I did not find any place call that in Spark code. Did I missed anything? Best Regards Jun Feng Liu IBM China

Re: HA support for Spark

era.com To Jun Feng Liu/China/IBM@IBMCN, 2014-12-11 01:34 cc Reynold Xin r

Ooyala Spark JobServer

Hi, I am wondering the status of the Ooyala Spark Jobserver, any plan to get it into the spark release? Best Regards Jun Feng Liu IBM China Systems Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: liuj...@cn.ibm.com BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West

Re: Spark authenticate enablement

I see. Thank you, it works for me. It looks confusing to have two ways expose configuration though. Best Regards Jun Feng Liu IBM China Systems Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: liuj...@cn.ibm.com BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West

Spark authenticate enablement

authentication only work for like YARN model? Or I missed something with standalone model. Best Regards Jun Feng Liu IBM China Systems Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: liuj...@cn.ibm.com BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing

Re: Fine-Grained Scheduler on Yarn

Regards Jun Feng Liu IBM China Systems Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: liuj...@cn.ibm.com BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193 China Sandy Ryza sandy.r...@cloudera.com 2014/08/08 15:14 To Jun Feng Liu

Re: Fine-Grained Scheduler on Yarn

Any one know the answer? Best Regards Jun Feng Liu IBM China Systems Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: liuj...@cn.ibm.com BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193 China Jun Feng Liu/China/IBM 2014/08/07 15

Re: Fine-Grained Scheduler on Yarn

Thanks for echo on this. Possible to adjust resource based on container numbers? e.g to allocate more container when driver need more resources and return some resource by delete some container when parts of container already have enough cores/memory Best Regards Jun Feng Liu IBM China