Re: mesos fetch uri when behind a squid proxy

[Zhongyue Luo]

Any advise on this issue? I'm having the same problem.

On Fri, Oct 9, 2015 at 4:13 AM, David M  wrote:

> Hi everyone.
>
> I have Mesos cluster (0.24.1) for running Spark (1.5.2) that runs great.
>
> I have a requirement to move my Mesos cluster nodes behind a Squid http
> proxy.
> All cluster nodes previously had direct outbound Internet access so
> accessing SPARK_EXECUTOR_URI from a public source was not a problem.
>
> System-wide I have http_proxy and https_proxy environment variables set.
> Command line tools like curl and wget operate just fine against Internet
> resources.
> After configuring Maven's proxy settings the Mesos build completed
> successfully.
>
> I copied my /etc/hosts file to HDFS and attempted the WordCount example
> from:
> http://documentation.altiscale.com/spark-shell-examples-1-1
>
> It failed with this in the executors stderr file:
>
> I1008 15:39:48.417644 21698 logging.cpp:172] INFO level logging started!
> I1008 15:39:48.417819 21698 fetcher.cpp:414] Fetcher Info:
> {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/20151007-154648-2701359370-5050-25191-S3\/spark","items":[{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"http:\/\/
> d3kbcqa49mib13.cloudfront.net
> \/spark-1.5.1-bin-hadoop2.6.tgz"}}],"sandbox_directory":"\/var\/run\/mesos\/slaves\/20151007-154648-2701359370-5050-25191-S3\/frameworks\/20151008-123957-2701359370-5050-6382-0001\/executors\/20151007-154648-2701359370-5050-25191-S3\/runs\/507827fb-cfb0-4a1d-977d-9b9afb972c29","user":"spark"}
> I1008 15:39:48.418918 21698 fetcher.cpp:369] Fetching URI '
> http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz'
> I1008 15:39:48.418936 21698 fetcher.cpp:243] Fetching directly into the
> sandbox directory
> I1008 15:39:48.418949 21698 fetcher.cpp:180] Fetching URI '
> http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz'
> I1008 15:39:48.418958 21698 fetcher.cpp:127] Downloading resource from '
> http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz' to
> '/var/run/mesos/slaves/20151007-154648-2701359370-5050-25191-S3/frameworks/20151008-123957-2701359370-5050-6382-0001/executors/20151007-154648-2701359370-5050-25191-S3/runs/507827fb-cfb0-4a1d-977d-9b9afb972c29/spark-1.5.1-bin-hadoop2.6.tgz'
> Failed to fetch '
> http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz':
> Error downloading resource, received HTTP return code 400
> Failed to synchronize with slave (it's probably exited)
>
> Long troubleshooting story short it appears that libcurl isn't finding out
> about my proxy.
>
> In ./3rdparty/libprocess/3rdparty/stout/include/stout/posix/net.hpp
>
> I added
>
>   curl_easy_setopt(curl, CURLOPT_VERBOSE, true);
>   curl_easy_setopt(curl, CURLOPT_PROXY, "");
>   curl_easy_setopt(curl, CURLOPT_PROXYPORT, );
>
> before
>
> CURLcode curlErrorCode = curl_easy_perform(curl);
>
> I then recompiled Mesos and the WordCount example is successful.
>
> What is the correct way to set proxy so that libcurl will make use of it?
>
> Thank you.
> David
>
>
>
>


-- 
*Intel SSG/STO/BDT*
880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
China
+862161166500

Introducing the ceph-mesos framework

[Zhongyue Luo]

Hi list,

Me and my colleges developed a framwork called ceph-mesos. As you've
already predicted from its name, the framework is for scaling Ceph clusters
on Mesos.

Check out the code on Github https://github.com/Intel-bigdata/ceph-mesos

We've just announced this at the 2nd Beijing MUG meetup.Here is the link to
the presentation
https://docs.google.com/presentation/d/1AzcOD9Aug6BrWevdpHXgyXlcHUzVq2_MMRNvutLelJY/edit?usp=sharing
.

There is also a demo video
http://v.youku.com/v_show/id_XMTI3MjMxNTU5Ng==.html. Audio is in Chinese
but you won't have a problem following through even if you mute.

Thanks.


-- 
*Intel SSG/STO/BDT*
880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
China
+862161166500

Re: Register User Group n Bejing ,China

[Zhongyue Luo]

I think 16 and 23 doesn't make a difference. Tommy's group has already
scheduled a meetup and also registered first. I think we should merge with
http://www.meetup.com/Beijing-Mesos-User-Group/

On Mon, May 11, 2015 at 12:02 PM, Sam Chen usultra...@gmail.com wrote:

 Dave and Tommy
 We got 23 Mesosers in this User Group , And this Meetup will be funded by
 LinkerNetworks Inc, which is focusing on Mesos and relevant development .
  Do you guys think  it will be good to merge your guys into our big User
 Group .
 Looking forward to hear your guys feedback .

 Sam

 On Mon, May 11, 2015 at 8:48 AM, Dave Lester d...@davelester.org wrote:

  Hi Sam and Tommy,

 It's awesome to hear about your interest in a Beijing MUG!

 Could you guys potentially merge these separate meetup groups so we have
 a single group to link to?

 Thanks,
 Dave

 On Sun, May 10, 2015, at 05:14 PM, tommy xiao wrote:

 Hi Sam,

 nice to meet you by email. meetup.com have already approved my request(
 http://www.meetup.com/mail/178886170210516992/). Could you please have
 some time to talk about beijing mesos user meetup in May 17.

 Mesos Beijing Kickoff Meeting
 (http://www.meetup.com/Beijing-Mesos-User-Group/events/222428331/)


 2015-05-10 23:08 GMT+08:00 Sam Chen usultra...@gmail.com:

 Hi , Guys,
 My name is Sam , I am working in China ,Beijing .  We got bunch of guys
 who is very interested in Mesos . Could you please add our Meetup in your
 guys page
 :http://mesos.apache.org/community/user-groups/

 Our Meetup URL  : http://www.meetup.com/China-mesos-Meetup/  ,Appreciated
 .




 --
 Deshi Xiao
 Twitter: xds2000
 E-mail: xiaods(AT)gmail.com







-- 
*Intel SSG/STO/BDT*
880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
China
+862161166500

Re: Shenzhen MUG First Meetup !

[Zhongyue Luo]

Thanks Zhipeng for organizing this event. See you at Shenzhen!

On Sat, Apr 18, 2015 at 2:26 PM, Zhipeng Huang zhipengh...@gmail.com
wrote:

 Not in production at the moment, as far as I know :)

 On Sat, Apr 18, 2015 at 11:35 AM, haosdent haosd...@gmail.com wrote:

 does huawei use Mesos?

 On Fri, Apr 17, 2015 at 9:49 PM, Zhipeng Huang zhipengh...@gmail.com
 wrote:

 Our first meetup has been anounced ! Please check it out at
 http://www.meetup.com/Shenzhen-Mesos-User-Group/events/221879815/ . We
 will have about three topics, and Shenzhen MUG T-shirts :P

 --
 Zhipeng (Howard) Huang

 Standard Engineer
 IT Standard  Patent/IT Prooduct Line
 Huawei Technologies Co,. Ltd
 Email: huangzhip...@huawei.com
 Office: Huawei Industrial Base, Longgang, Shenzhen

 (Previous)
 Research Assistant
 Mobile Ad-Hoc Network Lab, Calit2
 University of California, Irvine
 Email: zhipe...@uci.edu
 Office: Calit2 Building Room 2402

 OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado




 --
 Best Regards,
 Haosdent Huang




 --
 Zhipeng (Howard) Huang

 Standard Engineer
 IT Standard  Patent/IT Prooduct Line
 Huawei Technologies Co,. Ltd
 Email: huangzhip...@huawei.com
 Office: Huawei Industrial Base, Longgang, Shenzhen

 (Previous)
 Research Assistant
 Mobile Ad-Hoc Network Lab, Calit2
 University of California, Irvine
 Email: zhipe...@uci.edu
 Office: Calit2 Building Room 2402

 OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado




-- 
*Intel SSG/STO/BDT*
880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
China
+862161166500

Re: MUG in Shanghai

[Zhongyue Luo]

Thanks. Will keep you posted.

On Wed, Mar 11, 2015 at 9:50 AM, Dave Lester d...@davelester.org wrote:

  Hi Zhongyue,

 This is great to see, I've added this to the Mesos User Group list on the
 Mesos website: http://mesos.apache.org/community/user-groups/

 I'd encourage you to send out a hello world message on your Weibo group
 so there's some content; I can help drive some interest there from our
 Twitter account, @ApacheMesos. Please also let the user list know when you
 host your first meetup!

 Cheers,
 Dave

 On Tue, Mar 10, 2015, at 06:19 PM, Nah, Zhongyue wrote:

 Hi,


 I’d like to start a MUG in Shanghai, China.


 Meetup and Twitter are not that convenient for us in China so I made a
 group page in a Chinese micro SNS site.


 http://weibo.com/3068800961/


 We plan to have a meetup soon in 2Q.


 Anything else required?


 Regards,

 Zhongyue


 --


 *Intel SSG/STO/BDT*

 880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
 China

 +862161166500







-- 
*Intel SSG/STO/DCST/CIT*
880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
China
+862161166500