Build failed in Jenkins: aurora-packaging-nightly #238

2016-03-25 Thread Apache Jenkins Server 
See 

--
[...truncated 20312 lines...]
Get:66 http://httpredir.debian.org/debian/ jessie/main python amd64 2.7.9-1 
[151 kB]
Get:67 http://httpredir.debian.org/debian/ jessie/main libapr1 amd64 1.5.1-3 
[95.3 kB]
Get:68 http://httpredir.debian.org/debian/ jessie/main libaprutil1 amd64 
1.5.4-1 [86.2 kB]
Get:69 http://httpredir.debian.org/debian/ jessie/main libasan1 amd64 4.9.2-10 
[195 kB]
Get:70 http://httpredir.debian.org/debian/ jessie/main libatomic1 amd64 
4.9.2-10 [8992 B]
Get:71 http://httpredir.debian.org/debian/ jessie/main libcap-ng0 amd64 0.7.4-2 
[13.2 kB]
Get:72 http://httpredir.debian.org/debian/ jessie/main libcilkrts5 amd64 
4.9.2-10 [40.1 kB]
Get:73 http://httpredir.debian.org/debian/ jessie/main libisl10 amd64 0.12.2-2 
[440 kB]
Get:74 http://httpredir.debian.org/debian/ jessie/main libcloog-isl4 amd64 
0.18.2-1+b2 [61.8 kB]
Get:75 http://httpredir.debian.org/debian/ jessie/main libglib2.0-0 amd64 
2.42.1-1 [2401 kB]
Get:76 http://httpredir.debian.org/debian/ jessie/main libcroco3 amd64 
0.6.8-3+b1 [135 kB]
Get:77 http://httpredir.debian.org/debian/ jessie/main librtmp1 amd64 
2.4+20150115.gita107cef-1 [59.8 kB]
Get:78 http://httpredir.debian.org/debian/ jessie/main libdbus-1-3 amd64 
1.8.20-0+deb8u1 [170 kB]
Get:79 http://httpredir.debian.org/debian/ jessie/main libdbus-glib-1-2 amd64 
0.102-1 [201 kB]
Get:80 http://httpredir.debian.org/debian/ jessie/main libunistring0 amd64 
0.9.3-5.2+b1 [288 kB]
Get:81 http://httpredir.debian.org/debian/ jessie/main libgettextpo0 amd64 
0.19.3-2 [138 kB]
Get:82 http://httpredir.debian.org/debian/ jessie/main libgomp1 amd64 4.9.2-10 
[37.8 kB]
Get:83 http://httpredir.debian.org/debian/ jessie/main libitm1 amd64 4.9.2-10 
[29.2 kB]
Get:84 http://httpredir.debian.org/debian/ jessie/main liblsan0 amd64 4.9.2-10 
[92.8 kB]
Get:85 http://httpredir.debian.org/debian/ jessie/main libmpdec2 amd64 2.4.1-1 
[85.7 kB]
Get:86 http://httpredir.debian.org/debian/ jessie/main libmpfr4 amd64 3.1.2-2 
[527 kB]
Get:87 http://httpredir.debian.org/debian/ jessie/main libossp-uuid16 amd64 
1.6.2-1.5+b1 [38.0 kB]
Get:88 http://httpredir.debian.org/debian/ jessie/main libpython2.7 amd64 
2.7.9-2 [1080 kB]
Get:89 http://httpredir.debian.org/debian/ jessie/main libexpat1-dev amd64 
2.1.0-6+deb8u1 [126 kB]
Get:90 http://httpredir.debian.org/debian/ jessie/main libpython2.7-dev amd64 
2.7.9-2 [18.6 MB]
Get:91 http://httpredir.debian.org/debian/ jessie/main libpython3.4-minimal 
amd64 3.4.2-1 [492 kB]
Get:92 http://httpredir.debian.org/debian/ jessie/main libpython3.4-stdlib 
amd64 3.4.2-1 [2088 kB]
Get:93 http://httpredir.debian.org/debian/ jessie/main libserf-1-1 amd64 
1.3.8-1 [52.8 kB]
Get:94 http://httpredir.debian.org/debian/ jessie/main libquadmath0 amd64 
4.9.2-10 [129 kB]
Get:95 http://httpredir.debian.org/debian/ jessie/main libsctp1 amd64 
1.0.16+dfsg-2 [27.6 kB]
Get:96 http://httpredir.debian.org/debian/ jessie/main libsigsegv2 amd64 
2.10-4+b1 [29.2 kB]
Get:97 http://httpredir.debian.org/debian/ jessie/main libsvn1 amd64 
1.8.10-6+deb8u2 [1077 kB]
Get:98 http://httpredir.debian.org/debian/ jessie/main libtsan0 amd64 4.9.2-10 
[212 kB]
Get:99 http://httpredir.debian.org/debian/ jessie/main libubsan0 amd64 4.9.2-10 
[82.4 kB]
Get:100 http://httpredir.debian.org/debian/ jessie/main libxau6 amd64 1:1.0.8-1 
[20.7 kB]
Get:101 http://httpredir.debian.org/debian/ jessie/main libxdmcp6 amd64 
1:1.1.1-1+b1 [24.9 kB]
Get:102 http://httpredir.debian.org/debian/ jessie/main libxcb1 amd64 1.10-3+b1 
[44.4 kB]
Get:103 http://httpredir.debian.org/debian/ jessie/main libx11-data all 
2:1.6.2-3 [126 kB]
Get:104 http://httpredir.debian.org/debian/ jessie/main libx11-6 amd64 
2:1.6.2-3 [729 kB]
Get:105 http://httpredir.debian.org/debian/ jessie/main libxext6 amd64 
2:1.3.3-1 [52.7 kB]
Get:106 http://httpredir.debian.org/debian/ jessie/main libxmuu1 amd64 
2:1.1.2-1 [23.3 kB]
Get:107 http://httpredir.debian.org/debian/ jessie/main python3.4-minimal amd64 
3.4.2-1 [1646 kB]
Get:108 http://httpredir.debian.org/debian/ jessie/main sgml-base all 1.26+nmu4 
[14.6 kB]
Get:109 http://httpredir.debian.org/debian/ jessie/main libmpc3 amd64 1.0.2-1 
[39.3 kB]
Get:110 http://httpredir.debian.org/debian/ jessie/main apt-utils amd64 
1.0.9.8.2 [368 kB]
Get:111 http://httpredir.debian.org/debian/ jessie/main less amd64 458-3 [124 
kB]
Get:112 http://httpredir.debian.org/debian/ jessie/main manpages all 3.74-1 
[997 kB]
Get:113 http://httpredir.debian.org/debian/ jessie/main at amd64 3.1.16-1 [45.4 
kB]
Get:114 http://httpredir.debian.org/debian/ jessie/main bsd-mailx amd64 
8.1.2-0.20141216cvs-2 [81.7 kB]
Get:115 http://httpredir.debian.org/debian/ jessie/main bzip2 amd64 1.0.6-7+b3 
[46.9 kB]
Get:116 http://httpredir.debian.org/debian/ jessie/main dbus amd64 
1.8.20-0+deb8u1 [291 kB]
Get:117 http://httpredir.debian.org/debian/ jessie/main file amd64 
1:5.22+15-2+deb8u1 [60.4 kB]
Get:118

Re: Populate DiscoveryInfo in Mesos

2016-03-25 Thread Zhitao Li 
Hi Stephan,

Thanks for looking at that prototype patch.

I'll update the patch with the review comments, and probably add a global
flag of "populate_discovery_info" to toggle this behavior.

About the optional fields: I think it'll be hard to come up a good set of
rules applicable to all orgs using Aurora + Mesos, because cluster
management and service discovery stack could differ from org to org.

In a recent Mesos work group, some experience folks (Jie Yu and Ben Mahler)
mentioned some ideas of *TaskInfoDecorator, *which is some optional and
configurable plugin on Aurora scheduler side to allow operator to set
additional fields before sending the message to Mesos. I like such idea
because it would enable Aurora users to experiment faster. Do you think
this is an interesting idea worth pursuing?


On Fri, Mar 25, 2016 at 1:42 PM, Erb, Stephan 
wrote:

> I had a closer look at the Mesos documentation, and a design document
> might be unnecessary. Most of the values are optional. We can therefore
> leave them out until we have a proper usecase for them.
>
> I left a couple of comments in the review request.
> 
> From: Zhitao Li 
> Sent: Tuesday, March 22, 2016 21:15
> To: dev@aurora.apache.org
> Subject: Re: Populate DiscoveryInfo in Mesos
>
> Hi Stephan,
>
> Sorry for the delay on follow up on this. I took a quick look at Aurora
> code, and it's actually quite easy to pipe this information to Mesos (see
> https://reviews.apache.org/r/45177/ for quick prototype).
>
> I'll take a stab to see how I can get Mesos-DNS to work with this
> prototype.
>
> IMO, if this is something the community is interested, the main questions
> would be 1) how various fields would be mapped in different Aurora usages,
> and 2) to which level should opt-in/opt-out configured for populating such
> information.
>
> I actually don't have too much insights on how these usage conventions
> would be set (through command line of scheduler or job configuration?)
>
> Do you think a design doc is the best action here, or a more involved
> questionnaire about which fields would be useful for community, or what
> value they should take?
>
> On Mon, Mar 7, 2016 at 1:00 AM, Erb, Stephan 
> wrote:
>
> > That sounds like a good idea! Great.
> >
> > If you go ahead with this, please be so kind and start by posting a short
> > design document here on mailinglist (similar to those here
> > https://github.com/apache/aurora/blob/master/docs/design-documents.md,
> > but probably shorter).
> >
> > This will allow us to split the discussion of the design from discussing
> > the actual implementation. I believe this is necessary, as the
> > DiscoveryInfo protocol is quite flexible (
> >
> http://mesos.apache.org/documentation/latest/app-framework-development-guide/
> > ).
> >
> > Thanks,
> > Stephan
> >
> >
> > 
> > From: Zhitao Li 
> > Sent: Monday, March 7, 2016 00:05
> > To: dev@aurora.apache.org
> > Subject: Populate DiscoveryInfo in Mesos
> >
> > Hi,
> >
> > It seems like Aurora does not populate the "discovery" field in either
> > TaskInfo or ExecutorInfo in mesos.proto
> > <
> >
> https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L438
> > >
> > .
> >
> > I'm considering adding this to support retrieving port map in Mesos
> > directly. This would enable us to discovery this information directly
> from
> > Mesos side, and also enables us to build one universal service discovery
> > solution for multiple frameworks including Aurora.
> >
> > If no objection, I'll create a JIRA ticket for this task.
> >
> > Thanks.
> > --
> > Cheers,
> >
> > Zhitao Li
> >
>
>
>
> --
> Cheers,
>
> Zhitao Li
>



-- 
Cheers,

Zhitao Li

Re: Populate DiscoveryInfo in Mesos

2016-03-25 Thread Erb, Stephan 
I had a closer look at the Mesos documentation, and a design document might be 
unnecessary. Most of the values are optional. We can therefore leave them out 
until we have a proper usecase for them.

I left a couple of comments in the review request. 

From: Zhitao Li 
Sent: Tuesday, March 22, 2016 21:15
To: dev@aurora.apache.org
Subject: Re: Populate DiscoveryInfo in Mesos

Hi Stephan,

Sorry for the delay on follow up on this. I took a quick look at Aurora
code, and it's actually quite easy to pipe this information to Mesos (see
https://reviews.apache.org/r/45177/ for quick prototype).

I'll take a stab to see how I can get Mesos-DNS to work with this prototype.

IMO, if this is something the community is interested, the main questions
would be 1) how various fields would be mapped in different Aurora usages,
and 2) to which level should opt-in/opt-out configured for populating such
information.

I actually don't have too much insights on how these usage conventions
would be set (through command line of scheduler or job configuration?)

Do you think a design doc is the best action here, or a more involved
questionnaire about which fields would be useful for community, or what
value they should take?

On Mon, Mar 7, 2016 at 1:00 AM, Erb, Stephan 
wrote:

> That sounds like a good idea! Great.
>
> If you go ahead with this, please be so kind and start by posting a short
> design document here on mailinglist (similar to those here
> https://github.com/apache/aurora/blob/master/docs/design-documents.md,
> but probably shorter).
>
> This will allow us to split the discussion of the design from discussing
> the actual implementation. I believe this is necessary, as the
> DiscoveryInfo protocol is quite flexible (
> http://mesos.apache.org/documentation/latest/app-framework-development-guide/
> ).
>
> Thanks,
> Stephan
>
>
> 
> From: Zhitao Li 
> Sent: Monday, March 7, 2016 00:05
> To: dev@aurora.apache.org
> Subject: Populate DiscoveryInfo in Mesos
>
> Hi,
>
> It seems like Aurora does not populate the "discovery" field in either
> TaskInfo or ExecutorInfo in mesos.proto
> <
> https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L438
> >
> .
>
> I'm considering adding this to support retrieving port map in Mesos
> directly. This would enable us to discovery this information directly from
> Mesos side, and also enables us to build one universal service discovery
> solution for multiple frameworks including Aurora.
>
> If no objection, I'll create a JIRA ticket for this task.
>
> Thanks.
> --
> Cheers,
>
> Zhitao Li
>



--
Cheers,

Zhitao Li