MesosCon Asia CFP deadline extended

2017-03-27 Thread Jie Yu 
Chinese version below/ 中文详情请见下文

In order to allow for more submissions, we have extended the CFP deadline
for MesosCon Asia until April 5th 11:59pm PDT.
http://events.linuxfoundation.org/events/mesoscon-asia/program/cfp

None of the other submission dates have changed. Community voting on
submissions will still happen between April 10th and 16th. We will let you
know whether or not your talk has been accepted on April 22nd (as planned),
so you should have plenty of time to make travel arrangements.

MesosCon Asia will have simultaneous interpretation and we encourage
chinese talks, but request that you submit your abstract in English to make
community voting simpler.

Looking forward to many great submissions and talks,
Your MesosCon PMC

-

MesosCon Asia CFP截止日期延至4月5日

为了获得更多的演讲提案，我们将今年MesosCon Asia的CFP提交截止日期延至美西太平洋时间4月5日11:59pm。
http://events.linuxfoundation.org/events/mesoscon-asia/program/cfp

其余截止日期皆维持不变，演讲提案的社区公开投票仍将在4月10日至4月16日间进行。为了让您获得足够充裕的时间提前筹备旅行计划，我们将于4月22日通知您演讲提案是否获得通过。

MesosCon
Asia将提供同声传译服务，同时我们鼓励大家使用中文进行演讲，但您的演讲提案摘要(abstract)仍需使用英文撰写，以便于社区进行投票。

期待大家的提案及演讲，
MesosCon PMC

Proposal: Precompiled headers for Windows use in cmake

2017-03-27 Thread Jeff Coffler 
I had previously send an E-Mail to the Mesos DEV list on Tue 2/14/2017 11:29 
AM. I've attached that message here for reference (hopefully the attachment 
will make it across).

Work has proceeded, and I'm working out some commits to apply to the changes 
for Windows (cmake only).

While I was asked to work out timing for Linux platforms, I was unable to do so 
as the Linux code doesn't currently build with precompiled headers due to 
overloading of the std namespace (Set/set, as I recall). We could hack sources 
to build with precompiled headers, but this would be throw-away work and thus 
doesn't seem all that useful an engineering expenditure. But without the 
permanent changes needed there to avoid namespace pollution, or hack/throw away 
changes, it's not possible to get timing tests for PCH use on Linux at this 
time.

For Windows, here are the results of timing tests that I've done:

Build times on Windows (no PCH):
  1. Full build time (everything including 3rd party products): 24:49.47 
(1489.47s total)
  2. Time to rebuild all of Mesos itself: 21:8.63 (1268.63s total)
  3. Time for a build with no changes: 0:30.47 (30.47s total)
  4. Modified file 3rdparty/stout/include/stout/os/os.hpp (added a comment)
  5. Time for an incremental build: 36:55.55 (2215.55s total)
 (Very odd that an incremental takes longer than a full build!)
  6. Time for an incremental build (just mesos-agent): 0:14:24.73 (864.73s 
total)
  7. Time for an incremental build, mesos-agent only: 0:16:42.57 (1002.57s 
total)
 (Includes speedup to linker flags)


Build times on Windows (with PCH):
  1. Full build time (everything including 3rd party products): 19:54.49 
(1194.49s total)
  2. Time to rebuild all of Mesos itself: 0:15:49.58 (949.58s total)
  3. Time for a build with no changes: 0:0:21.72 (21.72s total)
  4. Modified file 3rdparty/stout/include/stout/os/os.hpp (added a comment)
  5. Time for an incremental build: 0:34:40.64 (2080.64s total)
  6. Time for an incremental build (just mesos-agent): 0:11:53.57 (713.57s 
total)
  7. Time for an incremental build, mesos-agent only: 0:10:43.10 (643.10s total)
 (Includes speedup to linker flags)

To be clear here, more work can be done. We pretty much just optimized building 
mesos-agent. There are other targets that would benefit from PCH as well. 
However, unless/until we start doing major portions of work in those areas, it 
may not make sense. (For example, we're unlikely to do major pieces of redesign 
to most of our 3rd party components, nor do we have the ability to change those 
build processes anyway.)

The above timing tests include the time to actually build the PCH. In the full 
build case, the reduction is close to 25%. In the incremental build case (#7), 
the reduction is closer to 33%. In both cases, we believe that this change is 
worthwhile.

Any questions/concerns? If not, I'll work on finalizing the commits and getting 
them tested in the next few days.

/Jeff
--- Begin Message ---
Proposal For Build Improvements

The Mesos build process is in dire need of some build infrastructure 
improvements. These improvements will improve speed and ease of work in 
particular components, and dramatically improve overall build time, especially 
in the Windows environment, but likely in the Linux environment as well.


Background:

It is currently recommended to use the ccache project with the Mesos build 
process. This makes the Linux build process more tolerable in terms of speed, 
but unfortunately such software is not available on Windows. Ultimately, 
though, the caching software is covering up two fundamental flaws in the 
overall build process:

1. Lack of use of libraries
2. Lack of precompiled headers

By not allowing use of libraries, the overall build process is often much 
longer, particularly when a lot of work is being done in a particular 
component. If work is being done in a particular component, only that library 
need be rebuilt (and then the overall image relinked). Currently, since there 
is no such modularization, all source files must be considered at build time. 
Interestingly enough, there is such modularization in the source code layout; 
that modularization just isn't utilized at the compiler level.

Precompiled headers exist on both Windows and Linux. For Linux, you can refer 
to 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.gnu.org%2Fonlinedocs%2Fgcc%2FPrecompiled-Headers.html=02%7C01%7CJeff.Coffler%40microsoft.com%7C063d6dbc04304614f82f08d4550fc22d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636226973566412987=cacmJfK7ofxwx%2B663qCWVw3QzDdg61Eqps7ZV1Trxd0%3D=0.
 Straight from the GNU CC documentation: "The time the compiler takes to 
process these header files over and over again can account for nearly all of 
the time required to build the project."

In my prior use of precompiled headers, each C or C++ file generally took about 
4 seconds to compile. After switching to precompiled headers, the

Re: Re : Re: protbuf to json not compatible

2017-03-27 Thread Benjamin Mahler 
As a note, if you just need the protobufs you should be able to run protoc
against the published protos for the version you're building against
without needing to pull in the 'mesos.native' package. The 'mesos.native'
package provides scheduler and executor implementations (via talking to the
C++ mesos libs), and it sounds like you just need the .proto files?

I'm curious how other projects deal with "publishing" protobuf, whether
they publish packages compiled by "protoc" per language or just publish the
.protos. Seems like the latter is the more reasonable one for projects to
do.

On Sat, Mar 25, 2017 at 12:40 AM, Olivier Sallou 
wrote:

>
> - Benjamin Mahler  a écrit :
> > James, I'm curious, do you know specifically what the incompatibility is?
> >
> > Olivier, if you're dealing with protobuf already and trying to send it to
> > mesos, there's no need to use JSON. Unless you have a requirement to do
> so?
>
> I can manage json, this is fine.
> Sending protobuf mean sending whole accept message as protobuf, not task
> definition only. But for this I need mesos.native python package, and i
> want to avoid this. So i will switch to full json.
>
> Olivier
>
>
>
> > There are some outstanding issues with our JSON<->Protobuf conversion,
> > specifically we currently are inconsistent from proto3 when it comes to
> the
> > int(32|64), fixed(32|64), uint(32|64) handling, for one (we don't allow
> > strings on the input side (tomek is addressing that), and we don't use
> > strings on the output side).
> >
> > On Fri, Mar 24, 2017 at 12:44 AM, Olivier Sallou <
> olivier.sal...@irisa.fr>
> > wrote:
> >
> > >
> > >
> > > On 03/24/2017 04:02 AM, James Peach wrote:
> > > >> On Mar 23, 2017, at 7:58 PM, James Peach  wrote:
> > > >>
> > > >>> On Mar 23, 2017, at 1:54 AM, Olivier Sallou <
> olivier.sal...@irisa.fr>
> > > wrote:
> > > >>>
> > > >>> Hi,
> > > >>>
> > > >>> when transforming a protobug message to json with MessageToJson,
> the
> > > >>> json is not compatible with the json format expected by Mesos
> master.
> > > >> This is because you generated the protobuf bindings with proto3
> > > compiler. AFAICT they made an incompatible change to the JSON wire
> format.
> > > This bites you when using the jsonpb Go package, for example. I ended
> up
> > > post-processing the generated Go code to correct the field names.
> > > > Sorry I forgot to mention that the other workaround is to generate
> the
> > > protobuf bindings with the proto2 compiler.
> > > Thanks
> > > My first workaround is to generate json directly, not a big deal in my
> > > case, but I wanted to understand.
> > >
> > > Olivier
> > > >
> > > > J
> > >
> > > --
> > > Olivier Sallou
> > > IRISA / University of Rennes 1
> > > Campus de Beaulieu, 35000 RENNES - FRANCE
> > > Tel: 02.99.84.71.95
> > >
> > > gpg key id: 4096R/326D8438  (keyring.debian.org)
> > > Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
> > >
> > >
>
>