What is the disadvantage of using Java's built in Serialization? Tyler
On Wed, Nov 5, 2014 at 10:00 PM, Michael Starch <[email protected]> wrote: > I need to serialize a JobSpec and children (Job and JobInput) to a byte[]. > Java can do this automatically by marking all three as Serializable. > > The work around is to manually serialize to a private inner struct and back > out again. The inner class will have members for each member in the > JobSpec and children. Java can the auto-serialize that without changing > the other three. > > It is ugly, and essentially a reimplementation of those three > classes....but it is entirely self-contained. > > Michael > On Nov 5, 2014 6:45 PM, "Chris Mattmann" <[email protected]> wrote: > > > Hey Mike, > > > > Hmm, what’s the work around just so I know > > what we’re trading against? > > > > Cheers, > > Chris > > > > ------------------------ > > Chris Mattmann > > [email protected] > > > > > > > > > > -----Original Message----- > > From: Michael Starch <[email protected]> > > Reply-To: <[email protected]> > > Date: Wednesday, November 5, 2014 at 6:31 PM > > To: <[email protected]> > > Subject: Re: Extra Compiler Tools > > > > >That is basically what I did. Regardless, protobuff proves to be > overkill. > > > > > >If I mark those classes as serializable, the correct solution is 2 lines > > >of > > >code. (protobuff was like 20). Wrote a test case, and it works > > >perfectly. > > > > > >If I cannot make JobSpec Job and JonInput implement Serializable then > the > > >work around is simple too. > > > > > >What do you think? Should I mark them as Serializable, or use a > > >work-around. Either is a better solution than protobuff. > > > > > >Michael > > >On Nov 5, 2014 4:44 PM, "Chris Mattmann" <[email protected]> > > wrote: > > > > > >> Mike, have you looked at this yet? > > >> > > >> > > >> > > > http://techtraits.com/build%20management/maven/2011/09/09/compiling-proto > > >>co > > >> l-buffers-from-maven/ > > >> > > >> > > >> I’m going to play with it tonight and see if > > >> I can help here. Do you have some files I can test > > >> with? Can you attach them to JIRA or dropbox them to me > > >> so I can scope? > > >> > > >> Cheers, > > >> Chris > > >> > > >> ------------------------ > > >> Chris Mattmann > > >> [email protected] > > >> > > >> > > >> > > >> > > >> -----Original Message----- > > >> From: Michael Starch <[email protected]> > > >> Reply-To: <[email protected]> > > >> Date: Wednesday, November 5, 2014 at 5:37 PM > > >> To: <[email protected]> > > >> Subject: Re: Extra Compiler Tools > > >> > > >> >Ok....time for an audible. Protoc needs to be built from source, no > > >> >binary > > >> >distributions available. Thus I am going to purge proto-buffers from > > >>the > > >> >new code and be done with it. > > >> > > > >> >Any problem making the following classes/interfaces implement > > >> >java.io.Serializable: > > >> > > > >> >JobSpec > > >> >Job > > >> >JobInput > > >> > > > >> >Doing so would allow apache and native java serialization and thus we > > >> >wouldn't need something like proto-buffers. > > >> > > > >> >-Michael > > >> >Thanks Mike +1 > > >> > > > >> >------------------------ > > >> >Chris Mattmann > > >> >[email protected] > > >> > > > >> > > > >> > > > >> > > > >> >-----Original Message----- > > >> >From: Michael Starch <[email protected]> > > >> >Reply-To: <[email protected]> > > >> >Date: Wednesday, November 5, 2014 at 12:31 PM > > >> >To: <[email protected]> > > >> >Subject: Re: Extra Compiler Tools > > >> > > > >> >>Looks like you followed the same reasoning chain that I did. Yes, I > > >>came > > >> >>to the same conclusion that ant-build was best. > > >> >> > > >> >>I wasn't sure how to download protoc, but you just answered > > >>that....so I > > >> >>think this is a great solution! > > >> >> > > >> >>Thanks, > > >> >> > > >> >>Michael > > >> >> > > >> >> > > >> >>On Wed, Nov 5, 2014 at 10:23 AM, Chris Mattmann > > >> >><[email protected]> > > >> >>wrote: > > >> >> > > >> >>> Hi Mike, > > >> >>> > > >> >>> Thanks for flushing this out. > > >> >>> > > >> >>> My thoughts on the below: > > >> >>> > > >> >>> > > >> >>> -----Original Message----- > > >> >>> From: Michael Starch <[email protected]> > > >> >>> Reply-To: <[email protected]> > > >> >>> Date: Wednesday, November 5, 2014 at 12:12 PM > > >> >>> To: <[email protected]> > > >> >>> Subject: Re: Extra Compiler Tools > > >> >>> > > >> >>> >I tried this approach. The plugin requires a path to the "protoc" > > >>tool > > >> >>>and > > >> >>> >thus a working installation. This is what prompted the > discussion. > > >> >>> > > >> >>> Ah - no worries, what you could do is: > > >> >>> > > >> >>> 1. only enable to plugin if -Pwith-mesos is enabled; and > > >> >>> > > >> >>> > > > >> >>> >Running the plugin under a profile works. > > >> >>> > > >> >>> Yep. > > >> >>> > > >> >>> > However, not running the plugin > > >> >>> >causes compile errors in dependant code. Excluding this code > > >>except > > >> >>> >within > > >> >>> >the profile doesn't seem to work, and is considered by some to be > > >>bad > > >> >>>form > > >> >>> >because there is nothing inside the jar file that notes which > > >>profiles > > >> >>> >were > > >> >>> >used to compile. > > >> >>> > > >> >>> Got it. Suggestion here would be: > > >> >>> > > >> >>> 2. create a new module, cas-resource-mesos, and inside of that > > >>module, > > >> >>> take one of the following approaches, assuming the module is > > >>activated > > >> >>> when -Pwith-mesos is enabled: > > >> >>> > > >> >>> 2a. Maven Antrun like so (in this old example): > > >> >>> > > >> >>> > > >> > > http://stackoverflow.com/questions/1578456/integrate-protocol-buffers-in > > >> >>>t > > >> >>>o- > > >> >>> maven2-build > > >> >>> > > >> >>> (pro: more flexibility in case protoc isn¹t there; to fail on > > >>error; to > > >> >>> only compile if > > >> >>> protoc is available > > >> >>> > > >> >>> 2b. Maven protobuf plugin > > >> >>> http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html > > >> >>> > > >> >>> Here¹s how to enable a module with a profile: > > >> >>> > > >> >>> > > >> >>> > > >> > > http://blog.soebes.de/blog/2013/11/09/why-is-it-bad-to-activate-slash-de > > >> >>>a > > >> >>>ct > > >> >>> ive-modules-by-profiles-in-maven/ > > >> >>> > > >> >>> > > >> >>> It seems like that is a bad idea though, based on that discussion. > > >> >>> > > >> >>> So, here¹s another option: > > >> >>> > > >> >>> 1. Inside of cas-resource (no special new module or anything else) > > >> >>> 2. include some custom Ant magic via a build.xml file and the > Maven > > >> >>> AntRun plugin: > > >> >>> 2a. test if protoc is on the system path, and if not, download > it, > > >> >>>e.g., > > >> >>> into the target directory (gets deleted on clean) > > >> >>> 2b. call protoc and compile after 2a > > >> >>> > > >> >>> I would suggest this solution as I think it¹s the most robust and > > >> >>>ensures > > >> >>> we always have a cas-resource that includes mesos and compiled > > >> >>>correctly. > > >> >>> > > >> >>> Cheers, > > >> >>> Chris > > >> >>> > > >> >>> > > > >> >>> >Any ideas on how to continue? > > >> >>> > > > >> >>> >Michael > > >> >>> > On Nov 5, 2014 11:04 AM, "Chris Mattmann" > > >><[email protected]> > > >> >>> >wrote: > > >> >>> > > > >> >>> >> Hi Mike, > > >> >>> >> > > >> >>> >> Great discussion. It would be nice if there was > > >> >>> >> a protoc Maven plugin: > > >> >>> >> > > >> >>> >> http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html > > >> >>> >> > > >> >>> >> > > >> >>> >> Looks like there is. My suggestion: > > >> >>> >> > > >> >>> >> 1. use a Profile, something like -Pwith-mesos and > > >> >>> >> then when activated; > > >> >>> >> 2. call the above plugin if -Pwith-mesos is activated > > >> >>> >> in the resource manager > > >> >>> >> > > >> >>> >> Sound good? > > >> >>> >> > > >> >>> >> Cheers, > > >> >>> >> Chris > > >> >>> >> > > >> >>> >> ------------------------ > > >> >>> >> Chris Mattmann > > >> >>> >> [email protected] > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> -----Original Message----- > > >> >>> >> From: Michael Starch <[email protected]> > > >> >>> >> Reply-To: <[email protected]> > > >> >>> >> Date: Wednesday, November 5, 2014 at 11:46 AM > > >> >>> >> To: <[email protected]> > > >> >>> >> Subject: Extra Compiler Tools > > >> >>> >> > > >> >>> >> >All, > > >> >>> >> > > > >> >>> >> >I am trying to integrate apache-mesos with our resource > manager. > > >> >>> >>However, > > >> >>> >> >mesos uses a technology called "protobuff" from Google for > > >> >>> >> >marshaling/unmarshaling data. > > >> >>> >> > > > >> >>> >> >This requires running a tool called "protoc" to generate a > > >>source > > >> >>>file > > >> >>> >>in > > >> >>> >> >java. What is the best way to integrate this step into our > > >>build > > >> >>> >>process? > > >> >>> >> > > > >> >>> >> >Options I can conceive of: > > >> >>> >> > -Check in generated java file > > >> >>> >> > -Require "protoc" installation to build resource manager > > >> >>> >> > -Separate extra resource package into new module > > >> >>> >> > > > >> >>> >> >None of these ideas are very clean. > > >> >>> >> > > > >> >>> >> >Any other ideas? I tried setting up a profile to only compile > > >> >>>these > > >> >>> >> >sources when selected, but that turned out not to work. > > >> >>> >> > > > >> >>> >> >-Michael Starch > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> > > >> >>> > > >> >>> > > >> > > >> > > >> > > > > > > >
