+1 On Sat, Jul 23, 2016 at 12:01 AM, Jacob Wilder < [email protected]> wrote:
> I'll put forward my vote that we stick with JMH and separate out any > benchmarks that depend on it if/when we distribute binary artifacts. JMH > seems to be better maintained (starting from the fact that it's possible to > use it without needing to modify what you get from the source repo) and it > seems to be easier to use. > > If nobody has any qualms with it I will close my pull request and we'll > keep using JMH. > > — > Jacob WIlder > > On Thu, Jul 21, 2016 at 7:23 AM, Tim Ellison <[email protected]> > wrote: > > > On 21/07/16 06:47, Suneel Marthi wrote: > > > JMH license is not compatible with Apache and hence this thread. > > > But if JMH is only being used for tests and if we don't package the > tests > > > into project artifacts on Apache mirrors, I am thinking we shuld be > fine > > to > > > leave JMH as is. > > > > It gets a bit tricky, so here goes: > > - the Pirk tests are Apache licensed, so obviously they are ok to > > package and redistribute. > > > > - the JMH library itself is licensed as GPLv2+classpath exception. > > This cannot be redistributed by Pirk as all Apache software must be > > distributed under the Apache License 2.0. > > > > - it is ok for Pirk to have a dependency on tools (such as gpg2 for > > artefact signing), platforms (such as Java), and /optional/ runtime > > plug-ins that have an 'incompatible' license. These are provided by the > > end-user, not the Pirk project. > > > > - where Pirk chooses to bundle the output from tools, you have to check > > the terms that come with each tool to know if there are any restrictions. > > > > > I defer to Tim or Joe to make a call here. > > > > I see no problem with continuing to use JMH for benchmarking tests. > > > > Like all Apache projects, our primary distribution will be source code. > > When it comes to creating a convenience binary artefact we will audit > > it to check there it does not include incompatibly licensed binaries. > > > > HTH > > > > Regards, > > Tim > > > > > On Wed, Jul 20, 2016 at 6:00 PM, Jacob Wilder < > > > [email protected]> wrote: > > > > > >> Returning to the JMH discussion, I spent a short period of time > > integrating > > >> Google Caliper and much more time figuring out how to run it. > > >> > > >> The answer is to use: `java -cp > > >> caliper-1.0-SNAPSHOT-all.jar:pirk-0.0.1-SNAPSHOT-exe.jar:. > > >> com.google.caliper.runner.CaliperMain > > >> org.apache.pirk.benchmark.PaillierBenchmark` > > >> > > >> You need to provide your own caliper-1.0-SNAPSHOT-all.jar. To get > > Caliper > > >> to compile I had to change their pom file to bump both > com.google.dagger > > >> dependencies from 2.1-SNAPSHOT to 2.4. I don't like that I have only a > > >> vague idea why this makes a difference. > > >> > > >> At the same time, I've looked around and it looks like we are fine to > > >> continue using JMH (which, if I am to extrapolate off of my experience > > in > > >> the past two days) is easier to use. There are other Apache projects > > that > > >> continue to use JMH under their classpath exemption (including > > dropwizard's > > >> Metrics). Can we articulate a reason why we must switch and why it > > would be > > >> better to? I'm totally fine with cancelling this PR. > > >> > > >> — > > >> Jacob WIlder > > >> > > >> On Mon, Jul 18, 2016 at 11:34 AM, Suneel Marthi <[email protected]> > > >> wrote: > > >> > > >>> ....and we shuld move the Beam and Streaming integration discussions > > to > > >> a > > >>> different thread. > > >>> > > >>> On Mon, Jul 18, 2016 at 11:32 AM, Suneel Marthi <[email protected]> > > >>> wrote: > > >>> > > >>>> > > >>>> > > >>>> On Mon, Jul 18, 2016 at 11:03 AM, Chris Harris < > > >>> [email protected]> > > >>>> wrote: > > >>>> > > >>>>> I agree separating the benchmarking code out might be a good idea. > > >>> Maybe > > >>>>> we want to make this a multi-module project and the benchmarking > be a > > >>>>> submodule? Won't JMH still need to included via the child pom in > the > > >>>>> benchmarking though; I don't think it can be marked as "provided" > or > > >>>>> anything, right? Glad to hear if you had a different thought on how > > to > > >>> go > > >>>>> with this though. > > >>>>> > > >>>>> Maybe it's good to create submodules for all the adapters (eg. > Spark, > > >>> MR, > > >>>>> Flink, Storm etc...) too? Then we won't have just one jar needing > to > > >>>>> carry > > >>>>> around all those dependencies that aren't used (and reduce > potential > > >>>>> version conflicts). I don't know if this would shake things up too > > >> much > > >>>>> to > > >>>>> do this restructuring now, or if it is better to do now before we > > >> move > > >>>>> too > > >>>>> far down the line. We'd have to look at what's Pirk "core" vs not, > > >> but > > >>> I > > >>>>> don't think that's too tough to discern right now. Most of the > > >> adapter > > >>>>> code is in org.apache.pirk.responder. > > >>>>> > > >>>> > > >>>> I have a suggestion here. Its not a good idea to be having > submodules > > >> for > > >>>> every streaming engine that's out there. In the long run, its gonna > be > > >> a > > >>>> maintenance and compatibility nightmare with every release of Flink, > > >>> Spark > > >>>> etc... > > >>>> > > >>>> I am guilty of having wasted the last 2 yrs of my life (and fellow > > >>>> committers time) in adding support for Spark, Flink, H2O on the > Apache > > >>>> Mahout project as distributed backend engines. > > >>>> > > >>>> I wish now that Apache Beam was around in 2014, I would have to then > > >> just > > >>>> support Beam and be abstracted away from all of the other Streaming > > >>>> frameworks. > > >>>> > > >>>> As a new project, I would suggest that we look into integrating with > > >> Beam > > >>>> and keep the codebase lean and not bloat the project with all the > > >>> different > > >>>> Streaming engines. > > >>>> > > >>>> Beam is a unified Batch + Streaming processing engine from google. > All > > >> of > > >>>> the other Streaming engines like Flink, Spark, Apex, Storm etc... > are > > >> now > > >>>> vying to provide native runners that can support Beam. This kind'a > > >> makes > > >>>> Beam an abstraction over every other streaming framework. > > >>>> > > >>>> As an application developer, I would write my jobs against the Beam > > API > > >>>> and have the option of being able to execute the same as a Spark > batch > > >>> job, > > >>>> Flink streaming/batch job etc. This completely shields the developer > > >> from > > >>>> having to support the plethora of streaming engines out there. > > >>>> > > >>>> On the Mahout project, we built a complete logical layer for coding > up > > >> ML > > >>>> algorithms, and a physical layer that translates the logical plan to > > >> run > > >>> on > > >>>> different execution engines like Spark, Flink. There was no Beam > then > > >>> when > > >>>> we started that effort in early 2014. > > >>>> I would do it a different way today given Beam. > > >>>> > > >>>> It would be real cool by way of publicity and building a community > for > > >>>> Pirk, if Pirk were to be one of the few projects out there that > > support > > >>>> Beam. > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>> Regards, > > >>>>> Chris > > >>>>> > > >>>>> > > >>>>> > > >>>>> On Mon, Jul 18, 2016 at 6:25 AM, Tim Ellison < > [email protected]> > > >>>>> wrote: > > >>>>> > > >>>>>> On 17/07/16 16:57, Ellison Anne Williams wrote: > > >>>>>>> Suneel -- Thanks for creating the JIRA issue and pointing out the > > >>>>>> licensing > > >>>>>>> problems. I see that JMH is under the GNU GPL2 ( > > >>>>>>> http://openjdk.java.net/legal/) which is not compatible with the > > >>>>> Apache > > >>>>>>> license (http://www.apache.org/legal/resolved.html). > > >>>>>>> > > >>>>>>> It appears that Flink just removed the benchmarking code instead > > >> of > > >>>>>>> re-porting it to another option. > > >>>>>>> > > >>>>>>> I would like us to port it to another license-compatible > > >>> benchmarking > > >>>>>>> framework such as Google Caliper (or something similar) instead > of > > >>>>>> removing > > >>>>>>> the code as the benchmarking is important for encryption > > >>> optimization. > > >>>>>>> > > >>>>>>> Thoughts? > > >>>>>> > > >>>>>> JMH is GPLv2 with classpath exception [1], which means that it > > >> cannot > > >>> be > > >>>>>> distributed as part of the ALv2 licensed works (Pirk), but there > is > > >> no > > >>>>>> problem with using this library as a tool / dependency at runtime. > > >>>>>> Afterall, there is no Java runtime that allows for redistribution > > >>> under > > >>>>>> ALv2 either! > > >>>>>> > > >>>>>> That said, running the "mvn package" target *does* put JMH > generated > > >>>>>> code into the resulting pirk-0.0.1-SNAPSHOT.jar -- which then begs > > >> the > > >>>>>> question why Pirk is putting test code into the library? > > >>>>>> > > >>>>>> So if the benchmark code were not part of the delivery then you > can > > >>>>>> continue to use JMH, but if that is there for a reason then we > would > > >>>>>> have to switch to a compatible licensed framework. > > >>>>>> > > >>>>>> [1] > > >>> http://hg.openjdk.java.net/code-tools/jmh/file/c050a47b2b37/LICENSE > > >>>>>> > > >>>>>> Regards, > > >>>>>> Tim > > >>>>>> > > >>>>> > > >>>> > > >>>> > > >>> > > >> > > > > > >
