+1 So, folks will have to come to the repo and pull the code directly if they would like to run the benchmarks as it will not be packed in the artifacts.
Let's just go ahead and update the pom accordingly (we already have pom-benchmarks.xml that will build for benchmarking). On Sat, Jul 23, 2016 at 1:47 AM, Suneel Marthi <[email protected]> wrote: > +1 > > On Sat, Jul 23, 2016 at 12:01 AM, Jacob Wilder < > [email protected]> wrote: > > > I'll put forward my vote that we stick with JMH and separate out any > > benchmarks that depend on it if/when we distribute binary artifacts. JMH > > seems to be better maintained (starting from the fact that it's possible > to > > use it without needing to modify what you get from the source repo) and > it > > seems to be easier to use. > > > > If nobody has any qualms with it I will close my pull request and we'll > > keep using JMH. > > > > — > > Jacob WIlder > > > > On Thu, Jul 21, 2016 at 7:23 AM, Tim Ellison <[email protected]> > > wrote: > > > > > On 21/07/16 06:47, Suneel Marthi wrote: > > > > JMH license is not compatible with Apache and hence this thread. > > > > But if JMH is only being used for tests and if we don't package the > > tests > > > > into project artifacts on Apache mirrors, I am thinking we shuld be > > fine > > > to > > > > leave JMH as is. > > > > > > It gets a bit tricky, so here goes: > > > - the Pirk tests are Apache licensed, so obviously they are ok to > > > package and redistribute. > > > > > > - the JMH library itself is licensed as GPLv2+classpath exception. > > > This cannot be redistributed by Pirk as all Apache software must be > > > distributed under the Apache License 2.0. > > > > > > - it is ok for Pirk to have a dependency on tools (such as gpg2 for > > > artefact signing), platforms (such as Java), and /optional/ runtime > > > plug-ins that have an 'incompatible' license. These are provided by > the > > > end-user, not the Pirk project. > > > > > > - where Pirk chooses to bundle the output from tools, you have to > check > > > the terms that come with each tool to know if there are any > restrictions. > > > > > > > I defer to Tim or Joe to make a call here. > > > > > > I see no problem with continuing to use JMH for benchmarking tests. > > > > > > Like all Apache projects, our primary distribution will be source code. > > > When it comes to creating a convenience binary artefact we will audit > > > it to check there it does not include incompatibly licensed binaries. > > > > > > HTH > > > > > > Regards, > > > Tim > > > > > > > On Wed, Jul 20, 2016 at 6:00 PM, Jacob Wilder < > > > > [email protected]> wrote: > > > > > > > >> Returning to the JMH discussion, I spent a short period of time > > > integrating > > > >> Google Caliper and much more time figuring out how to run it. > > > >> > > > >> The answer is to use: `java -cp > > > >> caliper-1.0-SNAPSHOT-all.jar:pirk-0.0.1-SNAPSHOT-exe.jar:. > > > >> com.google.caliper.runner.CaliperMain > > > >> org.apache.pirk.benchmark.PaillierBenchmark` > > > >> > > > >> You need to provide your own caliper-1.0-SNAPSHOT-all.jar. To get > > > Caliper > > > >> to compile I had to change their pom file to bump both > > com.google.dagger > > > >> dependencies from 2.1-SNAPSHOT to 2.4. I don't like that I have > only a > > > >> vague idea why this makes a difference. > > > >> > > > >> At the same time, I've looked around and it looks like we are fine > to > > > >> continue using JMH (which, if I am to extrapolate off of my > experience > > > in > > > >> the past two days) is easier to use. There are other Apache projects > > > that > > > >> continue to use JMH under their classpath exemption (including > > > dropwizard's > > > >> Metrics). Can we articulate a reason why we must switch and why it > > > would be > > > >> better to? I'm totally fine with cancelling this PR. > > > >> > > > >> — > > > >> Jacob WIlder > > > >> > > > >> On Mon, Jul 18, 2016 at 11:34 AM, Suneel Marthi <[email protected] > > > > > >> wrote: > > > >> > > > >>> ....and we shuld move the Beam and Streaming integration > discussions > > > to > > > >> a > > > >>> different thread. > > > >>> > > > >>> On Mon, Jul 18, 2016 at 11:32 AM, Suneel Marthi < > [email protected]> > > > >>> wrote: > > > >>> > > > >>>> > > > >>>> > > > >>>> On Mon, Jul 18, 2016 at 11:03 AM, Chris Harris < > > > >>> [email protected]> > > > >>>> wrote: > > > >>>> > > > >>>>> I agree separating the benchmarking code out might be a good > idea. > > > >>> Maybe > > > >>>>> we want to make this a multi-module project and the benchmarking > > be a > > > >>>>> submodule? Won't JMH still need to included via the child pom in > > the > > > >>>>> benchmarking though; I don't think it can be marked as "provided" > > or > > > >>>>> anything, right? Glad to hear if you had a different thought on > how > > > to > > > >>> go > > > >>>>> with this though. > > > >>>>> > > > >>>>> Maybe it's good to create submodules for all the adapters (eg. > > Spark, > > > >>> MR, > > > >>>>> Flink, Storm etc...) too? Then we won't have just one jar > needing > > to > > > >>>>> carry > > > >>>>> around all those dependencies that aren't used (and reduce > > potential > > > >>>>> version conflicts). I don't know if this would shake things up > too > > > >> much > > > >>>>> to > > > >>>>> do this restructuring now, or if it is better to do now before > we > > > >> move > > > >>>>> too > > > >>>>> far down the line. We'd have to look at what's Pirk "core" vs > not, > > > >> but > > > >>> I > > > >>>>> don't think that's too tough to discern right now. Most of the > > > >> adapter > > > >>>>> code is in org.apache.pirk.responder. > > > >>>>> > > > >>>> > > > >>>> I have a suggestion here. Its not a good idea to be having > > submodules > > > >> for > > > >>>> every streaming engine that's out there. In the long run, its > gonna > > be > > > >> a > > > >>>> maintenance and compatibility nightmare with every release of > Flink, > > > >>> Spark > > > >>>> etc... > > > >>>> > > > >>>> I am guilty of having wasted the last 2 yrs of my life (and fellow > > > >>>> committers time) in adding support for Spark, Flink, H2O on the > > Apache > > > >>>> Mahout project as distributed backend engines. > > > >>>> > > > >>>> I wish now that Apache Beam was around in 2014, I would have to > then > > > >> just > > > >>>> support Beam and be abstracted away from all of the other > Streaming > > > >>>> frameworks. > > > >>>> > > > >>>> As a new project, I would suggest that we look into integrating > with > > > >> Beam > > > >>>> and keep the codebase lean and not bloat the project with all the > > > >>> different > > > >>>> Streaming engines. > > > >>>> > > > >>>> Beam is a unified Batch + Streaming processing engine from google. > > All > > > >> of > > > >>>> the other Streaming engines like Flink, Spark, Apex, Storm etc... > > are > > > >> now > > > >>>> vying to provide native runners that can support Beam. This kind'a > > > >> makes > > > >>>> Beam an abstraction over every other streaming framework. > > > >>>> > > > >>>> As an application developer, I would write my jobs against the > Beam > > > API > > > >>>> and have the option of being able to execute the same as a Spark > > batch > > > >>> job, > > > >>>> Flink streaming/batch job etc. This completely shields the > developer > > > >> from > > > >>>> having to support the plethora of streaming engines out there. > > > >>>> > > > >>>> On the Mahout project, we built a complete logical layer for > coding > > up > > > >> ML > > > >>>> algorithms, and a physical layer that translates the logical plan > to > > > >> run > > > >>> on > > > >>>> different execution engines like Spark, Flink. There was no Beam > > then > > > >>> when > > > >>>> we started that effort in early 2014. > > > >>>> I would do it a different way today given Beam. > > > >>>> > > > >>>> It would be real cool by way of publicity and building a community > > for > > > >>>> Pirk, if Pirk were to be one of the few projects out there that > > > support > > > >>>> Beam. > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > >>>>> Regards, > > > >>>>> Chris > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> On Mon, Jul 18, 2016 at 6:25 AM, Tim Ellison < > > [email protected]> > > > >>>>> wrote: > > > >>>>> > > > >>>>>> On 17/07/16 16:57, Ellison Anne Williams wrote: > > > >>>>>>> Suneel -- Thanks for creating the JIRA issue and pointing out > the > > > >>>>>> licensing > > > >>>>>>> problems. I see that JMH is under the GNU GPL2 ( > > > >>>>>>> http://openjdk.java.net/legal/) which is not compatible with > the > > > >>>>> Apache > > > >>>>>>> license (http://www.apache.org/legal/resolved.html). > > > >>>>>>> > > > >>>>>>> It appears that Flink just removed the benchmarking code > instead > > > >> of > > > >>>>>>> re-porting it to another option. > > > >>>>>>> > > > >>>>>>> I would like us to port it to another license-compatible > > > >>> benchmarking > > > >>>>>>> framework such as Google Caliper (or something similar) instead > > of > > > >>>>>> removing > > > >>>>>>> the code as the benchmarking is important for encryption > > > >>> optimization. > > > >>>>>>> > > > >>>>>>> Thoughts? > > > >>>>>> > > > >>>>>> JMH is GPLv2 with classpath exception [1], which means that it > > > >> cannot > > > >>> be > > > >>>>>> distributed as part of the ALv2 licensed works (Pirk), but there > > is > > > >> no > > > >>>>>> problem with using this library as a tool / dependency at > runtime. > > > >>>>>> Afterall, there is no Java runtime that allows for > redistribution > > > >>> under > > > >>>>>> ALv2 either! > > > >>>>>> > > > >>>>>> That said, running the "mvn package" target *does* put JMH > > generated > > > >>>>>> code into the resulting pirk-0.0.1-SNAPSHOT.jar -- which then > begs > > > >> the > > > >>>>>> question why Pirk is putting test code into the library? > > > >>>>>> > > > >>>>>> So if the benchmark code were not part of the delivery then you > > can > > > >>>>>> continue to use JMH, but if that is there for a reason then we > > would > > > >>>>>> have to switch to a compatible licensed framework. > > > >>>>>> > > > >>>>>> [1] > > > >>> > http://hg.openjdk.java.net/code-tools/jmh/file/c050a47b2b37/LICENSE > > > >>>>>> > > > >>>>>> Regards, > > > >>>>>> Tim > > > >>>>>> > > > >>>>> > > > >>>> > > > >>>> > > > >>> > > > >> > > > > > > > > > >
