There were some issues in extracting the incorrect L&N files from the executable jar and re-jarring so as to be fully functioning in a cluster setting (extracting and fixing the L&N files was easy, getting it to fully function on the cluster after re-jarring was not). Thus, I've decided to stick with the source only release for now as releasing the binary artifacts is not necessary. I will publish the URL and send the voting email in a second. If folks object to the source only release, please let me know.
On Sat, Aug 20, 2016 at 1:38 PM, Ellison Anne Williams < [email protected]> wrote: > Agreed - let's get through the release and then start a separate thread to > discuss the submodule breakout > > On Sat, Aug 20, 2016 at 1:13 PM, Suneel Marthi <[email protected]> > wrote: > >> For this release, lets add the README as has been suggested. >> >> >> --------------- >> >> Breaking out the project into submodules would be a much more involved >> discussion that should also include as to how we want to break the project >> into individual artifacts - like for Responder, Querier, Spark Responder, >> Storm Responder, Benchmarks, etc... >> >> For reference, projects like Mahout, Flink etc that have several >> submodules >> produce different jar files for each of them and there's one tar or zip >> that would package all of the jars. >> >> See http://www.apache.org/dist/mahout/0.12.2/ >> >> That would be something we need to discuss following this release. On >> projects like Flink, Kafka major refactoring or new feature design are >> usually discussed via shared google documents wherein others could comment >> or propose changes. Kafka calls these feature changes as 'KLIP' and Flink >> calls them 'FLIP'. >> >> We may want to start something similar for Pirk. >> >> On Sat, Aug 20, 2016 at 12:49 PM, Ellison Anne Williams < >> [email protected]> wrote: >> >> > Will do thanks. >> > >> > Just to be clear -- the META-INF dir with the binary L&N files will be >> > removed and the binary-related L&N files will be moved to as assembly >> > directory (or something similar) when we break out the submodules. In >> the >> > meantime the META-INF (with binary L&N files) as in the PR will be >> merged >> > after adding the README as suggested. Folks will be responsible for >> > updating the binary L&N files as they add dependencies. >> > >> > On Sat, Aug 20, 2016 at 12:41 PM, Suneel Marthi < >> [email protected]> >> > wrote: >> > >> > > +1 >> > > >> > > On Sat, Aug 20, 2016 at 12:40 PM, Tim Ellison <[email protected]> >> > > wrote: >> > > >> > > > On 20 August 2016 at 17:04, Ellison Anne Williams < >> > > > [email protected]> >> > > > wrote: >> > > > >> > > > > Yes, if that's acceptable, then I can go that route. >> > > > > >> > > > > I can allow the artifacts to be generated and pushed to Nexus, I >> can >> > > > remove >> > > > > the binary artifacts, fix the L&N files, rehash/sign, and upload. >> > > Then, I >> > > > > can close the repo (which will run through all signing >> verifications) >> > > and >> > > > > provide the URL for voting. Once we get the submodules in place, >> the >> > > L&N >> > > > > file placement will be taken care of automatically for binary >> > artifacts >> > > > and >> > > > > the manual intervention won't be necessary. >> > > > > >> > > > > Would it be OK to go ahead and merge the PR with the binary L&N >> files >> > > in >> > > > > the bin-license-notice dir in META-INF? That way, folks can be >> > > > responsible >> > > > > for keeping them updated as they add any new dependencies. >> > > > > >> > > > >> > > > That works for me. Unless other mentors object, I say we go with >> this >> > > > plan. >> > > > >> > > > To avoid any confusion you may want to consider creating a brief >> readme >> > > > file in the META-INF to explain that the L&N files in there only >> relate >> > > to >> > > > the exe-jar. >> > > > >> > > > Regards, >> > > > Tim >> > > > >> > > > >> > > > >> > > > > On Sat, Aug 20, 2016 at 11:51 AM, Tim Ellison < >> [email protected] >> > > >> > > > > wrote: >> > > > > >> > > > > > On 20 August 2016 at 16:23, Ellison Anne Williams < >> > > > > > [email protected]> >> > > > > > wrote: >> > > > > > >> > > > > > > Thanks Tim - yes, I've been following that thread with >> interest. >> > To >> > > > > speak >> > > > > > > to the documentation for Pirk releases, I have been working >> on a >> > > page >> > > > > for >> > > > > > > the website documenting our release process (and the gotchas) >> > > > > > step-by-step. >> > > > > > > Once we complete a successful release vote internally, I will >> put >> > > > forth >> > > > > > the >> > > > > > > page for comment. I agree with the discussion on the >> > > > general@incubator >> > > > > > > thread that the release process for a project needs to be >> > > documented >> > > > > such >> > > > > > > that anyone can walk into the project and complete a release >> - it >> > > > > > shouldn't >> > > > > > > be a mystical endeavor. >> > > > > > >> > > > > > >> > > > > > I don't know precisely what steps you are following at the >> moment, >> > > but >> > > > if >> > > > > > there is an opportunity to manually intervene and fix-up the >> binary >> > > JAR >> > > > > > licenses before the artefacts are hashed / signed / uploaded >> then >> > > that >> > > > > > could simply be a "documented step" until the scripts handle it >> > all. >> > > > > > >> > > > > > ... or even produce the JAR, then >> > > > > > - delete the exe-jar's sig/hash >> > > > > > - rearrange the L&N files >> > > > > > - re-sign/hash >> > > > > > >> > > > > > Provided it is a repeatable process for the binaries, I think >> that >> > > > would >> > > > > be >> > > > > > ok. >> > > > > > >> > > > > > WDYT? >> > > > > > >> > > > > > Regards, >> > > > > > Tim >> > > > > > >> > > > > > >> > > > > > > On Sat, Aug 20, 2016 at 10:58 AM, Tim Ellison < >> > > [email protected] >> > > > > >> > > > > > > wrote: >> > > > > > > >> > > > > > > > On 20 August 2016 at 04:52, Ellison Anne Williams < >> > > > > > > > [email protected]> >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > After spending a bit of time taking a look at the >> submodule >> > > > > > breakout, I >> > > > > > > > > would rather wait and break the project out into >> submodules >> > in >> > > a >> > > > > more >> > > > > > > > > deliberate manner (not as a first pass just for the binary >> > > > > > artifacts). >> > > > > > > > > >> > > > > > > > > As submodules seem to be required to get the L&N files for >> > > binary >> > > > > > > > > distributions up to par, I propose that we proceed with a >> > > source >> > > > > only >> > > > > > > > > release now. We will slate the submodules for our next >> > release >> > > > (or >> > > > > > so). >> > > > > > > > > >> > > > > > > > > We've certainly learned tons about the release process >> over >> > the >> > > > > last >> > > > > > > week >> > > > > > > > > and I think that it would be good to go ahead and exercise >> > the >> > > > full >> > > > > > > > process >> > > > > > > > > to completion. >> > > > > > > > > >> > > > > > > > > Unless anyone objects, I will try to get the source >> release >> > > > > artifacts >> > > > > > > up >> > > > > > > > > for voting over the weekend. >> > > > > > > > > >> > > > > > > > > Thanks! >> > > > > > > > > >> > > > > > > > >> > > > > > > > Sure. Thanks for your continued focus on this Ellison Anne. >> > > > > > > > >> > > > > > > > FYI there is a discussion underway on [email protected] >> > that >> > > > is >> > > > > > > > relevant to this thread... >> > > > > > > > >> > > > > > > > http://mail-archives.apache.org/mod_mbox/incubator- >> > > > > > general/201608.mbox/% >> > > > > > > > 3C5113b38e-e52b-0c16-eced-903e00fc4477%40apache.org%3E >> > > > > > > > >> > > > > > > > I mention it as I expect the Incubator PMC are likely to >> assess >> > > > > Pirk's >> > > > > > > > release by this criteria, given it will be fresh in people's >> > mind >> > > > > when >> > > > > > > our >> > > > > > > > vote is proposed. >> > > > > > > > >> > > > > > > > Regards, >> > > > > > > > Tim >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
