Hi folks,
so I was now able to build (including all tests) with Java 8 and 9 ...
currently trying 10 ...
Are there any objection that some maven dependencies get updated to more recent
versions? I mean ... the hbase-client you're using is more than 5 years old ...
Chris
Am 20.04.20, 00:29 schrieb "Andrew Musselman" <andrew.mussel...@gmail.com>:
No problem; would 10:00 a.m. Pacific next Friday the 24th work for you time
zone-wise?
On Sun, Apr 19, 2020 at 12:15 PM Christofer Dutz <christofer.d...@c-ware.de>
wrote:
> Sorry ...
>
> didn't see your response ... into Mahout too deep ;-)
> Guess we have to postpone this to sometime over the week.
>
> Chris
>
>
> Am 19.04.20, 19:56 schrieb "Andrew Musselman" <andrew.mussel...@gmail.com
> >:
>
> *what time
>
> On Sun, Apr 19, 2020 at 10:51 Andrew Musselman <
> andrew.mussel...@gmail.com>
> wrote:
>
> > I think it's safe to move to 1.8; yeah what tune is good for you?
> I'm in
> > Pacific time zone and am flexible this afternoon.
> >
> > Trevor you free?
> >
> > On Sun, Apr 19, 2020 at 09:37 Christofer Dutz <
> christofer.d...@c-ware.de>
> > wrote:
> >
> >> Yikes! ... well guess then I can't help you folks as it's almost
> >> impossible to get my hands on a 1.7 version.
> >>
> >> What's preventing you from going to 1.8+?
> >>
> >> Any your build says 1.8 and above:
> >> <requireJavaVersion>
> >> <version>[1.8,)</version>
> >> </requireJavaVersion>
> >>
> >> Regarding the artifacts ... would it be ok to have the maven
> artifacts
> >> using classifiers and perhaps the files in the libs and
> distribution to
> >> follow the typical scala sheme?
> >>
> >> Chris
> >>
> >> Am 19.04.20, 17:56 schrieb "Trevor Grant" <trevor.d.gr...@gmail.com
> >:
> >>
> >> Yea, we have a requirement on 1.7. we need to get it up to 8,
> but
> >> considered that a different issue.
> >>
> >> Maven throws warning, sbt breaks down entirely (when
importing)-
> >> hence why
> >> we were using a script to replace 2.11 w 2.12
> >>
> >> On Sun, Apr 19, 2020, 10:33 AM Christofer Dutz <
> >> christofer.d...@c-ware.de>
> >> wrote:
> >>
> >> > Well I have been compiling with various jdks from 8 to 14.
> However I
> >> > noticed that if I select a jdk and start the build that the
> Scala
> >> parts
> >> > Seefahrt to the version in my systems path. Which then
> results in
> >> all sorts
> >> > of major version problems.
> >> >
> >> > For the Scalatest plugin I managed to have it use the maven
> >> java.home...
> >> > The Scala compiler is being a little trickyer.
> >> >
> >> > Will continue this evening. Keep your fingers crossed.
> >> >
> >> > Also does maven totally not like variables in artifactids.
> >> >
> >> > Chris
> >> > ________________________________
> >> > Von: Trevor Grant <trevor.d.gr...@gmail.com>
> >> > Gesendet: Sonntag, 19. April 2020 15:41
> >> > An: Mahout Dev List <dev@mahout.apache.org>
> >> > Betreff: Re: Hi ... need some help?
> >> >
> >> > Hey Chris,
> >> >
> >> > First of all- thank you so much for your work so far.
> >> >
> >> > The build 100% currently works, the only issue is with
> releasing
> >> binaries.
> >> > Are you compiling w 1.7? Not really sure why artifact IDs are
> >> changing?
> >> > That's very strange.
> >> >
> >> > To answer your question regarding the community block- no, it
> is
> >> not as if
> >> > there is a company that pays us to work on Mahout and the
> rest is
> >> > community. What it means is that while we appreciate those
> code
> >> > contributions, no one still actively involved is maintaining
> them
> >> and we
> >> > advise people to use at their own risk. The refactor to a
> >> `community`
> >> > folder was meant to reflect the reality of the situation.
> >> >
> >> > The only test issues I was aware of were with Javadocs when
> trying
> >> to do a
> >> > release with Java 1.8+ those would fail due to formatting.
> >> >
> >> > I feel like you're hitting some unique issues, and maybe we
> could
> >> help you
> >> > out / there may exist simple solutions.
> >> >
> >> > Thank you again for your help!
> >> >
> >> > tg
> >> >
> >> >
> >> >
> >> >
> >> > On Sun, Apr 19, 2020 at 2:17 AM Christofer Dutz <
> >> christofer.d...@c-ware.de
> >> > >
> >> > wrote:
> >> >
> >> > > Hi folks,
> >> > >
> >> > > So yesterday I invested several hours in cleaning up your
> build.
> >> I got
> >> > > quite far, currently trying to get the tests to pass. For
> some
> >> > > Scala-Java-major-version problems, but working on fixing
> them.
> >> > >
> >> > > However sind things will be different. I hope that's OK.
> >> Especially with
> >> > > the artifact ids.
> >> > >
> >> > > Just as a question: does the build currently work at all?
> >> Especially in
> >> > > the community block I had to fix quite a few API changes
> where
> >> the code
> >> > in
> >> > > the blocks were for older versions of libraries.
> >> > >
> >> > > I hope I get some more tests to pass today.
> >> > >
> >> > > Chris
> >> > > ________________________________
> >> > > Von: Andrew Musselman <a...@apache.org>
> >> > > Gesendet: Donnerstag, 16. April 2020 20:35
> >> > > An: Mahout Dev List <dev@mahout.apache.org>
> >> > > Betreff: Re: Hi ... need some help?
> >> > >
> >> > > Looking forward to working on this with you; thanks again!
> >> > >
> >> > > On Thu, Apr 16, 2020 at 11:09 AM Christofer Dutz <
> >> > > christofer.d...@c-ware.de>
> >> > > wrote:
> >> > >
> >> > > > Hi Andrew,
> >> > > >
> >> > > > guess I'll start with the fork and contact you folks on
> slack.
> >> > > >
> >> > > > Chris
> >> > > >
> >> > > >
> >> > > >
> >> > > > Am 16.04.20, 19:43 schrieb "Andrew Musselman" <
> a...@apache.org>:
> >> > > >
> >> > > > Chris, thank you for your help..
> >> > > >
> >> > > > Yeah if you fork what's in master you can see what
> state
> >> it's in;
> >> > we
> >> > > > are in
> >> > > > the #mahout channel in tge-asf slack and this is also
> a
> >> fine way to
> >> > > > keep
> >> > > > track of discussion.
> >> > > >
> >> > > > We could file a JIRA ticket as well, however you
> prefer to
> >> work.
> >> > > >
> >> > > > Best
> >> > > > Andrew
> >> > > >
> >> > > > On Thu, Apr 16, 2020 at 06:59 Christofer Dutz <
> >> > > > christofer.d...@c-ware.de>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Trevor,
> >> > > > >
> >> > > > > ok ... first of all ... the Mahout PMC is defining
a
> >> "community
> >> > > > > maintained" library which is not maintained by the
> mahout
> >> PMC?!?!
> >> > > > > I thought at Apache everything is about Community
> over
> >> code. So
> >> > is
> >> > > a
> >> > > > > company driving the non-community stuff?
> >> > > > >
> >> > > > > But back to your build issues:
> >> > > > > I had a look and I too encountered these comments
> and
> >> remarks and
> >> > > > > sometimes patterns I recognized and could imagine
> why
> >> they were
> >> > > > created.
> >> > > > > Yes quite a bit of the build could be cleaned up
and
> >> simplified a
> >> > > > lot.
> >> > > > >
> >> > > > > So how about I create a fork and try to do a
> cleanup of
> >> the
> >> > build.
> >> > > > > Usually I also leave comments about what I do as I
> hope
> >> I'll not
> >> > be
> >> > > > the
> >> > > > > only one maintaining a build and documenting things
> helps
> >> people
> >> > > > feel more
> >> > > > > confident.
> >> > > > >
> >> > > > > However in some cases I will have questions ... so
> would
> >> someone
> >> > be
> >> > > > > available on Slack for quick questions?
> >> > > > >
> >> > > > > Usually switching to another build system does
> solve some
> >> > problems
> >> > > > ...
> >> > > > > mostly the reason to switch is that it solved the
> main
> >> problem
> >> > that
> >> > > > you are
> >> > > > > having with the old.
> >> > > > > However you usually notice too late that you get
> yourself
> >> a lot
> >> > of
> >> > > > new
> >> > > > > problems. I remember doing some contract work for
an
> >> insurance
> >> > > > company and
> >> > > > > they were totally down Maven-road but then had to
> build
> >> something
> >> > > > with SBT
> >> > > > > ... in the end I compiled the thing on my laptop,
> copied
> >> it to a
> >> > > USB
> >> > > > stick
> >> > > > > and told the people what was on the stick and that
> I'll be
> >> > having a
> >> > > > coffee
> >> > > > > and will be back in 30 minutes. When I came back
> the sick
> >> wasn't
> >> > at
> >> > > > the
> >> > > > > same place and the build problem was "solved" ;-)
> >> > > > >
> >> > > > > So I think it's quite good to stick to maven ...
> that is
> >> very
> >> > > > mature, you
> >> > > > > can do almost everything you want with it and it
> >> integrates
> >> > > > perfectly into
> >> > > > > the Apache infrastructure.
> >> > > > >
> >> > > > > But that's just my opinion.
> >> > > > >
> >> > > > > So if you want me to help, I'll be happy to be of
> >> assistance.
> >> > > > >
> >> > > > >
> >> > > > > Chris
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > Am 16.04.20, 15:28 schrieb "Trevor Grant" <
> >> > > trevor.d.gr...@gmail.com
> >> > > > >:
> >> > > > >
> >> > > > > Hey Christopher,
> >> > > > >
> >> > > > > I would agree with what Stevo outlined but add
> some
> >> more
> >> > > context
> >> > > > and a
> >> > > > > couple related JIRA issues.
> >> > > > >
> >> > > > > For 0.14.0 We did a big refactor and finally
> moved the
> >> > > MapReduce
> >> > > > based
> >> > > > > Mahout all into what we called "community/"
> that is
> >> community
> >> > > > > maintained,
> >> > > > > which is to say, we're not maintaining it
> anymore
> >> (sunset
> >> > > began I
> >> > > > > think in
> >> > > > > 2015).
> >> > > > >
> >> > > > > But all of our POMs were so huge and fat
because
> >> they'd been
> >> > > > layered up
> >> > > > > over the years by people coming and going and
> >> dropping in
> >> > > code. I
> >> > > > > wouldn't
> >> > > > > call these drive- bys, its just been over 10
> years
> >> and people
> >> > > > come and
> >> > > > > go.
> >> > > > > Such is the life of Apache Projects. So we had
a
> >> situation
> >> > > where
> >> > > > a lot
> >> > > > > of
> >> > > > > the old Map Reduce stuff and the POMs were
> considered
> >> > > > "old-magic" no
> >> > > > > one
> >> > > > > really knew how it was all tied together, but
we
> >> didn't want
> >> > to
> >> > > > mess
> >> > > > > with
> >> > > > > it for fear of breaking something in the "new"
> Mahout
> >> (aka
> >> > > > Samsar)
> >> > > > > which is
> >> > > > > the Scala/Spark based library that it is now*
> (to
> >> others in
> >> > the
> >> > > > > community:
> >> > > > > I know it runs on other engines, but for
> simplicity,
> >> I'm just
> >> > > > calling
> >> > > > > it
> >> > > > > "runs-on-spark").
> >> > > > >
> >> > > > > For 0.14.0 We decided to trim out as much of
> that
> >> which was
> >> > > > possible.
> >> > > > > We
> >> > > > > did some major liposuction on POMs, re
organized
> >> things, etc.
> >> > > > This was
> >> > > > > done
> >> > > > > by commenting out a section, then seeing if it
> would
> >> still
> >> > > > build. So
> >> > > > > the
> >> > > > > current release
> >> > > > > _does_ build. And aside for some CLI driver
> issues
> >> which are
> >> > > > outlined
> >> > > > > in
> >> > > > > [1], the project runs fairly smooth. (An SBT
> would
> >> probably
> >> > > > solve [1],
> >> > > > > I
> >> > > > > believe Pat Ferrel has made his own SBT script
> to
> >> compile
> >> > > > Mahout, which
> >> > > > > solved that problem for them).
> >> > > > >
> >> > > > > The issue we ran into with the releases (and
the
> >> reason I
> >> > think
> >> > > > you're
> >> > > > > here), is that we also somewhere along the line
> >> commented out
> >> > > > something
> >> > > > > that was important to the release process.
> Hence why
> >> 0.14.0
> >> > > > released
> >> > > > > source
> >> > > > > only.
> >> > > > >
> >> > > > > Since 2008, there has been a lot of great work
> on
> >> generating
> >> > > > plugins
> >> > > > > for
> >> > > > > doing Apache releases. Instead of the awkward
> hacks
> >> that made
> >> > > up
> >> > > > the
> >> > > > > old
> >> > > > > poms (literally comments that said, "this is a
> hack,
> >> there's
> >> > > > supposedly
> >> > > > > something better coming from ..." dated like
> 2012),
> >> we would
> >> > > > like to
> >> > > > > do it
> >> > > > > the "right way" and incorporate the appropriate
> >> plugins.
> >> > > > >
> >> > > > > Refactoring to SBT was _one_ proposed solution.
> We're
> >> also OK
> >> > > > > continuing to
> >> > > > > use Maven, and I agree with what you said about
> the
> >> cross
> >> > > > compiling. We
> >> > > > > actually have a script that just changes the
> scala
> >> version.
> >> > We
> >> > > > tried
> >> > > > > using
> >> > > > > the classifiers but there were issues in SBT,
> but the
> >> way
> >> > > you're
> >> > > > > proposing
> >> > > > > sounds a lot more pro than the route we were
> trying
> >> for.
> >> > > > >
> >> > > > > That said- we'd be OK just releasing one
> scala/spark
> >> version
> >> > at
> >> > > > a time.
> >> > > > > But getting the convenience binaries to
> >> release/publish would
> >> > > be
> >> > > > a
> >> > > > > major
> >> > > > > first step.
> >> > > > >
> >> > > > > Also, we really appreciate the help,
> >> > > > >
> >> > > > > tg
> >> > > > >
> >> > > > >
> >> > > > > [1]
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Thu, Apr 16, 2020 at 4:50 AM Christofer Dutz
> <
> >> > > > > christofer.d...@c-ware.de>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Stevo,
> >> > > > > >
> >> > > > > > so let me summarize what I understood:
> >> > > > > >
> >> > > > > > - There are some modules in mahout that are
> built
> >> with
> >> > Scala,
> >> > > > some
> >> > > > > with
> >> > > > > > java and some with both (At least that's what
> I see
> >> when
> >> > > > checking
> >> > > > > out the
> >> > > > > > project)
> >> > > > > > - The current build uses Scala 2.11 to build
> the
> >> Scala
> >> > code.
> >> > > > > > - The resulting libraries are only compatible
> with
> >> Scala
> >> > 2.11
> >> > > > > >
> >> > > > > > Now you want to also publish versions
> compatible
> >> with Scala
> >> > > > 2.12?
> >> > > > > >
> >> > > > > > If that's the case I think Maven could easily
> add
> >> multiple
> >> > > > executions
> >> > > > > > where each compile compiles to different
> output
> >> > directories:
> >> > > > > > - Java --> target/classes
> >> > > > > > - Scala 2.11 --> target/classes-2.11
> >> > > > > > - Scala 2.12 --> target/classes-2.12
> >> > > > > >
> >> > > > > > Then the packaging would also need a second
> >> execution ...
> >> > > each
> >> > > > of the
> >> > > > > > executions bundling the classes and the
> >> corresponding scala
> >> > > > output.
> >> > > > > > Ideally I would probably use maven
> classifiers to
> >> > distinguish
> >> > > > the
> >> > > > > > artifacts.
> >> > > > > >
> >> > > > > > <dependency>
> >> > > > > > <groupId>org.apache.mahout</groupId>
> >> > > > > > <artifactId>mahout-spark</artifactId>
> >> > > > > > <version>14.1-SNAPSHOT</version>
> >> > > > > > <classifier>2.11</classifier>
> >> > > > > > </dependency>
> >> > > > > >
> >> > > > > > Then it should all work in a normal maven
> build. In
> >> the
> >> > > > > distributions you
> >> > > > > > could also filter the versions according to
> their
> >> > > classifiers.
> >> > > > > >
> >> > > > > > So if this is the case, I could help you with
> this.
> >> > > > > >
> >> > > > > > Chris
> >> > > > > >
> >> > > > > >
> >> > > > > > Am 16.04.20, 09:39 schrieb "Stevo Slavić" <
> >> > ssla...@gmail.com
> >> > > >:
> >> > > > > >
> >> > > > > > Disclaimer: I'm not active Mahout
> maintainer
> >> for quite
> >> > a
> >> > > > while,
> >> > > > > have
> >> > > > > > some
> >> > > > > > historical perspective, take it with a
> grain of
> >> salt,
> >> > > > could be
> >> > > > > I'm
> >> > > > > > missing
> >> > > > > > the whole point you were approached for
> by a
> >> wide
> >> > margin
> >> > > of
> >> > > > > error.
> >> > > > > >
> >> > > > > > At a point Mahout, some of its modules,
> have
> >> turned
> >> > into
> >> > > a
> >> > > > scala
> >> > > > > > library, and there was need to cross
> publish
> >> those
> >> > > modules,
> >> > > > > across
> >> > > > > > different scala versions. Back than Maven
> scala
> >> plugin
> >> > > > didn't
> >> > > > > support
> >> > > > > > cross
> >> > > > > > publishing, it doesn't fit well with
> Maven's
> >> build
> >> > > > lifecycle
> >> > > > > concept
> >> > > > > > (multiple compile phases - one for each
> scala
> >> version,
> >> > > and
> >> > > > what
> >> > > > > not
> >> > > > > > would
> >> > > > > > be needed). Switching to sbt could have
> solved
> >> the
> >> > > problem.
> >> > > > > Switch was
> >> > > > > > deemed to be too big task, even though
> ages
> >> have been
> >> > > > spent on
> >> > > > > trying
> >> > > > > > to
> >> > > > > > apply Maven (profiles) + bash scripts and
> what
> >> not to
> >> > > > solve the
> >> > > > > > problem.
> >> > > > > > Trying to apply same approach over and
> over
> >> again and
> >> > > > expecting
> >> > > > > > different
> >> > > > > > results is not smart, no expert can help
> there.
> >> Mahout
> >> > > > > maintainers and
> >> > > > > > contributors, should consider alternative
> >> approach, one
> >> > > of
> >> > > > them
> >> > > > > being
> >> > > > > > switching to sbt - it's scala native,
> supports
> >> scala
> >> > > cross
> >> > > > > publishing,
> >> > > > > > supports publishing Maven compatible
> release
> >> metadata
> >> > and
> >> > > > > binaries.
> >> > > > > >
> >> > > > > > Kind regards,
> >> > > > > > Stevo Slavic.
> >> > > > > >
> >> > > > > > On Thu, Apr 16, 2020 at 9:15 AM
> Christofer Dutz
> >> <
> >> > > > > > christofer.d...@c-ware.de>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi folks,
> >> > > > > > >
> >> > > > > > > my name is Chris and I’m involved in
> quite a
> >> lot of
> >> > > > Apache
> >> > > > > projects.
> >> > > > > > > Justin approached me this morning,
> asking me
> >> if I
> >> > could
> >> > > > > perhaps help
> >> > > > > > you.
> >> > > > > > > He told me you were having trouble with
> doing
> >> Maven
> >> > > > releases.
> >> > > > > > >
> >> > > > > > > As Maven releases are my specialty,
> could you
> >> please
> >> > > > summarize
> >> > > > > the
> >> > > > > > issues
> >> > > > > > > you are having?
> >> > > > > > >
> >> > > > > > > Chris
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
>
>
